CN108446707B - Remote sensing image aircraft detection method based on key point screening and DPM confirmation - Google Patents

Abstract

The embodiment of the present invention provides a remote sensing image aircraft detection method based on key point screening and DPM confirmation. The method includes: acquiring a partial image in the airport image; the partial image is obtained according to the local feature information of the target to be detected; The processing result of the processed partial image is determined as the preliminary screening result of the target to be detected; and based on the preliminary screening result and the preset model established in advance, the area where the target to be detected is determined; and the area is Classify, obtain overlapping candidate areas of the same classification, and determine the target to be detected based on preset rules and the candidate areas. The method provided by the embodiment of the present invention can improve the understanding of many influencing factors in the airport by first determining the area where the target to be detected is located, then classifying the area, and obtaining overlapping candidate areas of the same classification, and then determining the target to be detected. The adaptability and detection accuracy of the target to be detected.

Description

Remote sensing image aircraft detection method based on key point screening and DPM confirmation

technical field

Embodiments of the present invention relate to the technical field of remote sensing image aircraft detection, and in particular to a remote sensing image aircraft detection method based on key point screening and DPM confirmation.

Background technique

At present, there are many methods to detect objects to be detected (such as airplanes) based on optical remote sensing images in airports. Due to the large differences in the dimensions of airplanes, the adaptability of existing algorithms is limited. In addition, the airport remote sensing image scene is complex, and there are building disturbances similar to the characteristics of the aircraft, which are prone to generate a large number of false alarms and are difficult to detect.

Some existing technologies use the description of the local characteristics of the aircraft to detect the aircraft, but the expression of the overall characteristics of the aircraft is poor, and it is only suitable for aircraft target detection in simple local scenes, which may occur in the complex and large field of view of the airport. Lots of false alarms. Some existing technologies separate the target by filtering and other methods on the basis of the surface features of the target, and then add subsequent identification processing to achieve detection, but such detection methods are not suitable for scenes with complex backgrounds and densely mixed building targets. There are also existing technologies that use template matching technology for aircraft detection, but the common template form is relatively fixed, with poor flexibility and poor compatibility; optical remote sensing image imaging is easily affected by weather factors such as load platform and light, which makes the aircraft in the remote sensing image. There is a certain deformation and shadows appear on the edge of the fuselage. The ordinary template matching technology has poor adaptability in the actual scene; and the size of the aircraft is often different, and the aircraft of different sizes need to establish corresponding templates. The level of complexity is greatly increased.

Therefore, how to avoid the above-mentioned defects, improve the adaptability to the many influencing factors in the airport and the detection accuracy of the target to be detected, has become an urgent problem to be solved.

SUMMARY OF THE INVENTION

In view of the problems existing in the prior art, the embodiments of the present invention provide a remote sensing image aircraft detection method based on key point screening and DPM confirmation.

An embodiment of the present invention provides a remote sensing image aircraft detection method based on key point screening and DPM confirmation, the method comprising:

obtaining a partial image in the airport image; the partial image is obtained according to the partial feature information of the target to be detected;

Determining the processing result of the preprocessed partial image as the initial screening result of the target to be detected; and determining the area where the target to be detected is located according to the initial screening result and the pre-established preset model;

The regions are classified, and candidate regions with overlapping parts of the same classification are obtained, and the to-be-detected target is determined according to preset rules and the candidate regions.

The remote sensing image aircraft detection method based on key point screening and DPM confirmation provided by the embodiment of the present invention firstly determines the region where the target to be detected is located, then classifies the region, obtains candidate regions with overlapping parts of the same classification, and then determines The target to be detected can improve the adaptability to many influencing factors in the airport and the detection accuracy of the target to be detected.

Description of drawings

In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the accompanying drawings that need to be used in the description of the embodiments or the prior art. Obviously, the accompanying drawings in the following description These are some embodiments of the present invention. For those of ordinary skill in the art, other drawings can also be obtained according to these drawings without creative efforts.

1 is a schematic flowchart of a remote sensing image aircraft detection method based on key point screening and DPM confirmation according to an embodiment of the present invention;

2 is an output result diagram of an aircraft DPM model according to an embodiment of the present invention;

3 is an image diagram of an airport before the target to be detected is determined according to an embodiment of the present invention;

4 is an image diagram of an airport after the target to be detected is determined according to an embodiment of the present invention;

FIG. 5 is an overall flow chart of a remote sensing image aircraft detection method based on key point screening and DPM confirmation according to an embodiment of the present invention.

Detailed ways

In order to make the purposes, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments These are some embodiments of the present invention, but not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative efforts shall fall within the protection scope of the present invention.

1 is a schematic flowchart of a remote sensing image aircraft detection method based on key point screening and DPM confirmation according to an embodiment of the present invention. As shown in FIG. 1 , a remote sensing image aircraft detection method based on key point screening and DPM confirmation provided by an embodiment of the present invention , including the following steps:

S1: Acquire a partial image in the airport image; the partial image is obtained according to the partial feature information of the target to be detected.

Specifically, the device obtains a partial image in the airport image; the partial image is obtained according to the partial feature information of the target to be detected. DPM (Deformable Part Model), as its name suggests, is a component-based detection algorithm. Keypoints can be understood as local features of the target to be detected. The target to be detected can be an airplane staying at the airport, and the local feature information can be salient points, that is, corner points, edge points, bright spots in dark areas, dark spots in bright areas and other rich local information.

S2: Determine the processing result of the preprocessed partial image as the initial screening result of the target to be detected; and determine the region where the target to be detected is located according to the initial screening result and a pre-established preset model.

Specifically, the device determines the processing result of the preprocessed partial image as the initial screening result of the target to be detected; and determines the area where the target to be detected is located according to the initial screening result and the pre-established preset model .

Specifically, it can be achieved by the following methods:

1. Suspect area screening based on key point density

1) Key point extraction

Since most of the areas in the entire airport map (corresponding to the airport image) do not contain the target to be detected, in order to reduce the calculation amount of the entire field of view detection, the suspected area is first screened. The SIFT (Scale-invariant feature transform) algorithm can be used to extract the prominent points in the airport image, that is, the corner points, edge points, bright spots in dark areas, dark spots in bright areas and other local images with rich local information. The Gaussian difference operator is used to calculate the local extreme points in the scale space, and the low contrast and edge response removal are performed on these local extreme points to obtain precisely positioned feature points.

2) Suspected block screening based on the density of block key points

The window size in the 1m resolution visible light remote sensing image after key point extraction is set to 256*256, and overlapping sliding windows are performed with a step size of 128, and the number of SIFT key points in each window is accumulated and counted. At this image resolution, usually the threshold of the number of key points in the information-rich candidate area of the airport area can be set to 25, the images in the window with the number of key points greater than the threshold are retained, and the remaining part of the image is directly set to zero, which can achieve the suspected The purpose of district screening (that is, to obtain preliminary screening results).

2. DPM full-size suspected aircraft target framing based on dual resolution

1) Detection and framing of suspected large aircraft targets based on DPM of original resolution images

a. Build the HOG feature pyramid

Since the DPM model has been determined during training, in order to detect objects of different sizes in the image, multi-scale analysis is required, and a feature pyramid needs to be constructed. The directional gradient histogram of the image is calculated by the HOG algorithm to construct the feature pyramid of the L layer.

λ is the sampling specification, that is, the number of layers that need to go down in order to obtain twice the resolution of a certain layer in the pyramid is λ. And the top layer of the pyramid is the HOG feature of the image at the original resolution.

b. Calculate the model response score

Assuming that the model obtained in the training phase has n parts, it can be defined as a (n+2) tuple (F ₀ , P ₁ , . . . , P _n , b). Where F ₀ is the root filter, Pi is the _ith component filter, and b is the offset. The response score of the filter on the lth layer of the HOG feature pyramid is:

p _i =( _xi ,y _i ,li _i )

Among them, H is the feature image pyramid established in the previous step; _{pi represents the point at the level l i of} the feature image pyramid at (x _i , y _i ); φ represents the feature vector of the point in H. When i=0, R represents the response score of the root filter in layer l ₀ ; when i>0, it represents the score response of the ith component filter in layer l _i . Since the resolution of the component filter is twice that of the root filter, li = 1 _{0 -λ} , that is, the resolution of the _li layer in the pyramid is twice that of the 1 ₀ layer.

According to the filter score calculated above, the high-scoring area is expanded while considering the deformation cost, and the component model is adjusted to find a combination at the position of the ₁₀ -λ layer of the pyramid

Among them, (dx, dy) represents the displacement of the component position relative to the anchor point (the ideal location of the component filter); d _i represents the offset vector; φ _d represents the cost weight of the offset.

c. Calculate the composite score

According to the given root model position p ₀ =(x ₀ , y ₀ , l ₀ ), while considering the cost of deformation, the high-scoring area is expanded, and the component model is adjusted to find the component model at the position of the l ₀ -λ layer of the pyramid The combination of positions (p ₁ , p ₂ , . . . , p _n ) maximizes the individual component model response scores.

The root filter response score and the component filter response scores after each extension and subsampling are added together, and finally the offset of the component model relative to the root model is added to obtain the comprehensive score of the l layer:

Among them, v _i represents the relative position of the anchor point of a component filter and the root filter; the double factor of (x ₀ , y ₀ ) is to unify the resolution to the feature pyramid layer where the component model is located; b represents the use of The offset of the component filter to it. From this, the score of each layer in the pyramid can be obtained, and the effective score can be selected by the method of non-maximum value suppression, and mapped back to the image, the rectangular frame that determines the target position can be obtained, and the coordinate information of the suspected frame in the image can be stored. is the area where the target to be detected is located.

2) Detection and framing of suspected small aircraft targets based on DPM of interpolated resolution images

Due to the large difference in the size of the aircraft in the image to be detected, only by constructing the image feature pyramid to detect at different resolution layers will cause missed detection of small aircraft, so the image is detected based on the interpolation resolution. Interpolate and enlarge the image of the suspected area extracted in step 1 to twice the size of the original image, and repeat all operations in step 1) for the image after the interpolation and enlargement, so as to detect the small plane in the image.

Fig. 2 is the output result diagram of the aircraft DPM model according to the embodiment of the present invention, wherein (a) is the output result diagram of the root model; (b) is the output result diagram of the component model; (c) is the output result diagram of the expanded component model.

It should be noted that the above-mentioned pre-established preset models need to be pre-trained before use, and the methods can be as follows:

Build a training database

Define c as the training target category, P as the positive sample set, N as the negative sample set, and the positive and negative sample set gives the training samples of the target category c. P is the positive sample database of artificially marked boxes, which is a set of two-tuples (I, B). where I is the image and B is the labeled box of the c-type object in image I. Due to the needs of the actual scene, the c-type target is selected as the aircraft, and N is the set of negative sample airport runways, airport bridges, and building corners. Manually frame the target for all sample images, and store the corresponding information data in the xml file to establish an aircraft model training database.

aircraft model training

Initialize the root filter

The trained hybrid model contains m components, and the positive sample database P of the artificially labeled boxes is sorted by aspect ratio and classified into m groups, denoted as P1,...,Pm. Choose a value greater than 80% of the rectangular box area as the root model area. Use the standard SVM algorithm to train m corresponding root filters, denoted as F1,...,Fm.

update root filter

The m root filters are combined to perform iterative optimization through the coordinate descent training algorithm.

Initialize part filter

Set the number of components in each component model to 8, place the components on both sides of the root filter in the form of mid-axis symmetry, place the first component in the highest energy region of the root filter, and set the energy in this region to zero. Repeat this operation until the part is placed. where the component filter has twice the resolution of the root filter. The setting of the number of 8 components is more precise than the default 6 components, which is more suitable for target detection in complex scenes, and will not be too fine to complicate the detection process.

Update an existing model

Use the existing model to detect the positive sample marked box in the training data set, and put the position with the highest score as the positive sample of this sample box and put it into the buffer. Similarly, the existing model is used to detect negative samples, and the position with the highest score is put into the buffer, up to the maximum limit of the file, and a new model is trained with the samples of the buffer. Iteratively update the model 10 times according to the above strategy to obtain the final model parameters.

S3: Classify the regions, obtain candidate regions with overlapping parts of the same classification, and determine the to-be-detected target according to preset rules and the candidate regions.

Specifically, the device classifies the regions, obtains candidate regions with overlapping parts of the same classification, and determines the to-be-detected target according to preset rules and the candidate regions. The area can include the vertex position information of the frame, that is, the above-mentioned suspected frame coordinate information, and the classification can include:

According to the vertex position information, obtain the center point position information of the frame; traverse all the center point position information, and calculate the Euclidean distance determined according to each two center point position information; The two regions are divided into regions of the same classification.

The specific description is as follows: the coordinate information of the suspected frame may be the coordinate position of the upper left corner of the frame body and the coordinate position of the lower right corner of the frame body, so as to calculate the position coordinates of the center point of each frame body (corresponding to the position information of the center point). Based on the coordinates of the center point of the first suspected box, compare it with the coordinates of the center points of other suspected boxes one by one, and calculate the Euclidean distance, which will be greater than the threshold (corresponding to the preset threshold, and the preset threshold can refer to the width of the aircraft model size) for autonomous setting) are divided into one category. Repeat this operation for the remaining unclassified suspected boxes until all the suspected boxes are classified and collected.

The step of determining the target area where the target to be detected is located may include:

Accumulate the preset pixel values of each area with overlapping parts of the same classification; extract candidate areas that are greater than or equal to the preset accumulated pixel value; The comparison result of the corresponding minimum circumscribed rectangle determines the target area where the target to be detected is located.

Do the following for each of the above suspected boxes:

The pixel value of each suspected box (corresponding to each area with overlapping parts of the same category) is set to 1 (that is, the preset pixel value is selected as 1), and the overlapping parts are accumulated. Assuming that the number of suspected boxes in the stack is n, extract the part whose pixel value is greater than or equal to n/2 after superimposition, which is A ₁ ,...,A _i , that is, the number of candidate regions is i, and find the extraction region A The smallest circumscribed rectangle B _i of _i , compare B _i with the area of each suspected frame (corresponding to the area of each candidate area) that constitutes A _i , if the area of B _i is greater than the bisection of the area of each suspected frame that constitutes A _i One is to keep B _i and remove all the suspected boxes that constitute A _i ; otherwise, remove B _i and keep all the suspected boxes that constitute A _i .

After this operation is performed for each type of frame, the target area determined by the reserved frame is used as the target to be detected, and its coordinates (position information corresponding to the target area) are saved and displayed in the airport image as the detection result.

The remote sensing image aircraft detection method based on key point screening and DPM confirmation provided by the embodiment of the present invention firstly determines the region where the target to be detected is located, then classifies the region, obtains candidate regions with overlapping parts of the same classification, and then determines The target to be detected can improve the adaptability to many influencing factors in the airport and the detection accuracy of the target to be detected.

FIG. 3 is an image of the airport before the target to be detected is determined according to the embodiment of the present invention; FIG. 4 is an image of the airport after the target to be detected is determined according to the embodiment of the present invention. As shown in FIGS. 3 and 4, it can be seen that the implementation of the present invention This example can accurately detect the target to be detected.

FIG. 5 is an overall flow chart of a remote sensing image aircraft detection method based on key point screening and DPM confirmation according to an embodiment of the present invention, and the specific description of FIG.

On the basis of the above embodiment, the area includes the vertex position information of the frame; accordingly, the classification of the area includes:

According to the vertex position information, obtain the center point position information of the frame.

Specifically, the device obtains the center point position information of the frame body according to the vertex position information. Reference may be made to the above-mentioned embodiments, and details are not repeated here.

Traverse all the center point position information, and calculate the Euclidean distance determined according to the position information of each two center points.

Specifically, the device traverses all the center point position information, and calculates the Euclidean distance determined according to each two center point position information. Reference may be made to the above-mentioned embodiments, and details are not repeated here.

The two regions corresponding to the Euclidean distance greater than the preset threshold are divided into regions of the same classification.

Specifically, the apparatus divides the two regions corresponding to the Euclidean distance greater than the preset threshold into regions of the same classification. Reference may be made to the above-mentioned embodiments, and details are not repeated here.

The remote sensing image aircraft detection method based on key point screening and DPM confirmation provided by the embodiment of the present invention can ensure the normal operation of the method by first classifying the area where the target to be detected is located.

On the basis of the above-mentioned embodiment, the overlapping area of the same category is obtained, and the target area where the target to be detected is located is determined according to a preset rule, including:

Accumulates the preset pixel values of each overlapping region of the same classification.

Specifically, the apparatus accumulates the preset pixel values of each same-classified area with overlapping parts. Reference may be made to the above-mentioned embodiments, and details are not repeated here.

Extract the candidate area that is greater than or equal to the preset accumulated pixel value.

Specifically, the apparatus extracts a candidate area that is greater than or equal to a preset accumulated pixel value. Reference may be made to the above-mentioned embodiments, and details are not repeated here.

The minimum circumscribed rectangle of each candidate area is obtained, and the target area where the target to be detected is located is determined according to the comparison result between the area of each candidate area and the corresponding minimum circumscribed rectangle.

Specifically, the device obtains the minimum circumscribed rectangle of each candidate region, and determines the target region where the target to be detected is located according to the comparison result between the area of each candidate region and the corresponding minimum circumscribed rectangle. Reference may be made to the above-mentioned embodiments, and details are not repeated here.

The remote sensing image aircraft detection method based on key point screening and DPM confirmation provided by the embodiment of the present invention determines the target area where the target to be detected is located based on the overlapping confidence, which can further improve the adaptability to numerous influencing factors in the airport, and the sensitivity of the target to be detected. Detection accuracy.

On the basis of the above-mentioned embodiment, the candidate regions with overlapping parts of the same category are obtained, and the target to be detected is determined according to preset rules and the candidate regions, including:

If the area of the candidate area is less than or equal to twice the area of the corresponding minimum circumscribed rectangle, the target area determined by the minimum circumscribed rectangle is used as the target to be detected.

Specifically, if the device determines that the area of the learned candidate region is less than or equal to twice the area of the corresponding minimum circumscribed rectangle, the target area determined by the minimum circumscribed rectangle is used as the target to be detected. Reference may be made to the above-mentioned embodiments, and details are not repeated here.

If the area of the candidate area is greater than twice the area of the corresponding minimum circumscribed rectangle, the target area determined by the candidate area is used as the target to be detected.

Specifically, if the device determines that the area of the learned candidate area is greater than twice the area of the corresponding minimum circumscribed rectangle, the target area determined by the candidate area is used as the target to be detected. Reference may be made to the above-mentioned embodiments, and details are not repeated here.

The remote sensing image aircraft detection method based on key point screening and DPM confirmation provided by the embodiment of the present invention can further improve the detection accuracy of the target to be detected by comparing the area of the candidate area and the corresponding minimum circumscribed rectangle area.

On the basis of the above embodiment, the method further includes:

Acquire the number n of overlapping regions of each same category, and the preset pixel value is 1; correspondingly, the preset accumulated pixel value is n/2.

Specifically, the device obtains the number n of the overlapping regions of each same category, and the preset pixel value is 1; correspondingly, the preset accumulated pixel value is n/2. Reference may be made to the above-mentioned embodiments, and details are not repeated here.

The remote sensing image aircraft detection method based on key point screening and DPM confirmation provided by the embodiment of the present invention can control the detection accuracy of the target to be detected more autonomously and flexibly by setting reasonable values for the preset pixel value and the preset accumulated pixel value.

On the basis of the above embodiment, after the step of determining the target to be detected, the method further includes:

Acquire location information of the target area, and display the location information in the airport image.

Specifically, the device acquires the location information of the target area, and displays the location information in the airport image. Reference may be made to the above-mentioned embodiments, and details are not repeated here.

The remote sensing image aircraft detection method based on key point screening and DPM confirmation provided by the embodiment of the present invention displays the location information of the target area in the airport image, which can facilitate the viewing of the location information.

On the basis of the above-mentioned embodiment, the target to be detected is an airplane staying at an airport.

Specifically, the target to be detected in the device is an airplane staying at an airport.

The remote sensing image aircraft detection method based on key point screening and DPM confirmation provided by the embodiment of the present invention can further improve the adaptability to many influencing factors in the airport and the detection of aircraft by selecting the target to be detected as the aircraft staying at the airport. precision.

Those of ordinary skill in the art can understand that all or part of the steps of implementing the above method embodiments can be completed by program instructions related to hardware, the aforementioned program can be stored in a computer-readable storage medium, and when the program is executed, execute It includes the steps of the above method embodiments; and the aforementioned storage medium includes: ROM, RAM, magnetic disk or optical disk and other media that can store program codes.

The above-described electronic equipment and other embodiments are only illustrative, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, It can be located in one place, or it can be distributed over multiple network elements. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution in this embodiment. Those of ordinary skill in the art can understand and implement it without creative effort.

From the description of the above embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus a necessary general hardware platform, and certainly can also be implemented by hardware. Based on this understanding, the above-mentioned technical solutions can be embodied in the form of software products in essence or the parts that make contributions to the prior art, and the computer software products can be stored in computer-readable storage media, such as ROM/RAM, magnetic A disc, an optical disc, etc., includes several instructions for causing a computer device (which may be a personal computer, a server, or a network device, etc.) to perform the methods described in various embodiments or some parts of the embodiments.

Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the embodiments of the present invention, but not to limit them; although the embodiments of the present invention have been described in detail with reference to the foregoing embodiments, ordinary The skilled person should understand that it is still possible to modify the technical solutions described in the foregoing embodiments, or to perform equivalent replacements on some or all of the technical features; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the present invention. The scope of the technical solutions of the embodiments of each embodiment.

Claims (4)

1. A remote sensing image airplane detection method based on key point screening and DPM confirmation is characterized by comprising the following steps:

acquiring a local image in an airport image; the local image is obtained according to local characteristic information of the target to be detected; dividing the target to be detected into a first type target to be detected and a second type target to be detected according to the size of the target to be detected;

determining the processing result of the preprocessed local image as the primary screening result of the target to be detected; determining the area of the target to be detected according to the primary screening result and a preset model established in advance;

classifying the regions, acquiring candidate regions with the same classification and overlapping parts, and determining the target to be detected according to a preset rule and the candidate regions;

acquiring the number N of each same classified region with an overlapping part, wherein the preset pixel value is 1; correspondingly, the preset accumulated pixel value is N/2;

the acquiring of the same classified areas with the overlapped parts and determining the target area where the target to be detected is located according to a preset rule comprise:

accumulating the preset pixel values of each same classified area with the overlapped part;

extracting a candidate area which is greater than or equal to a preset accumulated pixel value;

acquiring a minimum circumscribed rectangle of each candidate region, and determining a target region where the target to be detected is located according to a comparison result of the area of each candidate region and the corresponding minimum circumscribed rectangle;

if the area of the candidate region is less than or equal to twice of the area of the corresponding minimum circumscribed rectangle, taking the target region determined by the minimum circumscribed rectangle as the target to be detected;

if the area of the candidate region is larger than twice of the area of the corresponding minimum circumscribed rectangle, taking the target region determined by the candidate region as the target to be detected;

the determination of the primary screening result comprises the following steps:

framing a first type of target to be detected based on DPM of an original resolution image;

and framing the second class of targets to be detected of the DPM based on the interpolation resolution image.

2. The method of claim 1, wherein the region includes vertex position information of a frame; accordingly, the classifying the region includes:

acquiring the position information of the central point of the frame body according to the vertex position information;

traversing all the central point position information, and calculating the Euclidean distance determined according to every two central point position information;

and dividing two regions corresponding to the Euclidean distance larger than a preset threshold value into regions of the same classification.

3. The method according to any one of claims 1 to 2, wherein after the step of determining the object to be detected, the method further comprises:

and acquiring the position information of the target area, and displaying the position information in the airport image.

4. The method of claim 1, wherein the object to be detected is an aircraft parked at an airport.

Images