WO2019201029A1 - Candidate box update method and apparatus - Google Patents

Abstract

Provided in the present application are a candidate box update method and apparatus, relating to the field of computers. The candidate box update method comprises: on the basis of position information and a position vector of a first candidate box, obtaining the overlap degree of the first candidate box and a plurality of second candidate boxes, the first candidate box being a candidate box having the highest score of a classifier in a candidate box set, the second candidate boxes being the candidate boxes in the candidate box set other than the first candidate box, and the position vector comprising the position information of the plurality of second candidate boxes; and comparing the overlap degree of the first candidate box and the plurality of second candidate boxes with an overlap degree threshold to obtain an updated candidate box set. Using the technical solution of the present application can improve the efficiency of updating the candidate boxes and thereby increase the operating efficiency of image recognition and detection.

Description

Alternate frame updating method and device

This application claims the priority of the Chinese Patent Application filed on April 19, 2018, the Chinese National Intellectual Property Office, the application number is 201810355286.3, and the application name is "Alternative Box Update Method and Device", the entire contents of which are incorporated herein by reference. In the application.

Technical field

The present application relates to the field of computers, and in particular, to a method and apparatus for updating an alternative frame.

Background technique

Non-Maximum Suppression (NMS) is widely used in computer vision algorithms such as face recognition, edge detection or target detection. The classifier is an important part of the NMS computer vision algorithm, which can be used to detect objects in an image, such as whether it is a face or the like. For an object in the image, the classifier generates a plurality of candidate boxes. The classifier calculates the classifier score for each candidate box. In order to accurately detect or identify an object, only one optimal candidate frame is reserved for one object, and the content of the optimal candidate frame is used as an object to be identified or detected.

Specifically, the candidate box with the highest score of the classifier in all the candidate frames in the image is selected, and the overlapping area of the candidate frame with the highest score of the other candidate frame and the classifier is calculated by traversing, and the parts of the other candidate frames are deleted according to the overlapping area. Optional box. The candidate box with the highest score of the classifier is again selected in the other candidate boxes that have not been deleted, and the traversal calculation of the overlap area is performed again until all the alternative boxes except the optimal candidate frame are deleted. In a more complex image, the number of candidate frames is very large, resulting in an increase in the amount of traversal calculation, which in turn increases the computational complexity of the NMS algorithm, reducing the efficiency of image recognition and detection.

Summary of the invention

The present application provides a method and apparatus for updating an alternative frame, which can improve the efficiency of updating an optional frame, thereby improving the efficiency of image recognition and detection.

In a first aspect, the present application provides an alternative frame update method, including: obtaining, according to location information and a location vector of a first candidate frame, an overlap between a first candidate frame and a plurality of second candidate frames, An alternative box is an alternative box with the highest score of the classifier in the set of candidate boxes, and the second candidate box is an alternative box in the set of candidate boxes except the first candidate box, the position vector includes a plurality of second Position information of the candidate box; comparing the overlap degree and the overlap degree threshold of the first candidate box with the plurality of second candidate boxes to obtain an updated set of candidate boxes.

In the embodiment of the present application, by using the location information and the location vector of the first candidate frame, the degree of overlap between the first candidate frame and the plurality of second candidate frames can be obtained by one calculation. The calculated overlap and overlap degree thresholds of the first candidate box and the second candidate box may be used to obtain an candidate box in the updated candidate box set. In the process of updating the candidate box, the position vector can greatly reduce the time required to update the candidate box in the candidate box set, and the efficiency of updating the candidate box is improved, thereby improving the use of the candidate box in the present application. The efficiency of the method's NMS method, in turn, improves the efficiency of image recognition and detection.

In an implementation manner of the first aspect, the first candidate box is compared with the overlap degree and the overlap threshold of the plurality of second candidate boxes to obtain an updated set of candidate boxes, including: When the overlap degree of the second candidate box and the first candidate box in the second candidate box is less than or equal to the overlap degree threshold, determining that the second candidate box remains in the updated candidate box set; When the degree of overlap of the second candidate box and the first candidate box in the candidate box is greater than the overlap threshold, it is determined that the second candidate box is not retained in the updated set of candidate boxes.

Maintaining a second candidate frame with a degree of overlap with the first candidate frame that is less than or equal to the overlap degree threshold as an alternative frame in the updated candidate set of frames, and deleting the overlap with the first candidate frame is greater than the overlap threshold The second alternative box. Thereby, the number of candidate blocks participating in the next candidate box set update is reduced, and the amount of operation and occupied resources required for the next candidate box set update are correspondingly reduced, thereby further improving the efficiency of updating the candidate box set.

In an implementation manner of the first aspect, the candidate frame updating method further includes: using the location information of the first candidate frame in the updated candidate frame set and the updated location vector to obtain the updated candidate. The degree of overlap of the first candidate box in the set of boxes with the plurality of second candidate boxes in the updated set of candidate boxes, the updated position vector comprising a plurality of second alternatives in the updated set of candidate blocks Position information of the box; comparing the overlap between the first candidate box in the updated set of candidate boxes with the overlap and overlap thresholds of the plurality of second candidate boxes in the updated set of candidate boxes to be updated again The subsequent set of candidate boxes until the overlap between the first candidate box in the updated set of candidate boxes and the second candidate box in the updated set of candidate boxes is greater than the overlap threshold, stopping the candidate An update of the set of boxes; the first candidate box in the set of candidate boxes and the first candidate box in the set of candidate boxes after each update are used as target recognition frames.

In a scenario where there are multiple target objects in the image, by updating the candidate boxes in the candidate box set multiple times, multiple target recognition frames can be obtained, and each target recognition frame can identify one target object. Moreover, the degree of overlap between the first candidate frame and the second candidate frame is obtained by using the position vector for each update of the candidate frame set, thereby reducing the time taken to identify the plurality of target objects in the image, and improving the plurality of recognition images. The productivity of the target object.

In one implementation of the first aspect, the set of candidate boxes includes a plurality of candidate boxes for identifying faces in the image.

In a second aspect, the present application provides an alternative frame updating apparatus, including: an overlap acquiring module, configured to obtain a first candidate frame and multiple second devices according to location information and a location vector of the first candidate frame. The overlap of the marquee, the first candidate box is the candidate box with the highest score of the classifier in the set of candidate boxes, and the second candidate box is the candidate box of the set of candidate boxes except the first candidate box. The location vector includes location information of the plurality of second candidate frames, and an update module, configured to compare the overlap degree and the overlap threshold of the first candidate frame with the plurality of second candidate frames to obtain an updated candidate frame set.

In an implementation manner of the second aspect, the updating module is specifically configured to determine, when the overlap degree of the second candidate box and the first candidate box in the multiple second candidate frames is less than or equal to the overlap degree threshold The second candidate box is reserved in the updated candidate box set; when the overlap degree of the second candidate box and the first candidate box in the plurality of second candidate boxes is greater than the overlap degree threshold, determining the second standby The marquee does not remain in the updated set of alternate boxes.

In an implementation manner of the second aspect, the overlap acquiring module is further configured to obtain the updated candidate frame by using the location information of the first candidate frame in the updated candidate set of frames and the updated location vector. The degree of overlap of the first candidate box in the set with the plurality of second candidate boxes in the updated set of candidate boxes, the updated position vector including the plurality of second candidate boxes in the updated set of candidate boxes Position information; the update module is further configured to compare the overlap between the first candidate box in the updated set of candidate boxes and the overlap and overlap thresholds of the plurality of second candidate boxes in the updated set of candidate boxes To obtain the updated candidate box set again, until the overlap degree of the first candidate box in the updated candidate box set obtained by the overlap degree acquiring module and the second candidate box in the updated candidate box set All of which are greater than the overlap threshold, the update module is configured to stop updating the set of candidate boxes, and to use the first candidate box in the set of candidate boxes and the first candidate box in the set of candidate boxes after each update. , as the target recognition box.

In an implementation of the second aspect, the set of candidate boxes includes a plurality of candidate boxes for identifying faces in the image.

The second aspect and the candidate frame updating device in the implementation of the second aspect can also achieve the same technical effects as the candidate frame updating method in the above technical solution.

In a third aspect, the present application provides an alternative frame updating apparatus, including a memory, a processor, and a program stored on the memory and executable on the processor, and the processor implements an alternative as in the foregoing technical solution when executing the program. The frame update method can achieve the same technical effect as the candidate frame update method in the above technical solution.

In a fourth aspect, the present application provides a storage medium on which a program is stored, and when the program is executed by the processor, an optional frame update method in the foregoing technical solution is implemented, and an optional block update in the foregoing technical solution can be achieved. The same technical effect of the method.

DRAWINGS

1 is a flowchart of an alternate frame update method in an embodiment of the present application;

2 is a flowchart of a method for non-maximum value suppression in the embodiment of the present application;

FIG. 3 is a schematic diagram showing changes of an alternative box for two screenings in NMS calculation according to an embodiment of the present application; FIG.

4 is a schematic diagram of an alternative frame in the image to be tested in the embodiment of the present application;

FIG. 5 is a schematic structural diagram of an apparatus for updating an alternative frame according to an embodiment of the present application;

FIG. 6 is a schematic structural diagram of hardware of an alternative frame updating device in an embodiment of the present application.

detailed description

The present application provides an alternative frame updating method and apparatus, which can be applied to a scene of visual image processing. In a scene of visual image processing, it is necessary to identify a target object or track a target object in the image to be measured. To identify the target object in the image under test, a target recognition frame can be generated. The content in the target recognition frame is the target object in the image under test. In the process of identifying the target object, a plurality of identification frames may be generated for one target object, and the plurality of identification frames are candidates for the target recognition frame, hereinafter referred to as an optional frame (including the first candidate frame and the first Two alternative boxes). An optimal candidate box (ie, the first candidate box) is selected as an target candidate box in an candidate box of a target object. In some cases, the target image may include multiple target objects, and multiple rounds of candidate boxes need to be filtered to obtain an optimal candidate box corresponding to all target objects. The screening of each round of alternative boxes, except for the screening of the last round of alternate boxes, will be updated to get the set of candidate boxes that are filtered in the next round of candidate boxes. The update of the candidate boxes in the set of candidate boxes may employ the alternate box update method in this application.

In the embodiment of the present application, the candidate frame update in the embodiment of the present application may be implemented by adding a vector processing unit to the image processing device or adding a vector processing function to the existing unit in the image processing device. The candidate box update in the embodiment of the present application is specifically applicable to a method of Non-Maximum Suppression (NMS). The NMS method can be used in target object recognition for visual image processing. In one example, the image processing device may be specifically a non-maximum suppression device. The non-maximum suppression device may collect non-maximum suppression data including the classifier score of the candidate frame in the measured image and the position information of the candidate frame. The collected non-maximum suppression data can be transferred from the system double data rate (English: Double Data Rate, DDR) to the local storage space (ie, Local Memory) through integrated direct memory access (IDMA). )in. The size of the local storage space is limited, and the non-maximum suppression data in the system DDR needs to be transferred to the local storage space in batches. The unit implementing the vector processing function in the non-maximum suppression device may generate a position vector according to the vectorization instruction and the non-maximum value suppression data, and perform calculation of the position vector and other data until all the recognition target objects in the measured image are obtained. Or track the target recognition frame of the target object.

FIG. 1 is a flowchart of an alternate frame update method in an embodiment of the present application. As shown in FIG. 1, the candidate frame updating method may include step S101 and step S102.

In step S101, the degree of overlap between the first candidate frame and the plurality of second candidate frames is obtained according to the location information of the first candidate frame and the location vector.

The first candidate box is an candidate box with the highest score of the classifier in the set of candidate boxes. The second alternative box is an alternative box in the set of candidate boxes other than the first candidate box. The set of candidate boxes includes a plurality of candidate boxes. The location information can characterize the location of the candidate frame in the image being measured. For example, if the candidate frame is a rectangular frame, the position information may be the coordinates of the four vertices of the rectangular frame in the image to be measured, or the position information may be a pair of vertices on the diagonal line of the rectangular frame in the image to be tested. The coordinates in . Based on the location information of the two candidate boxes, the degree of overlap of the two candidate boxes can be calculated.

The location vector includes location information of a plurality of second candidate frames. That is to say, the position vector includes a plurality of elements, and one element is position information of a second alternative frame. Using the position information of the first candidate frame and the position vector, the degree of overlap of the first candidate frame and the plurality of second candidate frames can be calculated at one time.

In step S102, the first candidate box is compared with the overlap degree and the overlap degree threshold of the plurality of second candidate boxes to obtain an updated set of candidate boxes.

It should be noted that the degree of overlap may be an Intersection Over Union (IOU) or a ratio of the area of the overlap to the area of the candidate frame, which is not limited herein. Correspondingly, the overlap degree threshold may be a overlap area threshold, or may be a ratio threshold of the overlap area to the candidate frame area, and is not limited herein.

Wherein, the second candidate box with the degree of overlap with the first candidate box being less than or equal to the overlap degree threshold is reserved as an alternative box in the updated set of candidate boxes. The second candidate box having a degree of overlap with the first candidate box that is greater than the overlap degree threshold is not retained in the updated set of candidate boxes. For example, the second candidate box with the overlap degree of the first candidate frame greater than the overlap degree threshold is deleted. That is, the second candidate box with the degree of overlap with the first candidate box being greater than the overlap threshold does not participate in the next round of candidate box screening. The calculation amount required for updating the next round of the candidate box set is reduced, and the resources occupied by the calculation of the next round of the candidate box set are also reduced, and the efficiency of updating the candidate box set is improved, thereby further improving The efficiency of the NMS method.

The NMS method can be implemented by using the candidate frame update method of the embodiment of the present application. Specifically, the overlap degree calculation in the step S101 and the step S102 and the updated content of the candidate frame set can be repeated by using the updated candidate frame set until the update is performed. The degree of overlap of the first candidate block in the subsequent set of candidate blocks with the second candidate block in the updated set of candidate blocks is greater than the overlap threshold, and the updating of the set of candidate blocks may be stopped. All of the first candidate boxes obtained during the process of updating the candidate box multiple times can be used as the target recognition box.

In the embodiment of the present application, by using the location information and the location vector of the first candidate frame, the degree of overlap between the first candidate frame and the plurality of second candidate frames can be obtained by one calculation. The calculated overlap and overlap degree thresholds of the first candidate box and the second candidate box may be used to obtain an candidate box in the updated candidate box set. In the process of updating the candidate box, the location vector can greatly reduce the time required for updating the candidate box in the candidate box set, and the efficiency of updating the candidate box is improved, thereby improving the utilization of the embodiment of the present application. The efficiency of the NMS method of the marquee update method improves the efficiency of image recognition and detection.

The following is an example of identifying a target object in a measured image by using a non-maximum suppression method, and explaining the details of the non-maximum suppression method using the candidate frame update method and the candidate frame update method therein.

FIG. 2 is a flowchart of a method for non-maximum value suppression in the embodiment of the present application. As shown in FIG. 2, the method of non-maximum value suppression may include steps S201 to S205.

In step S201, in the candidate box set, the candidate box with the highest score of the classifier is selected as the first candidate box.

Wherein, the initial set of candidate boxes includes all candidate boxes on the image to be tested. The classifier calculates the classifier score for each candidate box. By inputting the measured image and the candidate box on the measured image into the classifier, the classifier score of each candidate frame on the measured image can be obtained. The classifier can gradually improve the classification and scoring standards through machine learning. The classifier score reflects the degree to which each candidate box matches the target object in the image being measured. The higher the classifier score of the candidate box, the higher the degree to which the candidate box matches the target object in the image under test.

Multiple candidate boxes may be generated for each target object in the image being measured. In order to be able to identify the first candidate box in the measured image that best matches each target object, multiple screenings are required. A first candidate box is obtained in each screening, and the first candidate box is an alternative box that best matches the target object. For example, the candidate box with the highest score of the classifier can be selected as the first candidate box.

The candidate frame updating method and the NMS method in the embodiments of the present application can be applied to scenes such as face recognition, edge detection, or target detection. For example, in a face recognition scene, the set of candidate boxes contains a plurality of candidate boxes for identifying faces in the image. The face in the image can be recognized more efficiently by using the candidate frame updating method and the NMS method in the embodiment of the present application.

In step S202, a position vector is generated using the position information of the second candidate frame.

The second candidate box includes an alternative box in the set of candidate boxes except the first candidate box. For example, there are 6 candidate boxes in the measured image, and the 6 candidate boxes are D1, D2, D3, D4, D5 and D6. Wherein, the classifier score of the candidate box D1 is 0.98, the classifier score of the candidate box D2 is 0.81, the classifier score of the candidate box D3 is 0.69, and the classifier score of the candidate box D4 is 0.48, the candidate box D5. The classifier score is 0.24 and the classifier score for candidate box D6 is 0.58. Then in the first screening, the first candidate box is the candidate box D1, and the second candidate box is the candidate box D2, the candidate box D3, the candidate box D4, the candidate box D5, and the candidate box D6.

It should be noted that the number of generated position vectors may be one or plural, and is not limited herein. Each location vector includes location information for a plurality of second candidate frames. The position information of each second candidate box can be used as an element in the position vector. The number of elements in the position vector can be preset, for example, according to a specific work scene and work requirements, and is not limited herein.

In one example, the number of second candidate boxes may be divisible by the number of elements preset by the position vector, ie, at least one position vector is generated. Then, in the candidate frame update process, the degree of overlap between the first candidate frame and all the second candidate frames is obtained according to the location information of the first candidate frame and the at least one location vector. For example, in the process of an alternate box update, the number of elements of the preset position vector is four, and the number of the second candidate frame is eight, and two position vectors can be generated.

In another example, the number of second candidate frames may not be divisible by the number of elements preset by the position vector, and based on the generation of the at least one position vector, there is also an integer single second candidate that does not form a position vector. The location information of the box. In the candidate frame update process, the degree of overlap of the first candidate frame with the second candidate frame involved in the location vector may be obtained according to the location information of the first candidate frame and the at least one location vector. In addition, the location information of the first candidate frame and the location information of the integer single second candidate frame may be used to obtain the degree of overlap between the first candidate frame and the integer single second candidate frame. For example, in one screening, there are 8 second candidate boxes, and the number of elements of the preset position vector is 3, and position information of 2 position vectors and 2 single second candidate boxes is generated.

In the above embodiment, the second candidate frame may be sorted in descending order of the classifier score, and the number of elements per preset position vector may be second selected in descending order of the classifier score. The position information of the box is used as a position vector. The remainder obtained by quoting the number of the second candidate frames and the number of elements of the preset position vector is selected according to the order of the classifier scores of the second candidate frame from low to high, and the remaining second candidate frames are selected. The position information of the remaining second candidate boxes is taken as the position information of the integer single second candidate frame. That is to say, if at least one position vector is generated, there is also position information of P second candidate frames in which the position vector is not formed, and P is a positive integer. Then, the location information of the P second candidate frames is the location information of the P second candidate frames with the lowest score of the classifier.

In step S203, vector calculation is performed on the position information and the position vector of the first candidate frame to obtain the degree of overlap between the first candidate frame and the second candidate frame.

A vectorization instruction can be generated in the case where vector calculation is required, and vector calculation is triggered according to the vectorization instruction. The vector calculation may calculate the degree of overlap of the first candidate box with the plurality of second candidate boxes in a position vector at one time. For example, let the number of second candidate boxes be 8, and the number of elements in the position vector be four. If the NMS method in the prior art is used, it is necessary to perform 8 calculations in order to obtain the overlap of the first candidate box and all the 8 second candidate boxes in one screening. With the NMS method in the embodiment of the present application, the degree of overlap between the first candidate box and all the eight second candidate boxes is obtained in one screening, and only two calculations are needed. The amount of calculation in the NMS method is greatly reduced, and the calculation time spent by the NMS method is reduced.

In step S204, if the degree of overlap between the first candidate frame and the second candidate frame is not all greater than the overlap threshold, the candidate set of frames is updated to include the overlap with the first candidate frame being less than or A second alternative box equal to the overlap threshold.

In step S205, if the degree of overlap between the first candidate box and the second candidate box is greater than the overlap degree threshold, the updating of the candidate box set is stopped.

In a scenario where multiple candidate frames are included in the image being measured, multiple screenings may be required. The first candidate box is selected in each screening, and the degree of overlap between the first candidate box and the second candidate box in the current screening is calculated. A second candidate box having a degree of overlap with the currently selected first candidate box that is less than or equal to the overlap degree threshold as an alternative box in the set of candidate boxes in the next screening. The screening is stopped until the overlap between the first candidate box and all the second candidate boxes in the current screening is greater than the overlap threshold. The first candidate box is obtained in each screening, that is, an alternative frame that best matches each of the target objects in the image to be measured is obtained.

In one example, a second candidate box with a degree of overlap with the first candidate box that is greater than the overlap threshold may be deleted in each screening. That is to say, the second candidate box whose overlap degree with the first candidate box in the current screening is greater than the overlap degree threshold is no longer involved in the calculation of the screening after the current screening. The calculation amount and the time taken for calculation are reduced, and the resources occupied by the calculation are also reduced, and the efficiency of the NMS method is further improved.

For example, an index number (ie, Index) can be set for each candidate box, and an attribute (ie, Flags) is set for each candidate box, and the attribute of the candidate box indicates the candidate box and the first candidate in the current filter. Whether the overlap of the boxes is greater than the overlap threshold. FIG. 3 is a schematic diagram showing changes of an alternative box for two screenings in NMS calculation in the embodiment of the present application. As shown in FIG. 3, an attribute of 0 indicates that the degree of overlap with the first candidate box is less than or equal to the overlap degree threshold, and an attribute of 1 indicates that the degree of overlap with the first candidate box is greater than the overlap threshold. In the m-1th screening, the attributes corresponding to the index numbers 0, 1, 2, and 3 are 0, 1, 1, and 0, respectively. In the m-1th filter, the candidate box with the attribute 1 is deleted, and the candidate box with the index numbers 0 and 3 is left in the mth filter.

The NMS method in the embodiment of the present application will be described below with two examples. FIG. 4 is a schematic diagram of an alternative frame in the image to be tested in the embodiment of the present application. As shown in FIG. 4, the test image includes a total of seven candidate boxes, which are an optional box D1, an optional box D2, an optional box D3, an optional box D4, an optional box D5, an optional box D6, and a standby device. Box D7. The classifier scores of the candidate block D1, the candidate box D2, the candidate box D3, the candidate box D4, the candidate box D5, the candidate box D6, and the candidate box D7 are 0.98, 0.81, 0.69, 0.48, 0.58, respectively. 0.24 and 0.36. The overlap threshold is the percentage of the overlap area to the area of the candidate frame, and the overlap threshold is 20%.

The first instance:

Let the number of elements in the position vector be 3, then generate 2 position vectors in the first screening. The seven candidate boxes can be arranged according to the classifier score, and the order of arrangement is the candidate box D1, the candidate box D2, the candidate box D3, the candidate box D6, the candidate box D4, the candidate box D7, and the candidate box. D5. In the first screening, the candidate box D1 with the highest score of the classifier is selected as the first candidate box, the candidate box D2, the candidate box D3, the candidate box D6, the candidate box D4, the candidate box D7, and the preparation Box D5 is the second alternative box. The first position vector includes position information of the candidate box D2, position information of the candidate box D3, and position information of the candidate box D6. The second position vector includes position information of the candidate box D4, position information of the candidate box D7, and position information of the candidate box D5. Then, the calculation needs to be performed twice, and the first calculation results the overlap degree between the first candidate box D1 and the candidate box D2, the candidate box D3 and the candidate box D6 in the first position vector, and the second calculation is obtained. The degree of overlap of each of the first candidate box D1 and the candidate box D4, the candidate box D5, and the candidate box D7 in the second position vector. The degree of overlap of the first candidate box D1 and the candidate box D2 is greater than 20%; the degree of overlap of the first candidate box D1 with the candidate box D3 is greater than 20%; the degree of overlap of the first candidate box D1 with the candidate box D6 Less than 20%; the degree of overlap of the first candidate box D1 and the candidate box D4 is greater than 20%; the degree of overlap of the first candidate box D1 with the candidate box D7 is less than 20%; the first candidate box D5 and the candidate box The overlap of D2 is less than 20%. Then, the candidate box D2 and the candidate box D3 are deleted. In the second screening, the set of candidate boxes includes an alternative box D6, an alternate box D4, an alternate box D7, and an alternate box D5. The candidate box D6 with the highest score of the classifier is selected as the first candidate box. Correspondingly, a position vector is generated, which includes the position information of the candidate box D4, the position information of the candidate box D7, and the position information of the candidate box D5. Performing a vector calculation on the position information of the first candidate box D6 and the one calculation vector to obtain the candidate box D4, the candidate box D7, and the candidate box D5 in the first candidate box D6 and the second calculation vector. The degree of overlap. The degree of overlap of the first candidate block D6 and the candidate box D4 is less than 20%; the degree of overlap of the first candidate box D6 with the candidate box D7 is greater than 20%; the degree of overlap of the first candidate box D6 with the candidate box D5 More than 20%. Then the candidate box D4 is deleted. In the third screening, the candidate box D7 is selected as the first candidate box, and the degree of overlap between the first candidate box D7 and the candidate box D5 is calculated. The degree of overlap of the first candidate block D7 with the candidate box D5 is less than 20%. Then, the candidate box D5 is also used as the first candidate box, and the update of the candidate box in the candidate box set is stopped, and the NMS calculation is completed.

The second example:

If the number of elements in the position vector is four, the position information of one calculation vector and two single second candidate frames is generated in the first screening. The seven candidate boxes can be arranged according to the classifier score, and the order of arrangement is the candidate box D1, the candidate box D2, the candidate box D3, the candidate box D6, the candidate box D4, the candidate box D7, and the candidate box. D5. In the first screening, the candidate box D1 with the highest score of the classifier is selected as the first candidate box. The location vector includes location information of the candidate frame D2, location information of the candidate frame D3, location information of the candidate frame D6, and location information of the candidate frame D4. The location information of the two single second candidate frames is the location information of the candidate frame D7 and the location information of the candidate frame D5, respectively. Then you need to do three calculations. The first calculation calculates the degree of overlap between the first candidate box D1 and the candidate box D2, the candidate box D3, the candidate box D6 and the candidate box D4 in the position vector, and the second calculation results the first candidate box. The degree of overlap of D1 with the candidate box D5 is calculated for the third time to obtain the degree of overlap between the first candidate box D1 and the candidate box D7. The degree of overlap of the first candidate box D1 and the candidate box D2 is greater than 20%; the degree of overlap of the first candidate box D1 with the candidate box D3 is greater than 20%; the degree of overlap of the first candidate box D1 with the candidate box D6 Less than 20%; the degree of overlap of the first candidate box D1 and the candidate box D4 is greater than 20%; the degree of overlap of the first candidate box D1 with the candidate box D7 is less than 20%; the first candidate box D5 and the candidate box The overlap of D2 is less than 20%. Then, the candidate box D2 and the candidate box D3 are deleted. In the second screening, the set of candidate boxes includes an alternative box D6, an alternate box D4, an alternate box D7, and an alternate box D5. The candidate box D6 with the highest score of the classifier is selected as the first candidate box. The degree of overlap of the first candidate box D6 with the candidate box D4, the candidate box D7, and the candidate box D5, respectively, is calculated. The degree of overlap of the first candidate block D6 and the candidate box D4 is less than 20%; the degree of overlap of the first candidate box D6 with the candidate box D7 is greater than 20%; the degree of overlap of the first candidate box D6 with the candidate box D5 More than 20%. Then the candidate box D4 is deleted. In the third screening, the candidate box D7 is selected as the first candidate box, and the degree of overlap between the first candidate box D7 and the candidate box D5 is calculated. The degree of overlap of the first candidate block D7 with the candidate box D5 is less than 20%. Then, the candidate box D5 is also used as the first candidate box, and the update of the candidate box in the candidate box set is stopped, and the NMS calculation is completed.

FIG. 5 is a schematic structural diagram of an apparatus for updating an alternative frame according to an embodiment of the present application. As shown in FIG. 5, the candidate box updating apparatus 300 may include an overlap degree acquiring module 301 and an updating module 302.

The overlap obtaining module 301 is configured to obtain the degree of overlap between the first candidate frame and the plurality of second candidate frames according to the location information and the location vector of the first candidate frame.

The first candidate box is an candidate box with the highest score of the classifier in the set of candidate boxes. The second alternative box is an alternative box in the set of candidate boxes other than the first candidate box. The location vector includes location information of a plurality of second candidate frames.

The updating module 302 is configured to compare the first candidate box with the overlap degree and the overlap degree threshold of the plurality of second candidate boxes to obtain an updated set of candidate frames.

The foregoing update module 302 is specifically configured to determine that the second candidate frame remains in the updated state when the overlap degree of the second candidate frame and the first candidate frame in the multiple second candidate frames is less than or equal to the overlap degree threshold. In the set of candidate boxes; determining that the second candidate box does not remain in the updated candidate when the degree of overlap of the second candidate box and the first candidate box in the plurality of second candidate boxes is greater than the overlap threshold In the box collection.

In an example, the overlap acquiring module 301 is further configured to use the location information of the first candidate frame in the updated candidate set of frames and the updated location vector to obtain the updated candidate set. The degree of overlap between an alternate box and a plurality of second candidate boxes in the updated set of candidate boxes. The updated location vector includes location information of a plurality of second candidate frames in the updated set of candidate frames.

The update module 302 is further configured to compare the overlap degree and the overlap threshold of the first candidate block in the updated set of candidate blocks with the plurality of second candidate blocks in the updated set of candidate blocks to obtain again The updated set of candidate boxes until the overlap degree of the first candidate frame in the updated candidate set of the block obtained by the overlap obtaining module 301 and the second candidate frame in the updated set of candidate blocks is greater than The overlap threshold, the update module 302 is further configured to stop updating the set of candidate boxes, and to use the first candidate box in the set of candidate boxes and the first candidate box in the set of candidate boxes after each update. , as the target recognition box.

In the face recognition scene, the set of candidate boxes contains a plurality of candidate boxes for identifying faces in the image. In the target recognition scenario, the set of candidate boxes contains a plurality of candidate boxes for identifying the target objects in the image.

For the specific functions and the beneficial effects of the above-mentioned overlapping degree obtaining module 301 and the updating module 302, reference may be made to the descriptions of the steps S101 and S102 in the above embodiment, and the related details of the steps S201 to S205, and details are not described herein again.

The alternate block update method and apparatus according to embodiments of the present application described in conjunction with FIGS. 1 through 5 may be implemented by the alternate block update device 400. FIG. 6 is a schematic structural diagram of hardware of an alternative frame updating device 400 in the embodiment of the present application.

The alternate frame update device 400 includes a memory 401, a processor 402, and a computer program stored on the memory 401 and executable on the processor 402.

In one example, the processor 402 described above can include a central processing unit (CPU), or an specific integrated circuit (ASIC), or can be configured to implement one or more integrated circuits of embodiments of the present application.

Memory 401 can include mass storage for data or instructions. By way of example and not limitation, the memory 401 may comprise a HDD, a floppy disk drive, a flash memory, an optical disk, a magneto-optical disk, a magnetic tape or a universal serial bus (USB) drive, or a combination of two or more of these. Memory 401 may include removable or non-removable (or fixed) media, where appropriate. Memory 401 may be internal or external to data query device 400, where appropriate. In a particular embodiment, memory 401 is a non-volatile solid state memory. In a particular embodiment, memory 401 includes a read only memory (ROM). Where appropriate, the ROM may be a mask programmed ROM, a programmable ROM (PROM), an erasable PROM (EPROM), an electrically erasable PROM (EEPROM), an electrically rewritable ROM (EAROM) or flash memory or A combination of two or more of these.

The processor 402 runs a program corresponding to the executable program code by reading the executable program code stored in the memory 401 for performing the alternate frame update method in the various embodiments described above and the NMS method including the candidate frame update. .

In one example, the alternate frame update device 400 can also include a communication interface 403 and a bus 404. As shown in FIG. 6, the memory 401, the processor 402, and the communication interface 403 are connected by the bus 404 and complete communication with each other.

The communication interface 403 is mainly used to implement communication between modules, devices, units and/or devices in the embodiments of the present application. Communication interface 403 is also accessible to input devices and/or output devices.

Bus 404 includes hardware, software, or both, coupling components of data querying device 400 to each other. By way of example and not limitation, bus 404 may include an accelerated graphics port (AGP) or other graphics bus, an enhanced industry standard architecture (EISA) bus, a front side bus (FSB), a super transfer (HT) interconnect, an industry standard architecture (ISA) Bus, Infinite Bandwidth Interconnect, Low Pin Count (LPC) Bus, Memory Bus, Micro Channel Architecture (MCA) Bus, Peripheral Component Interconnect (PCI) Bus, PCI-Express (PCI-X) Bus, Serial Advanced Technical Attachment (SATA) bus, Video Electronics Standards Association Local (VLB) bus or other suitable bus or a combination of two or more of these. Bus 404 may include one or more buses, where appropriate. Although a particular bus is described and illustrated in this application, the present application contemplates any suitable bus or interconnect.

An embodiment of the present application further provides a storage medium, where the program is stored, and when the program is executed by the processor, the candidate box update method in the foregoing embodiments and the NMS method including the candidate box update may be implemented.

The various parts of the specification are described in a progressive manner, and the same or similar parts between the various embodiments may be referred to each other, and each embodiment focuses on differences from other embodiments. In particular, for the device embodiment, the device embodiment, and the storage medium embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the description of the method embodiment.

The above is only a specific embodiment of the present application, but the scope of protection of the present application is not limited thereto, and any person skilled in the art can easily think of changes or substitutions within the technical scope of the present invention, and should cover Within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of protection of the claims.

Claims (10)

An alternative frame updating method, comprising: Obtaining an overlap degree between the first candidate frame and the plurality of second candidate frames according to the location information and the location vector of the first candidate frame, where the first candidate frame is the highest score of the classifier in the candidate frame set An alternative box, the second candidate box is an alternative box in the set of candidate boxes except the first candidate box, the location vector comprising the plurality of second candidate boxes location information; Comparing the first candidate box with the overlap and overlap thresholds of the plurality of second candidate boxes to obtain an updated set of candidate frames. The method of claim 1, wherein the comparing the first candidate box with the overlap and overlap thresholds of the plurality of second candidate boxes to obtain an updated candidate box Collections, including: Determining that the second candidate box remains updated after the overlap degree of the second candidate box and the first candidate box in the plurality of second candidate boxes is less than or equal to the overlap degree threshold In the set of alternative boxes; Determining that the second candidate box does not remain in the updated when the degree of overlap of the second candidate box and the first candidate box in the plurality of second candidate boxes is greater than the overlap degree threshold In the set of candidate boxes. The method of claim 2, further comprising: Using the location information of the first candidate box in the updated set of candidate boxes and the updated location vector, obtaining the first candidate box in the updated candidate box set and the updated candidate box set The degree of overlap of the second candidate frame, the updated location vector including location information of the plurality of second candidate frames in the updated set of candidate frames; Comparing the overlap between the first candidate block in the updated set of candidate boxes and the plurality of second candidate blocks in the updated set of candidate blocks and the overlap threshold to obtain the updated device again Selecting a box, until the overlap degree of the first candidate box in the updated candidate box set and the second candidate box in the updated candidate box set is greater than the overlap degree threshold, stopping the preparation Update of the marquee collection; The first candidate box in the set of candidate boxes and the first candidate box in the set of candidate boxes after each update are used as target recognition frames. The method according to any one of claims 1-3, wherein the candidate set of frames comprises a plurality of candidate boxes for identifying faces in the image. An alternative frame updating device, comprising: An overlap obtaining module, configured to obtain, according to the location information and the location vector of the first candidate frame, the degree of overlap between the first candidate frame and the plurality of second candidate frames, where the first candidate frame is an alternative An alternative box with the highest score of the classifier in the set of boxes, the second candidate box being an alternative box of the set of candidate boxes other than the first candidate box, the position vector including the plurality of Position information of the second candidate box; And an update module, configured to compare the first candidate box with the overlap degree and overlap threshold of the plurality of second candidate boxes to obtain an updated set of candidate frames. The device according to claim 5, wherein the update module is specifically configured to: Determining that the second candidate box remains updated after the overlap degree of the second candidate box and the first candidate box in the plurality of second candidate boxes is less than or equal to the overlap degree threshold In the set of alternative boxes; Determining that the second candidate box does not remain in the updated when the degree of overlap of the second candidate box and the first candidate box in the plurality of second candidate boxes is greater than the overlap degree threshold In the set of candidate boxes. The apparatus according to claim 6, wherein the overlap degree obtaining module is further configured to use the position information of the first candidate box in the updated set of candidate boxes and the updated position vector to obtain an update. The degree of overlap of the first candidate box in the set of candidate boxes with the plurality of second candidate boxes in the updated set of candidate boxes, the updated position vector including more of the updated set of candidate boxes Position information of the second candidate box; The update module is further configured to compare the overlap between the first candidate block in the updated set of candidate boxes and the plurality of second candidate blocks in the updated set of candidate blocks and the overlap threshold To obtain the updated set of candidate boxes again, until the first candidate box in the updated candidate box set obtained by the overlap degree acquisition module and the second candidate box in the updated candidate box set The degree of overlap is greater than the overlap threshold, the update module is configured to stop updating the set of candidate boxes, and to use the first candidate box in the set of candidate boxes and after each update The first candidate box in the set of candidate boxes is used as the target recognition box. The apparatus of any of claims 5-7, wherein the set of candidate boxes includes a plurality of candidate boxes for identifying faces in the image. An alternative frame updating device, comprising: Memory for storing programs; a processor for executing the program stored by the memory, the processor for implementing the alternate frame update method according to any one of claims 1 to 4 when the program is executed. A storage medium characterized by storing a program thereon, the program being executed by a processor to implement the alternative frame updating method according to any one of claims 1 to 4.

Images