JP2021131734A - Object detection device, object detection system, and object detection method - Google Patents

Abstract

[Problem] To support the construction of dictionary data to improve detection accuracy of objects within a measurement range. An object detection device 1 includes dictionary data 12D that includes a set of learning image data different from real image data that is a target of object detection, and learning scene data tagged with each learning image data. , an object detection unit 15 that detects an object from the real image data that is the target of object detection using dictionary data 12D, and real scene data that tags the real image data from the object detection result of the object detection unit 15. A real scene estimation unit 16 performs estimation, a scene similarity calculation unit 17 calculates the similarity between the estimated real scene data and the learning scene data of the dictionary data 12D, and outputs learning element data necessary for additional learning. It has an additional learning element output section 18. [Selection diagram] Figure 1

Description

The present invention relates to an object detection device, an object detection system, and an object detection method.

By analyzing the video data acquired by a surveillance camera or the like using video recognition technology, it is possible to recognize the detection target and surrounding objects. As a general image recognition technique, there is a method of using dictionary data created by machine learning such as a convolutional neural network.
In order to create the machine learning dictionary data, it is necessary to collect image data for learning in advance. Then, for a large amount of collected image data, the work (called annotation) of manually adding explanatory data such as the type, position, and size of the object reflected in each image data to the image data as tag data is performed. Will be done.

Originally, it would be good if we could learn a large amount of various scenes of surveillance cameras installed at various points, but considering the realistic man-hours, the variations of points and scenes are limited. Therefore, depending on the installation environment of the surveillance camera, there are many cases where the object detection accuracy based on the constructed dictionary data is lowered. As a countermeasure, there is a method of updating the dictionary data constructed in advance by additionally learning an image similar to the local camera angle of view and the shooting scene.

However, since new work such as annotation to images is required for additional learning, a method for efficiently constructing highly accurate dictionary data is required. In Patent Document 1, a system that applies dictionary data created from image data of detection targets taken in a plurality of camera directions to a local image and performs additional learning based on the dictionary data that outputs the highest likelihood. Have been described.

Japanese Unexamined Patent Publication No. 2016-15045

At event venues where multiple surveillance cameras are installed in the same space, even if the image data is the same subject, the shooting results of the subject in the image data will differ depending on the installation environment of each surveillance camera. There is also. Further, even if a plurality of image data are taken by the same surveillance camera at different times, the shooting result of the subject in the image data may be different depending on the difference in the position of the subject at each time.

Therefore, the dictionary data for collating with the image data (actual image data) in which the subject to be actually monitored is captured is the actual image by using the data learned from the image data set in an environment as close as possible to the actual image data. The detection accuracy of the subject is improved from the data.
However, in the conventional technique, machine learning has not been performed from the viewpoint of selecting dictionary data suitable for the environment of real image data. In Patent Document 1, only dictionary data to be a source of additional learning is selected by utilizing the orientation information of the detection target.

Therefore, the main object of the present invention is to support the construction of dictionary data for improving the detection accuracy of a subject within the measurement range.

In order to solve the above problems, the object detection device of the present invention has the following features.
The present invention includes a storage unit for dictionary data having a set of learning measurement data and learning scene data tagged with each learning measurement data.
An object detection unit that detects an object from the actual measurement data that is the target of object detection using the dictionary data,
An actual scene estimation unit that estimates the actual scene data to be tagged with the actual measurement data from the object detection result of the object detection unit, and
A scene similarity calculation unit that calculates the similarity between the estimated actual scene data and the learning scene data of the dictionary data, and
It has an additional learning element output unit that outputs learning element data required for additional learning of the dictionary data.
The additional learning element output unit
When the similarity calculated by the scene similarity calculation unit is higher than a predetermined threshold value, the learning element data based on the learning scene data used for the calculation of the similarity is output.
When the similarity calculated by the scene similarity calculation unit is equal to or less than a predetermined threshold value, the learning element data based on the actual scene data used for the calculation of the similarity is output.
Other means will be described later.

According to the present invention, it is possible to support the construction of dictionary data for improving the detection accuracy of a subject within the measurement range.

It is a block diagram of the object detection apparatus which concerns on Example 1 of this invention. It is a block diagram of the learning scene acquisition part which concerns on Example 1 of this invention. It is a block diagram of the additional learning element output part which concerns on Example 1 of this invention. It is a figure which shows an example of the learning image data which concerns on Example 1 of this invention. This is the learning scene data added to the learning image data of FIG. 4 according to the first embodiment of the present invention. It is explanatory drawing of the annotation data analysis part which concerns on Example 1 of this invention. It is a figure which shows an example of the real image data which concerns on Example 1 of this invention. It is the actual scene data added to the actual image data of FIG. 7 according to the first embodiment of the present invention. It is explanatory drawing of the real scene estimation part with respect to Example 1 of this invention. It is a figure which shows an example of the 2nd real image data which concerns on Example 1 of this invention. It is the second actual scene data which concerns on Example 1 of this invention. It is a 2nd explanatory drawing of the real scene estimation part which concerns on Example 1 of this invention. It is a flowchart which shows the process of the learning scene analysis part which concerns on Example 1 of this invention. It is a block diagram of the object detection apparatus which concerns on Example 2 of this invention.

Hereinafter, specific embodiments of the present invention (Examples 1 and 2) will be described with reference to the drawings.

FIG. 1 is a block diagram of the object detection device 1.
The object detection device 1 is configured as a computer having a CPU (Central Processing Unit) as an arithmetic unit, a memory as a main storage device, and a hard disk as an external storage device.
In this computer, the CPU operates a control unit (control means) composed of each processing unit by executing a program (also called an application or an abbreviation for application) read in the memory.

The camera 2 is a measuring device installed at a measurement site and outputs captured actual image data (actual measurement data) to be an object detection target to the camera information acquisition unit 14. The object detection device 1 may be in the same housing as the camera 2, or may be in a separate housing from the camera 2. Although the case where the measuring device is a monaural camera 2 will be described in the first embodiment, the measuring device is not limited to this, and can be applied to other sensors such as a stereo camera and a distance sensor.

The training image data 3 (details in FIG. 4) is a set of training image data (learning measurement data) that is collated with the actual image data for object detection. That is, the image data is classified into real image data that is a target of object detection and learning image data that is a material for constructing dictionary data 12D by machine learning.
Hereinafter, one image data (frame) constituting a set of image data is referred to as a "scene". The scene data is tag data added by annotation for each image data of each scene. For example, the actual scene data is added to the actual image data, and the learning scene data is added to the learning image data.
Therefore, the object detection device 1 compares the learning scene data with the actual scene data to specify the learning image data necessary for the additional learning of the dictionary data 12D (hereinafter, “learning element data”). Is output.

The object detection device 1 includes an annotation unit 11, a dictionary generation unit 12, a storage unit for dictionary data 12D, a learning scene acquisition unit 13, a camera information acquisition unit 14, an object detection unit 15, and an actual scene estimation unit 16. And a scene similarity calculation unit 17 and an additional learning element output unit 18.
The annotation unit 11 tags (annotates) the learning scene data with respect to each of the learning image data constituting the learning image data 3 (details are shown in FIG. 5).
The dictionary generation unit 12 constructs the combination data of the learning scene data generated by the annotation unit 11 and the learning image data 3 as the dictionary data 12D.
The learning scene acquisition unit 13 selects each scene from the learning image data 3, and acquires the learning scene data for each selected scene from the dictionary data 12D.

FIG. 2 is a configuration diagram of the learning scene acquisition unit 13.
The learning scene acquisition unit 13 includes an annotation data analysis unit 131, a video analysis unit 132, a camera parameter acquisition unit 133, and a learning scene analysis unit 134.
The annotation data analysis unit 131 acquires a position distribution map of the detection target by analyzing the acquired learning scene data (details are shown in FIG. 6).
The image analysis unit 132 analyzes the image in the learning image data 3 and acquires shooting conditions such as image quality as shooting environment information.
The camera parameter acquisition unit 133 acquires the parameters of the camera 2 that captured the image in the learning image data 3.
The learning scene analysis unit 134 analyzes the learning scene based on the position distribution map of the annotation data analysis unit 131, the shooting environment information of the video analysis unit 132, and the camera parameters of the camera parameter acquisition unit 133, and analyzes the analysis result. It is used as learning scene data (details are shown in FIG. 13).

Returning to FIG. 1, an outline of each component of the object detection device 1 will be described.
The camera information acquisition unit 14 acquires, for example, shooting environment information and camera parameters as actual scene data to be added to the actual image data in addition to the actual image data which is the captured image of the camera 2. The shooting environment information is information indicating shooting conditions of actual image data, and is, for example, shooting time zone information and shooting location information. The actual scene data is the same as the learning scene data, and the content thereof is not particularly limited.
The object detection unit 15 detects an object to be detected existing in the actual image data using the dictionary data 12D (details are shown in FIG. 5).
The actual scene estimation unit 16 estimates the actual scene data to be added to the actual image data based on the shooting environment information and the camera parameters acquired by the camera information acquisition unit 14 and the object detection result of the object detection unit 15.
The scene similarity calculation unit 17 calculates the similarity between the two scenes by comparing the learning scene data acquired by the learning scene acquisition unit 13 with the actual scene data estimated by the actual scene estimation unit 16.

FIG. 3 is a configuration diagram of the additional learning element output unit 18.
The additional learning element output unit 18 outputs learning element data necessary for additional learning of the dictionary data 12D from the similarity calculated by the scene similarity calculation unit 17. The additional learning element output unit 18 includes a dictionary fitness acquisition unit 181, an additional learning determination unit 182, and an additional learning element determination unit 183.
The dictionary fitness acquisition unit 181 acquires the fitness of the currently used dictionary data 12D to the actual scene data.
The additional learning determination unit 182 determines whether or not additional learning is necessary according to the fitness of the dictionary fitness acquisition unit 181.
The additional learning element determination unit 183 determines the learning element data required for the additional learning when the additional learning determination unit 182 determines that the additional learning is necessary, and outputs the learning element data to the user or another system. ..

FIG. 4 is a diagram showing an example of the learning image data 3.
In this learning image data 3a, two detection targets (persons 111 and 121 in this example) are captured. The detection frames 112 and 122 are rectangles surrounding the persons 111 and 121, and the start point coordinates 113 and 123 are the coordinates of the start point (upper left point) in the learning image data 3a of the detection frames 112 and 122.

FIG. 5 is learning scene data added to the learning image data of FIG.
The annotation unit 11 manually accepts the input of the detection frames 112 and 122 for each person 111 and 121 of the learning image data for each scene by using a GUI or the like. Further, the annotation unit 11 accepts input of detailed information of each person 111, 121 designated by the detection frames 112, 122. The following is an example of detailed information.
-Class information indicating what the target is (here, "Person" indicating a person) and reliability indicating the probability of being the class information-Start point coordinates 113,123 of the detection frame
-Detection frame size (width information and height information)
Each of these input information is added to the learning image data as learning scene data.

The dictionary generation unit 12 generates dictionary data 12D capable of detecting a person in an image from tag data (learning scene data) corresponding to the learning image data 3 by utilizing machine learning or the like. In other words, the dictionary generation unit 12 generates an inference model with the learning scene data as the correct answer label and the learning image data 3 as the input data, so that the object detection unit 15 uses the actual image data as the input data. The inference model (dictionary data 12D) makes it possible to identify the actual scene data (object to be detected).
In this embodiment, a plurality of events may be generated in advance from a pair of the plurality of learned image data 3 and the tag data. Further, the algorithm for generating the dictionary data 12D may be a general algorithm such as a convolutional neural network or AdaBoost, and is not particularly limited.

FIG. 6 is an explanatory diagram of the annotation data analysis unit 131. The annotation data analysis unit 131 calculates a position distribution map for each class to be detected for each training image data from the information (start point X, start point Y, width, height) of the detection frame of FIG. As a method of generating the position distribution map, for example, the following steps 1 to 3 are followed.
(Procedure 1) The position distribution 210 of the training image data 3 is divided into a plurality of small regions (here, 6 × 6 cells). The detection frame 112 of the person 111 on the left side corresponds to the area 211, and the detection frame 122 of the person 121 on the right side corresponds to the area 212.
(Procedure 2) For each cell, the degree of overlap with the detection frame is calculated as a ratio. For example, the area 211 overlaps with 4 cells in the vertical direction and 1 cell in the horizontal direction. It almost (90%) overlaps the top cell, 100% overlaps the second cell from the top, 100% overlaps the third cell from the top, and 40% overlaps the fourth cell from the top. ing.
(Procedure 3) The position distribution map 220 is generated based on the overlapping condition of the procedure 2. Here, as the cell value of the position distribution map 220, the degree of overlap of 100% is set to "1", and the degree of overlap of 0% is set to "0".

The number of cell divisions is not particularly limited and may be determined in consideration of the specifications of the PC that executes the processing, and the position distribution map is divided into a plurality of areas after the resolution of the learning image data 3 is reduced in advance. It may be generated. Further, the value of the position distribution map is not particularly limited as long as it indicates the ratio of the objects of each class existing in the cell, and the degree of overlap with the detection frame is not used, but the center of the detection frame. The method of calculating the abundance rate of each region by generating a normal distribution in which the value of the cell including the coordinates is "1" and the rectangular end is "0" is not particularly limited.

The image analysis unit 132 of FIG. 2 extracts the shooting environment information by analyzing the image of the learning image data 3. The type of shooting environment information is not particularly limited as long as it affects the object detection accuracy and can be acquired by analyzing the image. For example, by utilizing the scene recognition technology, the installation location such as outdoors or indoors. Information, information on shooting time zones such as daytime and nighttime by analyzing the brightness information of the image, and information on lens blur estimated by image analysis.

The camera parameter acquisition unit 133 acquires the camera parameters of the camera that captured the image of the learning image data 3. The types of parameters to be acquired include internal parameters such as focal length and lens distortion coefficient and external parameters such as camera depression angle and installation height. It is preferable to acquire all parameters, but some parameters are to be acquired. You may only get it. In the camera parameter acquisition unit 133, a part of the camera parameters may be estimated by image analysis, and for example, a method of estimating the external parameters of the camera by utilizing the vanishing point information of the image may be adopted. ..

In the explanation so far, it is assumed that one image data indicates one scene. On the other hand, when the dictionary data 12D contains a large amount of training image data, the calculation amount of the comparison processing between the actual scene data and the learning scene data by the scene similarity calculation unit 17 increases according to the number of training scene data. Resulting in. Therefore, the learning scene analysis unit 134 may reduce the number of comparison processes by grouping a plurality of learning scenes as one learning scene (details are shown in FIG. 13).

The learning scene data has been described above with reference to FIGS. 4 to 6. Hereinafter, the actual scene data will be described with reference to FIGS. 7 to 12.
The difference between the training image data and the actual image data is that the learning scene data, which is the correct answer data, can be taught by an external user via the annotation unit 11, but the actual scene data of the actual image data is automatically obtained. It needs to be analyzed and obtained.

FIG. 7 is a diagram showing an example of actual image data. In this actual image data 2a, one detection target (person 131 in this example) is shown. The detection frame 132 is a rectangle surrounding the person 131.
FIG. 8 is actual scene data added to the actual image data of FIG. 7. The actual scene estimation unit 16 uses the detection frame information indicating the detection frame 132 of the person 131 detected from the actual image data 2a by the object detection unit 15 to provide information indicating the actual scene data of the site where the camera 2 is installed. get.

FIG. 9 is an explanatory diagram of the actual scene estimation unit 16.
The acquisition process of the position distribution map 240 constituting the actual scene data by the actual scene estimation unit 16 is similar to the detection process of the position distribution map 220 by the annotation data analysis unit 131 shown in FIG. Specifically, the actual scene estimation unit 16 creates a position distribution map 230 including the region 231 corresponding to the detection frame 132. Then, the actual scene estimation unit 16 uses the reliable position distribution map 240 including the region 241 in which the reliability "0.8" of FIG. 8 is weighted (multiplied) with respect to each cell value of the region 231 of the position distribution map 230. Acquired as part of the actual scene data.
Further, the actual scene estimation unit 16 calculates the reliability position distribution map by performing the same processing as the generation process of the reliability position distribution map 240 for the plurality of captured images in the camera image. The shooting environment information acquired by the camera information acquisition unit 14 and the camera parameter information are combined and output.

As described above, the actual image data and the actual scene data of FIGS. 7 to 9 have been described as an example of data having a high degree of scene similarity with the learning image data and the learning scene data of FIGS. 4 to 6. That is, since the detection frame 132 in the actual image data of FIG. 7 and the detection frame 112 in the learning image data of FIG. 4 substantially match, both shooting environment information are similar, so that the actual image data It is easy to determine whether the person 131 and the person 111 in the learning image data are the same person.

On the other hand, the actual image data and the actual scene data of FIGS. 10 to 12 are examples of data having low scene similarity.
Although the same camera 2 was used for shooting, the person 131 on the left side in the actual image data 2a of FIG. 7 has moved as the person 141 on the back side of the center in the actual image data 2b of FIG. Therefore, in the actual scene data of FIG. 11 corresponding to the detection frame 142 of the person 141 of FIG. 10, since the detection frame 142 is small, its reliability is also “0.7”, which is smaller than “0.8” of FIG.
Further, the actual scene estimation unit 16 creates the position distribution map 250 of FIG. 12 including the area 251 corresponding to the detection frame 142 of the detection frame information of FIG. 11, and weights the position distribution map 250 with a reliability “0.7”. A reliable position distribution map 260 including the (multiplied) region 261 is created.

Although the actual scene data can be grasped more accurately as the number of position distribution maps to be acquired increases, a process of reducing the number of position distribution maps to be acquired may be added in order to reduce the processing cost. Hereinafter, a process for reducing the number of position distribution maps will be illustrated.
-A method that uses only a highly reliable position distribution map.
-A method of grouping as in the learning scene analysis unit 134. For example, the total difference between position distribution maps is calculated, and if the difference is less than or equal to a predetermined threshold, it is treated as a similar scene and classified into multiple groups. Method of using a representative position distribution map in the scene-As another method of grouping as in the learning scene analysis unit 134, grouping that considers the shooting environment information (bunching a set of members whose shooting environment information is similar to each other) is performed. , How to output only the representative position distribution map in each group.

The scene similarity calculation unit 17 calculates the similarity between scenes by the method illustrated below from the position distribution map, shooting conditions, and camera parameter information in the learning scene and the actual scene.
-A method of determining that the total difference of the position distribution map is small-A method of determining that the one with similar shooting environment information and camera parameters has a high degree of similarity-The final value is the sum of multiple similarities. How to make similarities

Note that the position distribution map 240 of FIG. 9 is similar to the position distribution map 220 of FIG. 6 in the detection results (regions 241 and 221) on the lower left side of the images corresponding to each other, so that the scene similarity is high. Calculated as a thing.
On the other hand, with respect to the position distribution map 260 of FIG. 12, the detection result (region 261) on the upper center of the image does not exist in the position distribution map 220 of FIG. 6 (cell value of the position distribution map 220 corresponding to the region 261). Is "0"), so the scene similarity is calculated as low.

The "similarity" described by the scene similarity calculation unit 17 is compared between one real image data (actual scene data) and one learning image data (learning scene data). It is an index for each scene.
On the other hand, the "fitness" described in the additional learning element output unit 18 from now on is calculated between N pieces of learning image data (dictionary data 12D which is the learning result from) for one piece of real image data. It is an index.
The fitness is used to determine whether additional learning is required for the current dictionary data 12D with respect to the accuracy of detecting a person or the like from the actual image data. In other words, even if the current dictionary data 12D is used, if the accuracy of detecting a person or the like from the predetermined real image data is high, additional learning is not necessary.
On the other hand, the similarity is used to specify what kind of learning element data of the additional learning is notified to the user to add the learning image data after it is determined that the additional learning is necessary.

The dictionary fitness acquisition unit 181 of FIG. 3 acquires the fitness of the dictionary data 12D constructed by the dictionary generation unit 12 to the actual scene. As a method of acquiring the degree of adaptability, the actual image data of the camera 2 to which the actual scene data is added (annotated) in advance is prepared, and the ratio of the number of objects detected by the dictionary data 12D to the total number of detection targets in the actual image data, etc. There is a method of adopting object detection accuracy.

The additional learning determination unit 182 determines that additional learning is necessary when the fitness of the dictionary data 12D acquired from the dictionary fitness acquisition unit 181 to the actual scene is lower than a predetermined threshold value. Instead of omitting the dictionary fitness acquisition unit 181, the user visually confirms the detection result such as the detection frame for the captured image of the camera 2, so that the user can determine whether or not additional learning is necessary. good.

The additional learning element determination unit 183 determines the learning element data having the contents illustrated below based on the similarity information obtained by the scene similarity calculation unit 17.
-Image type information indicating the type of learning image data required for additional learning.
-Learning method information such as how to annotate learning scene data.
-Dictionary type information indicating information related to the construction of dictionary data 12D.

Therefore, the additional learning element determination unit 183 determines the learning element data from the similarity information. The method illustrated below is a method for determining learning element data based on the learning scene data used for calculating the similarity when the similarity information of the position distribution map is higher than the threshold value.
-A method of presenting the learning image data of the (corresponding) group used for calculating the similarity information to the user and determining that the user manually collected similar images-Corresponding to the learning image data of the corresponding group A method of automatically creating a learning image by replacing the person area in the image with the person area in the learning image of another group by referring to the position distribution map of the learning scene data.

On the other hand, the method illustrated below is a method for determining learning element data based on the actual scene data used for calculating the similarity when the similarity information of the position distribution map is equal to or less than the threshold value.
-A method in which the position distribution map of the actual scene data and the information of the detection frame are clearly shown to the user, and the user manually collects similar images.-The position distribution map of the object detection by the dictionary data 12D is generated for the image, and the actual scene. After searching for an image that has a high degree of similarity to the data position distribution map, if there is an image with a high degree of similarity, a method of clearly indicating to the user that the image should be annotated.

When a plurality of dictionary data 12Ds are generated from different learning image data 3 in the dictionary generation unit 12, the learning image data 3 having the highest degree of similarity to the position distribution map of the actual scene is searched, and the corresponding learning image data is searched. There is also a method of clearly indicating to the user to use the dictionary data 12D corresponding to 3.
Further, although there is the learning image data 3 having the highest degree of similarity to the position distribution map of the actual scene, if the degree of similarity is lower than the threshold value, a method of presenting the corresponding dictionary data 12D and the necessary learning image to the user is adopted. You may.
Further, the additional learning element determination unit 183 includes a position distribution map of different objects, a position distribution map of the object that the object detection unit 15 failed to detect, and a position distribution map of the object that the object detection unit 15 erroneously detected. Any of them may be used to determine the learning element data.

FIG. 13 is a flowchart showing the processing of the learning scene analysis unit 134. As shown below, the learning scene analysis unit 134 analyzes the learning scene data using the output information from the annotation data analysis unit 131, the video analysis unit 132, and the camera parameter acquisition unit 133.
Hereinafter, as the counter variables in the flowchart of FIG. 13, the variable n for each training image data constituting the set of the training image data 3 and the variable m for each group generated as a result of grouping are used.

First, as an initialization process, the learning scene analysis unit 134 extracts one learning image data (n = 1) from the set of learning image data 3, and a new group (m = 1) including the extracted n = 1. Is created (S101). In addition to the learning image data (n = 1), the learning scene data (information such as a position distribution map) is also associated with the group (m = 1).

Hereinafter, in the outer loop shown in S102 to S121, the learning image data (n = 2, ..., N) are selected in order, and the learning scene data (position distribution map, shooting environment information, camera) corresponding to the selected learning image data is selected. After acquiring the parameter) (S103), the learning scene analysis unit 134 executes the inner loop.
In the inner loop shown in S111 to S117, the learning scene analysis unit 134 sequentially selects the already created groups (m = 1, ..., M).
The learning scene analysis unit 134 calculates the total difference of the position distribution maps in the selected group m (S112), and determines whether or not the total difference is less than the threshold value (S113). If Yes in S113, the process proceeds to S114, and if No, the process returns to S102.

If the learning scene analysis unit 134 completes the search for all the groups (m = 1, ..., M) (S114, Yes), the currently selected learning image data n does not correspond to any of the existing groups m. , A new group (m = M + 1) is created, and the currently selected learning image data n is assigned to the new group (S115). If there is an unsearched group (S114, No), the currently selected training image data n is assigned to the existing group m (S116).
As described above, by executing the outer loop shown in S102 to S121, each learning image data of the learning image data 3 is assigned to the group of the learning scene data.

In each group (m = 1, ..., M) of the created learning scene, the learning scene analysis unit 134 may further classify the learning images in the group according to the shooting environment information and the camera parameters (S122). The classification method is not particularly limited, and there are a method of classifying into indoor and outdoor, a method of classifying from information of shooting time zone, and the like. When the camera parameters are used for classification, for example, there is a method of classifying them into four, such as 0 to 10 degrees, 10 to 45 degrees, 45 to 80 degrees, and 80 to 90 degrees, based on the camera depression angle information. Not particularly limited.
Further, in the learning scene analysis unit 134, the maximum number of groups may be determined in advance when classifying the learning scenes into the number of groups of the learning scenes, and the difference information of the position distribution map between all the learning images is determined by the K-means method. A method of clustering learning images from the above may be used, and is not particularly limited.

According to the first embodiment described above, in the object detection device 1 that detects an object within the measurement range, the scene similarity calculation unit 17 calculates the similarity between the actual scene data and the learning scene data. It is possible to output learning element data for generating dictionary data 12D that recognizes an object in real image data with high accuracy.
In the first embodiment, the position information of the detection target between the learning scene and the actual scene is compared using the position distribution map, but any method can compare the positional relationship between the detection targets in the image. , Not particularly limited.

Further, in the first embodiment, the case where the detection target is limited to a person has been described, but the detection target is not limited to this, and additional learning elements are obtained by comparing the position distribution maps between each class with the detection target as a plurality of classes. Also, by combining the position distribution maps between the classes and calculating the degree of overlap between the classes, detailed analysis of the scene data becomes possible and the image with a high degree of similarity to the actual scene. A method such as efficiently performing the search may be adopted.

Further, in the first embodiment, when comparing the learning scene and the actual scene, the position distribution map of only the detection target was used, but the position distribution map other than the detection target was created and used for the comparison between the scenes. May be good. For example, even when only a person is originally detected, a more detailed comparison between the learning scene and the actual scene can be made by creating a position distribution map of furniture, obstacles, etc. around the person and comparing it with the learning scene. It may be carried out.
Further, in the first embodiment, the information of all the detection frames and the detection frames is used when creating the position distribution map, but a process for reducing the information may be added. For example, when classifying the learning scenes, processing such as ignoring the detection frame of a certain size or less may be added so that the number of groups does not become large.

FIG. 14 is a configuration diagram of the object detection device 10 of the second embodiment.
Comparing the object detection device 10 with the object detection device 1 of FIG. 1, the object detection device 10 deletes the additional learning element output unit 18 from the object detection device 1, and has a test data generation unit 19A and a dictionary data reconstruction unit. 19B and is added.

The test data generation unit 19A generates test data based on a set of actual image data of the camera 2 output from the camera information acquisition unit 14. As the test data, in addition to each actual image data, actual scene data (hereinafter, “test scene data”) for verifying the detection accuracy of the dictionary data 12D is added by the annotation unit 11. The method of selecting the set of the actual image data is not particularly limited, such as the method of selecting so that the variance of the position distribution map becomes large.

The dictionary data reconstruction unit 19B determines the similarity between the test scene data calculated by the scene similarity calculation unit 17 and the learning scene data, and the object detection accuracy of the dictionary data 12D in the test data calculated by the dictionary adaptability acquisition unit 181. From the above, the content of the reconstruction (correction) of the dictionary data 12D is determined.
As a method of reconstructing the dictionary data 12D, there is a method of reconstructing the dictionary data 12D by presenting correction contents such as exchanging the groups (m = 1, ..., M in FIG. 13) used for the dictionary data 12D. Therefore, the dictionary data reconstruction unit 19B does not have a plurality of learning scene groups according to the position distribution map for the type of learning scene, and the generated dictionary data 12D is not generated by using all the groups. In this case, search for a group that has a high degree of similarity to the actual scene data (that is, the detection accuracy is expected to improve).

As a correction process of the learning scene data (tag data) of the dictionary data 12D, the dictionary data reconstruction unit 19B can reduce the man-hours by annotating only a part of the detection targets from the annotation unit 11. Therefore, the dictionary data reconstruction unit 19B determines the learning scene data to be annotated from, for example, customer requirements and ease of visual confirmation.
In this way, the dictionary data reconstruction unit 19B replaces the configuration of the dictionary data 12D and modifies the learning scene data of the dictionary data 12D so that a group having a high degree of similarity to the actual scene data can be searched. The dictionary data 12D is updated repeatedly. As a result, dictionary data 12D with high detection accuracy can be efficiently constructed while suppressing the man-hours for annotation work.

According to the second embodiment described above, by presenting to the user a method of modifying the learning scene so that the object detection accuracy of the dictionary data 12D is high, the dictionary data with high detection accuracy can be efficiently suppressed while suppressing the annotation work as much as possible. You can build 12D.
In Example 2, when determining the detection accuracy of the dictionary data 12D in the test data, a position distribution map of the detection target that was not detected is generated, and the dictionary data reconstruction unit 19B sets the undetected position distribution map. A method of presenting the user to add annotations to a group of learning scenes having a position distribution map having a high degree of similarity may be adopted, and highly accurate dictionary data 12D can be efficiently generated by this method.

In addition, a position distribution map of the detection target class in which the false detection occurred in the test data is generated, and the dictionary data reconstruction unit 19B is used with a group of learning scenes having a position distribution map having a high degree of similarity to the false detection position distribution map. , You may present the correction contents such as replacing the current group.
Further, in the second embodiment, the detection result by the latest dictionary data 12D may be utilized when modifying the tag data. For example, by detecting an object with the latest dictionary data 12D for the learning image and outputting the learning image with a detection frame to a GUI screen or the like, it can be used as auxiliary information when the user performs annotation work. There is a method.

Further, in the second embodiment, instead of adding tag data to the learning image in the learning image data 3, only a part of the learning images is annotated, and the dictionary data reconstruction unit 19B gradually learns. A configuration in which the number of images is increased may be adopted. Specifically, after object detection is performed on the unannotated learning image using the latest dictionary data 12D to create a position distribution map, the user is presented to add a learning image close to the position distribution map of the actual scene. There is a way to do it. By repeatedly executing this method, it is possible to efficiently generate highly accurate dictionary data 12D while suppressing the number of learning images for which annotation work is to be performed.

Further, in the second embodiment, the configuration may be such that efficient dictionary data 12D can be generated according to the customer requirements. For example, based on information such as the target accuracy and the deadline for the final dictionary data 12D, the number of repetitions of the reconstruction process of the dictionary data 12D shown in the second embodiment is increased while minimizing the number of the learning image data 3. As a result, the dictionary data 12D that achieves the target accuracy can be efficiently constructed by changing the combination of the learning scene groups used by the dictionary data reconstruction unit 19B to generate the dictionary data 12D.

The present invention is not limited to the above-described embodiment, and includes various modifications. For example, the above-described embodiment has been described in detail in order to explain the present invention in an easy-to-understand manner, and is not necessarily limited to those having all the described configurations.
Further, it is possible to replace a part of the configuration of one embodiment with the configuration of another embodiment, and it is also possible to add the configuration of another embodiment to the configuration of one embodiment.
Further, it is possible to add / delete / replace a part of the configuration of each embodiment with another configuration. Further, each of the above configurations, functions, processing units, processing means and the like may be realized by hardware by designing a part or all of them by, for example, an integrated circuit.
Further, each of the above configurations, functions, and the like may be realized by software by the processor interpreting and executing a program that realizes each function.

Information such as programs, tables, and files that realize each function can be stored in memory, hard disks, recording devices such as SSDs (Solid State Drives), IC (Integrated Circuit) cards, SD cards, DVDs (Digital Versatile Discs), etc. Can be placed on the recording medium of.
In addition, control lines and information lines are shown as necessary for explanation, and not all control lines and information lines are necessarily shown in the product. In practice, it can be considered that almost all configurations are interconnected.
Further, the communication means for connecting each device is not limited to the wireless LAN, and may be changed to a wired LAN or other communication means.

1 Object detection device 2 Camera 3 Learning image data (learning measurement data)
10 Object detection device 11 Annotation unit 12 Dictionary generation unit 12D Dictionary data (storage unit)
13 Learning scene acquisition unit 14 Camera information acquisition unit 15 Object detection unit 16 Real scene estimation unit 17 Scene similarity calculation unit 18 Additional learning element output unit 19A Test data generation unit 19B Dictionary data reconstruction unit 131 Annotation data analysis unit 132 Video analysis Part 133 Camera parameter acquisition part 134 Learning scene analysis part 181 Dictionary adaptability acquisition part 182 Additional learning judgment part 183 Additional learning element determination part

Claims (10)

A storage unit of dictionary data having a set of learning measurement data and learning scene data tagged with each learning measurement data.
An object detection unit that detects an object from the actual measurement data that is the target of object detection using the dictionary data,
An actual scene estimation unit that estimates the actual scene data to be tagged with the actual measurement data from the object detection result of the object detection unit, and
A scene similarity calculation unit that calculates the similarity between the estimated actual scene data and the learning scene data of the dictionary data, and
It has an additional learning element output unit that outputs learning element data required for additional learning of the dictionary data.
The additional learning element output unit
When the similarity calculated by the scene similarity calculation unit is higher than a predetermined threshold value, the learning element data based on the learning scene data used for the calculation of the similarity is output.
An object detection device characterized in that when the similarity calculated by the scene similarity calculation unit is equal to or less than a predetermined threshold value, the learning element data based on the actual scene data used for the calculation of the similarity is output. The learning scene data and the actual scene data each have a position distribution map in the image data extracted from the position information of the detection target in the image data and the detection frame information indicating the size.
The object detection device according to claim 1, wherein the scene similarity calculation unit calculates the similarity between the position distribution maps. The additional learning element output unit includes the position distribution map of a different object, the position distribution map of an object that the object detection unit failed to detect, and the position distribution map of an object that the object detection unit erroneously detected. The object detection device according to claim 2, wherein the learning element data is determined by using any of the above. The object detection device further groups the set of the learning measurement data similar among the position distribution maps of the learning measurement data with respect to the set of the learning measurement data of the dictionary data, and generates the set by the grouping. The object detection device according to claim 2, further comprising a learning scene acquisition unit that causes the scene similarity calculation unit to calculate the similarity using the learning scene data for each group. The claim is characterized in that the additional learning element output unit determines whether or not the dictionary data used by the object detection unit needs to be modified based on fitness, which is the accuracy with which the object detection unit detects an object. Item 2. The object detection device according to item 1. The object detection device further determines the content of the reconstruction of the dictionary data used by the object detection unit based on the adaptability which is the accuracy of detecting the object by the object detection unit. The object detection device according to claim 1, wherein the object detection device has. The scene similarity calculation unit calculates the similarity between the actual scene data and the learning scene data by using at least one of the shooting time zone information, the shooting location information, and the camera parameters as the scene data. The object detection device according to claim 1, wherein the object detection device is characterized by the above. The additional learning element output unit is at least one of image type information indicating the type of the learning measurement data required for additional learning, learning method information of the learning scene data, and dictionary type information related to the construction of the dictionary data. The object detection device according to claim 1, wherein one is output as the learning element data. An object detection system having an object detection device and a measuring device.
The measuring device is either a monaural camera, a stereo camera, or a distance sensor.
The object detection device is
A storage unit of dictionary data having a set of learning measurement data and learning scene data tagged with each learning measurement data.
An object detection unit that detects an object from the actual measurement data measured by the measuring device as an object detection target by the dictionary data, and an object detection unit.
An actual scene estimation unit that estimates the actual scene data to be tagged with the actual measurement data from the object detection result of the object detection unit, and
A scene similarity calculation unit that calculates the similarity between the estimated actual scene data and the learning scene data of the dictionary data, and
It has an additional learning element output unit that outputs learning element data required for additional learning of the dictionary data.
The additional learning element output unit
When the similarity calculated by the scene similarity calculation unit is higher than a predetermined threshold value, the learning element data based on the learning scene data used for the calculation of the similarity is output.
An object detection system characterized in that when the similarity calculated by the scene similarity calculation unit is equal to or less than a predetermined threshold value, the learning element data based on the actual scene data used for the calculation of the similarity is output. A storage unit of dictionary data having a set of learning measurement data and learning scene data tagged with each learning measurement data, an object detection unit, an actual scene estimation unit, a scene similarity calculation unit, and the like. An object detection method executed by an object detection device having an additional learning element output unit.
The object detection unit detects an object from the actual measurement data, which is the object of the object detection, by the dictionary data.
The actual scene estimation unit estimates the actual scene data to be tagged with the actual measurement data from the object detection result of the object detection unit.
The scene similarity calculation unit calculates the similarity between the estimated actual scene data and the learning scene data of the dictionary data.
When the additional learning element output unit outputs the learning element data necessary for the additional learning of the dictionary data,
When the similarity calculated by the scene similarity calculation unit is higher than a predetermined threshold value, the learning element data based on the learning scene data used for the calculation of the similarity is output.
An object detection method characterized in that when the similarity calculated by the scene similarity calculation unit is equal to or less than a predetermined threshold value, the learning element data based on the actual scene data used for the calculation of the similarity is output.

Images