JP2009064162A - Image recognition system - Google Patents

Abstract

The acquired knowledge is systematically classified and retained and utilized in a system to improve real-time recognition processing and recognition accuracy.
When teacher data is input, the database management unit 7 extracts and clusters the feature amounts, classifies them into classes, adds attribute information, and stores them in the teacher database DB2. And by using the teacher data classified for each class, the current recognizer is evaluated and updated, so that it can be adapted to various environments and targets even if the amount of accumulated knowledge increases. Can be optimized efficiently and at high speed, and highly accurate and robust recognition can be realized.
[Selection] Figure 1

Description

  The present invention relates to an image recognition system that performs recognition processing by systematically classifying acquired knowledge in a system.

  In recent years, in image recognition technology that processes image data from a camera or the like and extracts a specific target, for example, an object moving in the environment or its movement from the image, the image data is extracted in advance under the environment used by the user. Learning algorithms for online use of various recognizers have been developed in order to ensure and improve recognition accuracy in situations not anticipated by developers.

  For example, Non-Patent Document 1 discloses a technique that uses a recognizer in which various image filters are combined in a tree structure. By automatically optimizing the tree structure image filter by genetic programming, Complex image recognition is possible (automatic construction method of tree-structured image conversion; ACTIT).

Patent Document 1 discloses a technique that extends ACTIT in order to enable extraction of a specific target from a moving image, in particular, a specific target with temporal change or displacement. In the technique of Patent Document 1, the processing structure of the tree structure image filter can be automatically acquired by genetic programming by providing teacher information, the system adapts to the environment used by the user, and is recognized with high accuracy and robustness. Is possible.
JP 2006-178857 A Shinya Aoki, 1 other person, “Automatic construction method of tree-structured image conversion ACTIT”, The Journal of the Institute of Image Information and Television Engineers, The Institute of Image Information and Television Engineers, 1999, Vol. 53, No. 6, p. 888-894

  In the above-described technology, images learned in the past and recognition processing are held in the system and are used in the current situation to perform adaptive recognition for the environment. For this reason, when learning various driving environments, the accumulated amount of recognition processing learned in the past becomes enormous, and enormous calculation is required when utilizing in the current state.

  Therefore, learning various environments and storing them unambiguously will not only be an obstacle to efficient utilization, but also a huge amount of knowledge (recognition processing) accumulated in the past for characteristic scenes. May apply average knowledge, and is not always effective.

  The present invention has been made in view of the above circumstances, and an image recognition system capable of systematically classifying acquired knowledge and maintaining and utilizing the acquired knowledge to improve real-time recognition processing and recognition accuracy. It is intended to provide.

  In order to achieve the above object, an image recognition system according to the present invention is an image recognition system that recognizes image data using a recognizer, and recognizes recognition processing acquired by learning and teacher information used for learning for each class. A database unit that classifies and holds, and a learning update unit that evaluates the recognizer using the teacher data for each class and adaptively learns and updates the recognizer.

  According to the present invention, acquired knowledge can be systematically classified and retained and utilized in the system, and real-time recognition processing and recognition accuracy can be improved.

  Embodiments of the present invention will be described below with reference to the drawings. 1 to 13 relate to an embodiment of the present invention, FIG. 1 is a basic configuration diagram of an image recognition system, FIG. 2 is an explanatory diagram showing an application example to a person extraction problem, and FIG. 3 is a tree-structured image filter 4 is an explanatory diagram illustrating integration of recognizer outputs, FIG. 5 is an explanatory diagram illustrating a flow of learning processing, FIG. 6 is a block diagram illustrating a configuration of a database management unit, and FIG. 7 is an image after filtering. FIG. 8 is an explanatory diagram showing classification according to a self-organizing map, FIG. 9 is an explanatory diagram showing evaluation of an integrated image, FIG. 10 is an explanatory diagram of replacement selection, and FIG. 11 is an explanatory diagram of sequential learning. FIG. 12 is an explanatory diagram showing the overall processing flow, and FIG. 13 is an explanatory diagram showing an example of processing.

  The image recognition system according to the present invention adaptively updates the recognizer currently used according to the environment while processing the image data input online by the recognizer, and can be more suitable for various environments and objects. It is intended to build a highly accurate and robust system. That is, the size and number of the recognizers are limited due to the relationship between processing time, memory space, and the like, and the required recognizer performance varies depending on the weather and environment.

  Even in such a situation, the learning results and recognition results of the image data input in the past are accumulated in the database held by the system, and the recognizer is updated online using the learning data accumulated in this database. It can learn adaptively according to various environments and objects, and can realize highly accurate and robust recognition.

  However, by learning various driving environments, the amount of recognition processing learned in the past becomes enormous, and the optimization calculation becomes enormous when using it in the current state. It can be an obstacle. For this reason, in the image recognition system of the present invention, the acquired knowledge (recognition processing) is systematically classified and retained and utilized in the system, thereby enabling real-time recognition processing to be performed efficiently. Yes.

  Note that the image data here includes not only visual information data captured by an image sensor such as a camera but also data in a pseudo image form in which a two-dimensional distribution of an object is detected by a laser radar or the like.

  As shown in FIG. 1, the image recognition system 1 in this embodiment integrates the recognition processing unit 2 that processes input image data in parallel by a plurality of recognizers 5, and outputs of the recognizers 5. Consists of an integration unit 3, a learning unit 4 that learnably updates a recognizer using teacher data that is a processing target, and a database unit 6 that systematically classifies and holds acquired knowledge (recognition processing). Has been.

  The configuration of the learning unit 4 will be described in detail. The learning unit 4 includes a recognizer evaluation unit 10 that evaluates each recognizer, and all recognizers (a currently used recognizer and a stock recognizer). A replacement selection unit 11 that obtains an optimal combination from among them and replaces the combination of the currently used recognizers with the optimal combination, and a sequential learning unit 12 that newly creates a recognizer based on teacher data are configured. Yes.

  Further, the database unit 6 stores, in detail, a recognizer database DB1 for storing a recognizer created in the past and a newly created recognizer, teacher data input in the past, and teacher data newly input. And a database management unit 7 that systematically classifies and manages the knowledge stored in each data DB1, DB2.

  Below, the example which mounts the image recognition system 1 in vehicles, such as a motor vehicle, processes the moving image from a vehicle-mounted camera, and extracts a pedestrian is demonstrated. As shown in FIG. 2, this is an application example to a person extraction problem of extracting a person appearing in regions QR1, QR2, QR3 indicated by broken lines from moving images Q1, Q2, Q3 of different scenes.

  As an in-vehicle camera that captures an input image, for example, a camera having an image sensor such as a CCD or a CMOS is used, and is disposed as an in-vehicle camera inside a windshield in the vicinity of a room mirror in a vehicle interior. With this in-vehicle camera, the front of the vehicle is imaged every predetermined time period (for example, 1/30 sec), and after a video process such as noise removal, gain adjustment, γ correction, etc., a predetermined gradation (for example, 256 gradation gray) An input image converted into a digital image of scale) is input to the recognition processing unit 2.

  Note that the image at the current time t and the previous time (t−k) is read from the memory and input to the recognition processing unit 2 every M frames. The values of k and M can be set as appropriate, and a plurality of different types of input images can be selected and input by other selection methods.

  The recognition processing unit 2 processes input images in parallel by a plurality of recognizers 5,..., And outputs a processed image obtained by extracting a target. In this embodiment, since the processing purpose is extraction of pedestrians from a landscape image in front of the vehicle, an image in which only pedestrians are extracted from the input image is output.

  As the recognizer 5, in this embodiment, as shown in FIG. 3, a tree-structured image filter in which a plurality of image filters F1, F2,..., Fn (n = 8 in the figure) are combined in a tree structure. Is adopted. As the image filter that becomes each node of this tree structure, there are used various existing image filters (for example, an average value filter, a Sobel filter, a binarization filter, etc.) and an image filter specialized in function according to the purpose. The optimal combination and total number of these image filters are genetic programming (GP) that extends the genetic algorithm (GA) genotype to handle structural representations (tree structure, graph structure, etc.) Obtained by learning through Genetic Programming).

  The recognizer 5 includes a tree-structured image filter, a neural network, a support vector machine, a recognizer using fuzzy, a recognizer that performs stereo image matching processing, a recognizer that processes a scanned image by a laser radar, and the like. It is also possible to use.

  Details of image processing by the tree-structured image filter employed in this embodiment are described in detail in Japanese Patent Application Laid-Open No. 2006-178857 by the present applicant. Here, the outline will be described.

  In the tree-structured image filter according to this embodiment, the tree structure is optimized through the following fitness evaluation, selection, crossover, mutation, fitness evaluation, and end determination processes, and is automatically generated by the GP. The processing program realizes an optimal conversion process from the original image to the target image.

[Evaluation of fitness]
Using the tree-structured image filter as an individual, the fitness of each individual in the randomly generated initial individual population is evaluated. The fitness is defined by the similarity between the image output from each individual and the target image, and is calculated using the following equation (1). Each individual is limited so that the number of terminal nodes constituting the tree structure does not exceed a preset maximum value (for example, 40) in the evolution process until optimization.
K = 1.0- (1 / R) · Σ f (Σ p W · │O-T│ / Σ p W · V) ... (1)
Where Σ _{f is} the sum of the number of frames f
Σ _p : Sum of pixels in one frame
K: Fitness
R: Number of learning sets (a combination of input images and teacher images as learning sets
Number of sets used for evaluation)
O: Output image
T: target image (image to be output by optimized processing)
W: Weighted image (represents the importance of the area in the target image,
An image in which the weight corresponding to the distance between the output image and the target image is defined for each pixel)
V: Maximum gradation

[Choice]
This is a process of selecting a parent group for replication of individuals, and selection and growth of individuals to be left in the next generation are performed based on the fitness K by methods such as roulette selection, expected value selection, ranking selection, tournament selection, and the like. In the tree-structured image filter of this embodiment, a set number of individuals are selected by selecting a tournament, and the elite of the individual having the maximum fitness K is simultaneously stored.

[Crossover, mutation]
This is a process of generating a child group by crossover and mutation from a parent group. Pair each selected individual, select each crosspoint at random, and perform crossover by one-point crossover, multipoint crossover, uniform crossover, etc. A child group is generated by crossing subtrees according to points. The generated child population is subjected to node mutation, insertion, deletion, etc. at a predetermined ratio for each individual, and a child population is generated by the mutation.

[Evaluation of fitness, end judgment]
Each of the individuals generated by the mutation is evaluated for the fitness described above, and the end of the optimization process is determined including the individual with the maximum fitness of the previous generation stored in elite. The end of this process is determined by whether or not the maximum number of generations to be executed has been reached, whether or not there is an individual that has reached a preset target fitness (whether or not the target individual has been obtained), etc. The

  When the number of generations has not reached the number of end generations, the process returns to the parent selection and the above processing steps are repeated. On the other hand, when the number of generations reaches the number of end generations, or when the maximum fitness value does not change during the predetermined number of generations, that is, when the maximum fitness value stagnates, To cancel the optimization and output the individual with the maximum fitness as a solution.

  The above tree structure optimization is performed in advance in offline pre-learning in order to deal with various scenes. Typical scenes such as daytime, nighttime, weather, environment (highways, highways, urban areas, etc.) As a specialized recognizer, it is stocked for each class to be described later in the recognizer database DB1.

  In the following description, the tree-structured image filter is appropriately described as “tree-structure filter row” or simply “tree”.

  Normal input image processing in the image recognition system 1 is executed by the recognition processing unit 2 and the integration unit 3, and a target is extracted from input images that are always sent online. That is, when an input image is processed in parallel by a plurality of tree structure filter strings of the recognition processing unit 2, the parallel outputs are averaged and integrated by the integration unit 3, and an integrated image is output as a recognition result.

  For example, as shown in FIG. 4, when an original image serving as input data is processed by four tree structure filter trains TR1, TR2, TR3, TR4, the original image is processed by each tree structure filter train TR1, TR2, TR3, TR4. An output weight Wi (i = 1, 2, 3, 4) is set for each of a plurality of output images, and an image integrated with the output weight Wi is output.

As shown in the following equation (2), the nth pixel value Pn in the integrated image is a pixel value PAn, PBn, PCn, corresponding to the output image from each tree structure filter array FA, FB, FC, FD. PDn is given as a weighted average value with output weights W1, W2, W3, and W4. Details of the output weight Wi will be described in the recognizing device replacement selection process in the learning unit 4 below.
Pn = (PAn × W1 + PBn × W2 + PCn × W3 + PDn × W4) / 4 (2)

  On the other hand, the learning unit 4 triggers the input of teacher data, as shown in FIG. 5, separately from the processing of the recognition processing unit 2 and the integration unit 3 for recognizing a target from input images that are always sent online. As described above, a process for adaptively updating the currently used recognizer according to the environment is executed in the background. In FIG. 5, a thick arrow line indicates the flow of the learning process, and a broken arrow line and a thin arrow line indicate the flow of the learning image and the recognizer, respectively.

  Schematically, when teacher data is created from input data, the teacher data is classified into classes by the database management unit 7 and stocked in the teacher database DB2. Then, using the teacher data classified for each class, the recognizer evaluation unit 10 evaluates the currently used tree structure filter string and the tree structure filter string stocked in the recognizer database DB1 for each class. .

  The evaluation result of the tree structure filter sequence in each class is referred to by the replacement selection unit 11, and the optimum combination of the tree structure filter sequences is determined. The optimal combination of tree structure filter sequences is based on the premise that a better evaluation can be obtained than the integration result of the current tree structure filter sequence forming the recognition processing unit 2, that is, a plurality of tree structure filter sequences currently used. Yes, as an absolute condition, it is necessary that the evaluation is not worse than the current combination of tree structure filter sequences.

  When there is no candidate tree structure filter sequence to be used, the sequential learning unit 12 creates a new tree structure filter sequence by learning using GP, which is the evolutionary optimization method described above (sequential learning). . Then, a combination including the tree structure filter sequence sequentially added by the sequential learning is repeatedly evaluated, and a plurality of tree structure filters of the current recognition processing unit 2 are determined by a combination of the optimum tree structure filter sequences finally determined. The columns are replaced partially or completely.

  Hereinafter, management of teacher data and knowledge data by the database management unit 7 will be described prior to detailed description of processing of the learning unit 4.

  The database management unit 7 uses, as input data, images taken during traveling, information on vehicle operations and vehicle states obtained via the in-vehicle network, images learned in the past and learning in the databases DB1 and DB2. The recognition process obtained by the above is appropriately managed, and information for efficiently controlling the replacement selection unit 11 and the sequential learning unit 12 is sent.

  As described above, each database DB1 and DB2 stores a large amount of learning results obtained by learning various driving environments, so that appropriate measures are required to efficiently utilize the current state. . For this reason, as shown in FIG. 6, the database management unit 7 includes functional units such as a feature amount extraction unit 7a, a teacher map creation unit 7b, an attribute setting unit 7c, and a class determination unit 7d. Acquired knowledge (recognition processing) is systematically classified in the system, enabling real-time recognition processing to be performed efficiently.

  When a teacher image created from risk information experienced during traveling is input, the feature amount extraction unit 7a extracts a feature amount from the teacher image. That is, feature amounts such as edge information, motion information, and color information (brightness, saturation, hue) are extracted from the teacher image, and the information is held as an N-dimensional vector. The N-dimensional vector can include vehicle information other than the image feature amount, for example, information such as a change in vehicle speed and yaw angle.

  The feature amount extraction in this case is data extraction for subsequent recognition. In general, data that is not correlated with the target recognition adversely affects the recognition. That is, in this feature quantity extraction process, it is not a good idea to increase the feature quantity unnecessarily, and conversely, not using a necessary feature quantity also deteriorates accuracy.

  For this reason, feature quantity selection as to which feature quantity should be used occurs as a problem, and if feature quantity selection here is performed in a learning manner, higher learning of the subsequent recognition processing is required, and the calculation amount / memory capacity This is disadvantageous for online learning. Therefore, in this embodiment, an example will be described in which the feature amount extraction portion is treated as fixed.

  In addition, when performing feature selection in a learning manner, it is only necessary to evaluate the recognition rate of the system as a reference and optimize the combination of each feature amount. Existing optimization methods such as simple search methods can be used.

  In the present embodiment, feature types of preset types are extracted. For example, the process is divided into a plurality of elements, and feature amounts set for each element are extracted. As the plurality of elements, preprocessing, feature amount calculation, region setting, and the like can be used. As shown below, 6 types of data can be extracted in the pre-processing, 10 types in the feature amount calculation, and 4 types in the region setting, and a total of 240 (6 × 10 × 4) dimensional data can be obtained by combining them. Can be extracted.

<Pretreatment>
Six-dimensional feature data can be extracted by performing six types of filter processing on the input image, sobel, vertical sobel, horizontal sobel, inter-frame difference, luminance, and saturation.

<Feature amount>
By performing 10 types of calculation processing on the pixel values of the filtered image, average, variance, maximum value, minimum value, horizontal centroid, vertical centroid, contrast, uniformity, entropy, fractal dimension, 10-dimensional feature data can be extracted.

<Area>
An area is set in the image, and four-dimensional feature amount data can be extracted for the entire set area, the left area in the set area, the right area, and the central area.

  The above 240-dimensional feature values may be narrowed down according to the calculation performance of the online system. In addition to images, vehicle data may also be used to extract 6-dimensional feature values such as the average and variance of the Sobel for the entire screen, the average of inter-frame differences for the entire screen, the variance, the vehicle speed, and the steering wheel angle. .

  In the above feature quantity extraction process, each feature quantity is normalized, but the theoretical range is inefficient. Therefore, the distribution of each feature quantity is evaluated in advance, and the evaluation result is used as a basis. The maximum value and the minimum value are set to, and normalized to a numerical value of 0 to 1. In that case, the maximum and minimum values may be changed dynamically. For example, when a value exceeding the maximum value or a value below the minimum value is input, the maximum value is expanded so that the range is expanded.・ Change the minimum value. Conversely, if the data near the minimum and maximum values has not been entered for a while, the range is changed to narrow.

  Although basic feature values are used here, it is also possible to calculate time-series fluctuations of feature values, such as calculating motion information using past frame information, and use the information as feature values. . Furthermore, in order to improve the accuracy of risk recognition as a whole, high-accuracy image processing can be added to this feature amount extraction processing. For example, past pedestrian recognition results, road white line recognition results, obstacle recognition You may make it incorporate in extraction data here including a result.

  When the feature amount is extracted from the teacher image as described above, the teacher map creating unit 7b classifies the feature amount space together with the teacher image obtained in the past to classify the class by class, and creates a map of the teacher image for each class. To do. Here, clustering is performed using a self-organizing map (SOM), which is a type of neural network that models the visual cortex of the cerebral cortex.

  In the SOM, units arranged in the M dimension (usually two dimensions) each have a vector value (referred to as a connection weight with the normal input), and the winner unit is determined with respect to the input based on the vector distance. The reference vector values of the winner unit and its surrounding units are updated so as to approach the input vector. By repeating this and learning so that the entire distribution of the input data can be optimally expressed, a space having a high data density represented by a representative vector is classified as a class.

  For example, after filtering a teacher image using a Sobel filter, 1 × 2 types of image feature amounts are extracted from one image using the average and variance of pixel values in the image, and as shown in FIG. When image feature amounts are extracted from 34 consecutive frames of images, if these image feature amounts are clustered by SOM, they can be classified into classes A, B, C, and D as shown in FIG.

  The classification of teacher images by SOM clustering is not deterministic and needs to be adaptively updated in accordance with various driving environments. For this reason, the teacher map creation unit 7b autonomously updates the clustering while traveling, and adaptively classifies the classes with respect to changes in the feature amount due to changes in the environment and time.

  After that, when the class to which the input teacher image belongs is determined, the attribute setting unit 7c adds attribute information to the teacher image and transfers it to the teacher database DB2. The attribute information added to the teacher image mainly includes the class to which the teacher image belongs and the distance from the center of the class to which the teacher image belongs, in addition to the distance from the center of another class in the feature amount space, and the learning set used. Add the average value of class attributes.

  When clustering by SOM is performed using a probabilistic model, probabilistic information according to the distance from the center of the class may be added as attribute information.

  Based on the teacher images classified for each class as described above, the recognizers are replaced for each class, and a database of recognizers corresponding to the teacher images for each class is formed in the recognizer data DB2. Next, the real-time replacement process of the recognizer will be described.

  The replacement of the recognizer is started when input data is input to the database management unit 7. First, the class determination unit 7d of the database management unit 7 determines which class the input data belongs to. To determine the class to belong to, the input data is projected onto the feature amount space obtained by the feature amount extraction unit 7a, and the class corresponding to the input data is determined from the map of the teacher image. Then, a recognizer having the determined class attribute is selected and replaced in real time.

  For example, when the feature quantity of the input data is the distance from the center of the class A with respect to the classes A, B, C, and D shown in FIG. 8, the class of the input data is A. The classifier recognizer (tree structure filter string) stocked in the recognizer database DB1 is used to replace the currently used recognizer (tree structure filter string).

  In this case, the class is not necessarily limited to a single class, and learning samples are extracted for a plurality of classes having a short distance in the feature amount space, and an average recognition process is performed in a plurality of scenes. May be.

  In the recognizer replacement process, first, the recognizer evaluation unit 10 of the learning unit 4 individually evaluates the currently used tree structure filter string and the tree structure filter string of the corresponding class in the recognizer database DB1. In evaluating the tree structure filter string, the tree structure filter string in the target class may be arranged in the recognizer database DB1, and the tree structure filter string having a low evaluation may be deleted.

  Specifically, image evaluation values of individual tree structure filter sequences are obtained using the teacher data, and further, evaluation is performed in consideration of the following conditions (a) to (d) in addition or selectively. . As the image evaluation value of the tree structure filter row, a value according to the fitness K in equation (1) can be used.

(A) Life (current time-time of creation) is the life of the tree, and the younger tree that has been recently made has a higher evaluation value.
(B) Number of uses Trees that have been used in the past have a high evaluation value.
(C) Size The smaller the tree, the higher the evaluation value.
(D) Usage status Evaluation of a currently used tree is higher than that of a tree used in the past.

For example, when evaluating a tree in consideration of the image evaluation value G, the life L, the number of uses S, and the use state TS, the evaluation value F can be obtained by the following equation (3).
F = G × α + L × β + S × γ + TS × δ (3)
Where α, β, γ, δ: constants

  The obtained evaluation value goes back in the past, and the accumulated value becomes the current evaluation value. As soon as the evaluation of all the tree structure filter columns in the class is completed, the processing of the replacement selection unit 11 is performed.

  The replacement selection unit 11 obtains a combination of N trees having the highest evaluation from all trees including the currently used tree and the tree stocked in the class. If the number of combinations N is less than a certain number M, a new tree is created by sequential learning and added, and when N = M, the tree group that has always processed the input data is new. Replace with a group of trees.

  The certain number M is the number of tree structure filter columns forming the recognition processing unit 2 (the number of tree structure filter columns used constantly). For example, a total of 20 tree structure filter columns are stocked in the recognizer database DB1. If the number of combinations is limited, the number of combinations is limited by the number of classes, and an optimal combination of up to 10 trees is obtained as a tree that is always used. Accordingly, efficient replacement selection can be performed in constructing a highly accurate recognition process corresponding to a traveling situation.

  When replacing a group of trees, the evaluation result of the integrated image based on the currently used tree combination is used as a reference. That is, as shown in FIG. 9, the original image, which is new teacher data, is integrated by parallel processing in the current tree group TR, the integrated image is compared with the target image, and the evaluation result is used as a reference. Judge whether to replace the new group of trees.

  Further, when an optimum tree is combined, evaluation is performed using an integrated image of the combined tree group. For example, as shown in FIG. 10, the classes corresponding to the recognizer database DB1 include trees TR1, TR2, TR3, TR4, and two trees TR1, TR2 from the trees TR1, TR2, TR3, TR4. Is selected, the integrated image created using the trees TR1 and TR2 is compared with the target image to calculate an evaluation value. If the calculated evaluation value is higher than the evaluation values of other combinations, the trees TR1 and TR2 are selected. If the calculated evaluation value is lower, the other trees are selected and evaluated in the same manner. By repeating such processing, all combinations are evaluated, and the combination having the highest evaluation is obtained.

For evaluation, the evaluation value is calculated using the formula defined below.
[Evaluation methods]
The evaluation value is defined by the similarity between the integrated image created by the new group of trees and the target image, and is calculated using the following equation (1) ′.
_{K = 1.0-Σ f (Σ} p W · │O-T│ / Σ p W · V) ... (1) '
Where Σ _{f is} the sum of the number of frames f
Σ _p : Sum of pixels in one frame
K: Evaluation value
O: Integrated image
T: Target image (image to be output by optimized processing)
W: Weighted image (represents the importance of the area in the target image,
An image in which the weight corresponding to the distance between the integrated image and the target image is defined for each pixel)
V: Maximum gradation

It should be noted that it is also possible to optimize the strength of the output of each tree at the same time as selecting the optimum one among the combinations of which trees to use. The strength of the output can be optimized by determining the output weight Wi described in the above equation (2) with reference to the evaluation value of each tree. For example, if the output weight for the output image (pixel value) PAn of the tree TR1 is [0.3] and the output weight for the output image (pixel value) PBn of the tree TR2 is [0.8], The n-th pixel value Pn becomes the value of the following expression (2) ′, and the evaluation value can be obtained from the integrated image with the output weight as described above.
Pn = (PAn × 0.3 + PBn × 0.8) / 2 (2) ′

  In this case, the combination of the output weight and the tree is a number obtained by raising the [weight type] to the power of [number of trees]. For example, the output weight candidates are [0], [0.3], [0.8]. , [1.0], and there are two trees, there are a total of 16 combinations of output weights and trees. The evaluation values are obtained for these 16 types, and the combination having the maximum evaluation value is obtained. Will be asked. The actual output weights are set to 10 types in increments of 0.1 from 0 to 1.

  In the replacement selection unit 11, all combinations of tree structure filter sequences are evaluated, and when the number N of tree groups that are optimal combinations is less than a certain number M, sequential learning in the sequential learning unit 12 is executed.

  The sequential learning unit 12 further corrects the output result of the optimal combination of N trees selected by the replacement selection unit 11, and sequentially learns until the optimal combination tree number N reaches a certain number M. And add trees.

  As a learning flow, for example, as shown in FIG. 11, if the combination selected by the replacement selection unit 11 is the trees TR1 and TR2, the tree is determined from the difference between the integrated image of the trees TR1 and TR2 and the target image. Weighting is performed on the location where TR1 and TR2 are wrong, and an image (corrected weight image) is created by weighting the wrong location as a correction point.

  For example, among the values of the target image, the luminance value 255 (most important) is an area where the person is instructed to be a person, an incorrect portion is compared with the luminance value 127 (important) when comparing the integrated image and the target image, and the other areas Is set to a luminance value of 1 (somewhat important), and a correction weight image is created. Then, a new tree TR3 'is created using the created correction weight image and added to the tree structure buffer.

  Next, an integrated image of the trees TR1, TR2, and TR3 'is obtained, and it is determined whether or not a new tree TR3' is to be added based on the evaluation value of the integrated image with respect to the target image. If the evaluation value exceeds the threshold value, as shown in FIG. 11, a tree TR3 ′ is added to form a new combination tree group TR1, TR2, TR3 ′. The learned tree TR3 ′ is not added, and learning is sequentially repeated. That is, similarly, a modified weight image is created, another new tree TR4 is created, and an integrated image based on a combination of the trees TR1, TR2, and TR4 is evaluated. Add trees until

  In practice, the fixed number M is set to 10 and trees are added until 10 trees are selected by the replacement selection. When the number N of trees reaches M, the sequential learning is terminated, and the tree group that has always processed the input data is replaced with the new tree group that has been created.

  In addition, the new tree is not only a tree that has been evolved as an initial individual by GP (genetic programming), but also a tree that is currently being used as an initial individual. The probability of selection at the time of calculation is set by the class attribute information, and the search is performed centering on the scene corresponding to the input teacher image, so that efficient replacement selection can be performed.

  The overall processing flow will be described with reference to FIG. As shown in FIG. 12, when the original image is input as new teacher data, it is processed in parallel by the M combination recognizers (tree structure filter train) of the current combination in the recognition processing unit 2, and the respective output results are displayed. Integrated. Q1 'in FIG. 13 is an example of an original image, and an image obtained by processing and integrating the original image Q1' with a recognizer is Q2 '. In this integrated image Q2 ', it can be seen that the recognizer currently used does not extract any person from the new teacher data.

  At this time, the class to which the original image belongs is determined by the database management unit 7, the classifier corresponding to the classifier database DB 1 and the currently used classifier are evaluated by the classifier evaluation unit 10, and then the replacement selection unit 11. To determine a new combination of recognizers, select N recognizers, and evaluate the integrated image. Q3 ′ in FIG. 13 shows an integrated image when a new combination of three tree structure filter rows is selected. In this integrated image Q3 ′, a person is extracted, but there is an erroneous extraction in the background. I understand that.

  This background erroneous extraction is learned so as to correct the image by the sequential learning in the sequential learning unit 12, and an integrated image as indicated by Q4 'in FIG. 13 is obtained. In the integrated image Q4 'in FIG. 13, it can be seen that background extraction is reduced while people are extracted. When the number of combinations of finally recognized recognizers reaches M after repeating this sequential learning, the current recognition processing unit 2 is updated with a new combination of recognizers to eliminate erroneous background extraction. can do.

  As described above, the image recognition system according to the present embodiment systematically classifies learning results and recognition results of image data input in the past in the system, accumulates them in a database, and stores the learning data accumulated in the database. Is used to update the recognizer online. As a result, even when the amount of accumulated knowledge increases, optimization can be performed efficiently and quickly for adaptive learning according to various environments and objects, and highly accurate and robust recognition is possible. Can be realized.

Basic configuration of image recognition system Explanatory drawing showing an application example to the person extraction problem Explanatory drawing showing a tree-structured image filter Explanatory diagram showing integration of recognizer outputs Explanatory diagram showing the flow of the learning process Block diagram showing the configuration of the database manager Explanatory drawing which shows the image feature-value after filtering Explanatory diagram showing classification by self-organizing map Explanatory drawing showing evaluation of integrated image Illustration of replacement selection Illustration of sequential learning Explanatory diagram showing the overall process flow Explanatory drawing showing an example of processing

Explanation of symbols

DESCRIPTION OF SYMBOLS 1 Image recognition system 2 Recognition processing part 3 Integration part 4 Learning part 5 Recognizer 6 Database part 7 Database management part 10 Recognizer evaluation part 11 Replacement selection part 12 Sequential learning part DB1 Recognizer database DB2 Teacher database

Claims (7)

An image recognition system for recognizing image data using a recognizer,
A database unit that classifies and holds recognition processing acquired by learning and teacher information used for learning for each class;
An image recognition system comprising: a learning update unit that evaluates the recognizer using teacher data for each class and adaptively learns and updates the recognizer. In the database section above,
The image according to claim 1, further comprising: a management unit that clusters learning images in a multi-dimensional feature amount space obtained from input data, classifies the learning images for each class, and sets attribute information for each classified class. Recognition system.   The image recognition system according to claim 2, wherein the clustering is updated online by autonomous learning.   The image recognition system according to claim 2, wherein the clustering is performed using a self-organizing map.   The image recognition system according to claim 2, wherein the attribute information includes information corresponding to a distance from the class in the feature amount space.   3. The image recognition system according to claim 2, wherein a learning sample at the time of recognition processing is extracted from a plurality of classes having a short distance in the feature amount space, and average recognition processing is performed on a plurality of scenes. A plurality of the above recognizers are provided.
In the learning update part,
A sequential learning unit that sequentially learns the integration results of the plurality of recognizers and creates a new recognizer;
A replacement selection unit that obtains an optimal combination from the class recognizers based on the attribute information including the recognizer created by the sequential learning and selectively replaces a plurality of recognizers currently used. The image recognition system according to any one of claims 2 to 6, wherein

Images