JP2018106618A - Image data classifying apparatus, object detection apparatus, and program therefor - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide an image data classification device which has versatility and enables more robust and accurate classification of image data, an object detection device which detects an object from image data more robustly and with high accuracy, and a program thereof. .. SOLUTION: An image data classification device 1 of the present invention is constructed according to a learning processing unit 2 that learns and constructs a decision tree from learning data prepared in advance by using a multi-scale convolution filter, and a learned decision tree. The identification processing unit 3 for classifying the input image to be identified by using the multi-scale convolution filter is provided. The object detection device 10 of the present invention is a determination process for determining the presence or absence of an object in an image of a scanning window based on the classification results of the image data classification device 1 and the image data classification device 1 according to the present invention, and the object. It is provided with classification result determination means 12 and 13 that execute a feature point selection process for selecting a feature point that becomes an image feature when there is an image in parallel. [Selection diagram] Fig. 1

Description

  The present invention relates to a technique for classifying image data, and in particular, an image data classifying apparatus that classifies images by machine learning, and a predetermined object (face, person, vehicle, etc.) in image data using the image data classifying apparatus. The present invention relates to an object detection device capable of detecting (object) of the above and a program thereof.

  In general, in order to classify image data, a classification technique based on a decision tree constructed by machine learning is often used. A decision tree is a technique for classifying input data based on if-then rules.

  In particular, at each node of the decision tree for classifying still image image data, a predetermined feature amount is calculated for the input image data (input image), and first two input images having the calculated feature amount are obtained. A node for separation. Then, the node is branched depending on whether or not the calculated feature value is larger than the node threshold value for separating the feature value into two. In the decision tree, this branch is repeated, and the classification result of the finally reached leaf node is determined as a label for the input image. The label is a label indicating a classification result of an object to be detected (an object such as a face, a person, or a vehicle).

  Here, a learning procedure of machine learning for constructing a decision tree will be described. As learning data for machine learning, an image group to which a correct label is assigned (positive example) and an image group to which no correct label is assigned (negative example) are prepared in advance. There are various machine learning algorithms for constructing a decision tree, such as ID3 and CART. Note that there are a plurality of correct answer labels or incorrect answer labels such as label 1, label 2,.

  For machine learning, various types of separation feature quantity groups (hereinafter referred to as “feature quantity pools”) for separating positive examples from negative examples and classifying positive examples are also prepared in advance. Is done. Based on the feature amounts in the feature amount pool, the image features of the positive and negative images (feature amounts that characterize each image) are calculated. In the same manner, the feature quantity characterizing the input image is calculated based on the feature quantity in the feature quantity pool for the input image to be classified using the decision tree.

  In order to construct a decision tree by machine learning, a feature amount that can best separate learning data (more precisely, image features of learning data) is selected from the feature amount pool, and is branched by a node threshold. It repeats in order so that the branched node may be further branched. The node threshold is determined for each node each time in order to separate the node to be separated into two.

  This branching is performed until the number of learning data belonging to the separation determination target node is equal to or lower than a predetermined threshold value, or until the separation accuracy of learning data in the separation determination target node is equal to or lower than a predetermined threshold value (that is, separation is performed). Repeat until accuracy is no longer desired. In addition, the quality of data separation is determined and a Gini coefficient, information gain, and the like are often used.

  By the way, in order to construct a decision tree with high separation accuracy, it is important what feature quantities (including feature quantities in the feature quantity pool and feature quantities that become image features) are used for branching nodes. It becomes.

  As a conventional technique, a technique is disclosed in which an input image is divided into two small regions, and a feature value is obtained by subtracting the sum of pixels in the second region from the sum of pixels in the first region. (For example, refer nonpatent literature 1). In Non-Patent Document 1, this feature amount is referred to as a Haar-like feature, and FIG. 1 in Non-Patent Document 1 shows an example of the Haar-like feature. A value obtained by subtracting the sum of the pixels in the white small area from the sum is used as the feature amount. In Non-Patent Document 1, a feature amount pool is obtained by changing the position and size of this small region in various ways.

  Also, a plurality of regular coordinate points (pixel coordinates) are assigned to the input image in advance so that they can be finely adjusted, two coordinate points are selected from the plurality of coordinate points (face feature points), and the selected two coordinate points Is disclosed as a feature amount (for example, see Non-Patent Document 2). FIG. 9 in Non-Patent Document 2 shows an example of the feature amount, and a feature amount pool is obtained by changing various combinations of two coordinate points to be selected. Non-Patent Document 2 proposes that a plurality of coordinate points that can be finely adjusted be defined in a local coordinate system rather than in an absolute coordinate system. Note that face feature point detection for image data can be used for person recognition.

P. Viola and M. Jones, “Robust Real-time Object Detection”, Technical Report Series, CRL 2001/1, February 2001. X.Cao, Y.Wei, F.Wen, and J.Sun, “Face Alignment by Explicit Shape Regression”, In Proc. CVPR, 2012.

  Non-Patent Document 1 proposes a feature amount designed exclusively for the purpose of detecting a region in which a face is reflected. Non-Patent Document 2 proposes a feature amount designed exclusively for the purpose of detecting a plurality of coordinate points (face feature points) from a face image.

  Since these conventional techniques are feature quantities designed exclusively for the purpose, they are not versatile and difficult to use for classification of image data other than the purpose.

  In general, image data classification applications include various object detection applications such as vehicle detection, vehicle feature point detection, or a combination of these in addition to face detection and face feature point detection. Setting the feature amount to be reduced is not versatile.

  Furthermore, in order to detect the presence / absence of a face in an input image and to detect a plurality of coordinate points (face feature points) from the face image by using these conventional techniques, first of all, Non-Patent Document 1 It is conceivable to detect a plurality of coordinate points (facial feature points) from the face image based on the technique of Non-Patent Document 2 for the input image after performing face detection based on this technique. I cannot say that.

  In addition, input images that are the targets of such face detection and facial feature points generally have a variety of noises and face orientations (deformation of face images). First, it is required to improve the accuracy of face detection. However, the performance of face detection by the technique of Non-Patent Document 1 is not sufficient from the viewpoint of practicality.

  For this reason, even if the input image has a variety of noise and the orientation of the object to be detected, it has versatility to enable robust object detection and to efficiently acquire the object feature points. Therefore, there is a demand for image data classification that enables image data to be classified more robustly and accurately, and an object detection technique that detects an object from image data more robustly and with high accuracy.

  In view of the above-described problems, an object of the present invention is to provide an image data classification apparatus that can classify image data with more versatility and more robustly, and an object that detects an object from image data more robustly and with high precision. It is in providing a detection apparatus and these programs.

  The image data classification device of the present invention is an image data classification device for classifying image data, and a learning processing unit that learns and constructs a decision tree from learning data prepared in advance using a multiscale convolution filter, An identification processing unit that classifies an input image to be identified using the multiscale convolution filter according to the learned decision tree.

  Further, in the image data classification device of the present invention, the learning processing unit includes a plurality of types of one or more reference coordinate points and a plurality of types of filter coefficients that are predetermined for each filter size. Feature amount pool means for holding a convolution filter and a plurality of predetermined filter sizes as a feature amount pool, and for each of a plurality of input learning data, the plurality of types of filter sizes according to the feature amount pool Multi-scale convolution filter processing using the plurality of types of convolution filters according to the plurality of convolution filters, and a plurality of convolution filters corresponding to the number of the plurality of types of convolution filters for each of the one or more reference coordinate points for each learning data. First convolution filter processing means for obtaining a convolution value of the one or more, the one or more reference coordinate points for each of all learning data, Based on the combination information of the plurality of types of convolution filters and the corresponding convolution values, all the learning data to be node-branched for the one or more reference coordinate points is most accurately separated into two. Separation accuracy calculation means for obtaining the type of convolution filter and a node threshold for this separation, and all learning data is separated into two as node branches based on the node threshold, and the convolution filter related to the node branch The decision tree is constructed by holding the type and the node threshold value related to the node branch in association with the node, and performing repetitive control to perform further node branching for all learning data after the node branch Node branching means.

  In the image data classification device according to the present invention, the node branching unit may determine whether the number of learning data belonging to the separation determination target node is equal to or less than a predetermined threshold or the separation accuracy of learning data in the separation determination target node. The decision tree is learned and constructed by performing the iterative control repeatedly until is less than or equal to a predetermined threshold value.

  In the image data classification device of the present invention, the identification processing unit includes a learning result storage unit that stores the decision tree constructed by the node branching unit, and the multiscale convolution filter according to the learned decision tree. And a second convolution filter processing means for classifying the input image to be identified using.

  Furthermore, an object detection device according to the present invention is an object detection device for detecting a predetermined object from an input frame image, wherein the input based on the image data classification device according to the present invention and a classification result by the image data classification device. Classification result determination means for executing in parallel a determination process for determining the presence or absence of an object in an image of a predetermined scanning window with respect to a frame image and a feature point selection process for selecting a feature point as an image feature when the object is present And.

  The object detection device according to the present invention includes the image data classification device according to the present invention. The image data classification device is configured such that the feature amount pool means is predetermined for each of the plurality of reference coordinate points and the filter size. A plurality of types of convolution filters composed of a plurality of types of filter coefficients and a plurality of types of filter sizes determined in advance as a feature amount pool; and the convolution filter processing unit includes the plurality of reference coordinates. A means for further obtaining a difference value of a convolution value between specific two coordinate points that can be updated among the points, wherein the separation accuracy calculating unit is configured to update between two specific coordinate points that can be updated among the plurality of reference coordinate points. Based on all combinations of difference values of convolution values, convolution that separates all the learning data of the node branch target into the two with the highest accuracy for the plurality of reference coordinate points The node branching unit separates all learning data into two as node branches based on the node threshold, and determines the node branch. By holding the type of the convolution filter and the node threshold value related to the node branch in association with the node, and performing repetitive control to perform further node branching on all the learning data after the node branch, It has a means for learning and building a decision tree.

  In the object detection apparatus according to the present invention, the classification result determination unit determines an initial value of a predetermined number of reference coordinate points among the plurality of reference coordinate points in the feature amount pool unit, and the predetermined number of reference coordinate points Reference coordinate point updating means for updating the image data classification device so as to correct the positional deviation of the positional relationship of the predetermined number of reference coordinate points by a local coordinate system having the initial value of each point as an origin. And

  Furthermore, the program according to the present invention is configured as a program for causing a computer to function as the image data classification apparatus according to the present invention, or as a program for causing a computer to function as the object detection apparatus according to the present invention.

  According to the image data classification technique of the present invention, it is possible to classify image data more robustly and accurately with general versatility, and it is possible to accurately estimate the label of the image data. Then, based on the image data classification technique according to the present invention, a target object can be detected from the image data.

It is a block diagram which shows schematic structure of the image data classification device of one Embodiment by this invention. It is a flowchart which shows the learning process in the image data classification device of one Embodiment by this invention. It is explanatory drawing of the learning process in the image data classification device of one Embodiment by this invention. It is the schematic of the decision tree constructed | assembled by the image data classification device of one Embodiment by this invention. It is a block diagram which shows schematic structure of the face detection apparatus of one Example comprised as an object detection apparatus of one Embodiment by this invention. It is explanatory drawing of the scanning window setting part in the face detection apparatus of an Example comprised as an object detection apparatus of one Embodiment by this invention. (A) is explanatory drawing of the face feature-value of three examples in the face detection apparatus of an Example comprised as an object detection apparatus of one Embodiment by this invention, (b) is an example of one Example which concerns on this invention It is explanatory drawing which illustrates the reference | standard coordinate updated in a local coordinate system about three face feature-values in a face detection apparatus, (c) The reference coordinate updated in an absolute coordinate system about three face feature-values as a comparative example It is explanatory drawing which illustrates this. It is explanatory drawing of operation | movement in the face detection apparatus of an Example comprised as an object detection apparatus of one Embodiment by this invention. It is a figure which shows the performance comparison with the technique of a nonpatent literature 1, and the face detection apparatus of one Example comprised as an object detection apparatus of one Embodiment by this invention.

[Image data classification device]
First, an image data classification device 1 according to an embodiment of the present invention will be described with reference to FIGS. 1 to 4.

(Device configuration)
FIG. 1 is a block diagram showing a schematic configuration of an image data classification device 1 according to an embodiment of the present invention. The image data classification device 1 is a device that classifies image data using a decision tree constructed by machine learning.

  In order to classify the image data of the input still image, each node of the decision tree calculates a predetermined feature amount for the image data (input image) to be classified, and first inputs the input image having the calculated feature amount. The node is divided into two nodes, and the node branches depending on whether or not the calculated feature value is larger than the node threshold value for separating the two. In the decision tree, this branch is repeated, and the classification result of the finally reached leaf node is determined as a label for the input image. The label indicates a classification result of an object (an object such as a face, a person, or a vehicle) to be detected.

  In the image data classification device 1 according to the present invention, a feature amount (including a feature amount in a feature amount pool and a feature amount to be an image feature) used for node branching is a conventional technique (in particular, Non-Patent Document 1, Unlike the technique (2), a feature quantity using a multi-scale convolution filter is used as a feature quantity having higher expressive ability.

  More specifically, in the image data classification device 1 according to the present invention, a plurality of types of one or more reference coordinate points that are determined in advance and a plurality of types of filter coefficients that are determined in advance for each filter size. The convolution filter and a plurality of predetermined filter sizes are used as feature amounts in the feature amount pool.

  Then, in the image data classification device 1 according to the present invention, for each of the reference coordinate points, a filter process using a convolution filter configured with specific filter coefficients is executed for each of the plurality of types of filter sizes, A value (convolution value g) obtained by normalizing and combining pixel values after the convolution filter processing for each type of filter size is used as a feature amount serving as an image feature.

  However, the feature amount according to the present invention is a feature amount that can represent any of the feature amounts in the techniques of Non-Patent Documents 1 and 2, and details thereof will be described later in the object detection apparatus 10 according to the present invention. .

That is, the feature amount as the image feature of the present invention represent will be described later with reference to FIG. 3, are collectively each convolution filter h _m plural kinds (m types) h (K + i, K + j) and, Each of the plurality of filter sizes N _n of this convolution filter is collectively made up of N × N (N is an odd number) pixels vertically and horizontally, and k (k is an integer of 1 or more) reference coordinate points P with respect to the input image f. _If each coordinate of _k = (x _k , y _k ) is collectively expressed as (x, y), a value obtained by normalizing and combining the pixel values after the convolution filter processing for each of the plural types of filter sizes (convolution) The value g) is defined as in equation (1). Note that a plurality of filter sizes N × N related to the convolution filter are set in advance as a feature amount pool, thereby constituting a multi-scale convolution filter process.

In this example, the value of each filter coefficient of the convolution filter h (K + i, K + j); (0 ≦ i, j <N), and the reference coordinate point P _k = (x _k , y that is the target pixel to which the convolution filter is applied. _{For k} ), a random setting is used as the feature amount pool. However, the reference coordinate point P _k to which the convolution filter is applied may be a coordinate point that is considered in advance according to the application. Depending on the application, the type of convolution filter h (K + i, K + j) used as a feature amount pool, the position of the reference coordinate point _Pk , and a plurality of filter sizes N × N related to the convolution filter can be set and changed from the outside. It is preferable to configure.

  Here, the image data classification device 1 according to the present invention configures multi-scaling by variously changing the filter size of the convolution filter. In the example described below, the filter size is increased to reduce the calculation cost. Instead of this, the corresponding image size is reduced by reducing the size of the target image. However, the filter size may be increased without changing the size of the target image.

  More specifically, referring to FIG. 1, the image data classification device 1 of the present embodiment includes a learning processing unit 2 that learns and constructs a decision tree from learning data using a multiscale convolution filter, And an identification processing unit 3 that estimates a label of an input image (still image) to be classified according to a decision tree learned using the multiscale convolution filter.

  The learning processing unit 2 includes a feature amount pool unit 21, a multi-resolution image generation unit 22, a filter convolution unit 23, a separation accuracy calculation unit 24, and a node branching unit 25. As learning data for machine learning, an image group to which a correct label is assigned (positive example) and an image group to which no correct label is assigned (negative example) are prepared in advance.

  The feature amount pool unit 21 includes one or more predetermined reference coordinate points, a plurality of types of convolution filters composed of a plurality of types of filter coefficients predetermined for each filter size, and a plurality of types determined in advance. Holds the filter size.

  The multi-resolution image generation unit 22 performs a plurality of resolution conversions on each of a plurality of input learning data in accordance with a feature amount pool (resolution corresponding to a plurality of types of filter sizes) held in the feature amount pool unit 21. A plurality of resolution images corresponding to each learning data are generated and output to the filter convolution unit 23.

  The filter convolution unit 23 uses a feature amount pool (individual reference coordinate points and individual convolutions) held in the feature amount pool unit 21 for a plurality of resolution images for each of a plurality of learning data obtained from the multi-resolution image generation unit 22. Convolution filter processing is executed according to the filter). Then, the filter convolution unit 23 performs convolution filter processing for each combination of a certain convolution filter having the same filter coefficient with respect to a certain reference coordinate point in the plurality of resolution images, for each of the plurality of types of filter sizes. A pixel value after the convolution filter processing is obtained, and a value obtained by normalizing and combining these pixel values (convolution value g) is obtained. Therefore, a plurality of convolution values g corresponding to the number of convolution filters of a plurality of types are obtained for each of one or more reference coordinate points for one learning data.

Therefore, one learning data, each reference coordinate point P _k, each predetermined number of filter size to N × convolution of plural kinds of convolved with N filter h _m a plurality of convolution values g respectively associated Is obtained. Thus, one or more reference coordinate point P _k, a plurality of kinds of convolution filter h _m, a combination of a plurality of convolution values g respectively associated with these, as a feature vector characterizing define the one learning data expressed.

  The multi-resolution image generation unit 22 and the filter convolution unit 23 perform the same processing for all learning data.

Then, the filter convolution unit 23, one or more and the reference coordinate point P _k, expressed as a feature vector characterizing define each learning data, a plurality of kinds of convolution filter h _m and, convolution value g that these respectively associated Is output to the separation accuracy calculation unit 24 in association with each learning data.

Separation accuracy calculation unit 24, the filter convolution unit 23, one or more and the reference coordinate point P _k for each of all learning data, and a plurality of kinds of convolution filter h _m, these by convolution respectively associated value g And a combination of a predetermined number of reference coordinate points P _k (correspondingly, individual convolution values g are obtained) among one or more reference coordinate points P _k . the type of convolution filter h _m separates the training data into two highest accuracy, and outputs a node threshold for this separation node bifurcation 25 asking. For the determination of the quality of separation, the same scale as in the prior art such as Gini coefficient and information gain is used.

Node splitter 25, based on the node threshold obtained from the separation accuracy computing unit 24, separates into two all learning data as node bifurcation, and the type of convolution filter h _m according to the node branch, the node branch Is stored in association with the node in order to construct a decision tree.

Further, the node branching unit 25 instructs the filter convolution unit 23 to perform further node branching for each of the branched nodes, and allocates learning data corresponding to each branched node, so that the node to be separated is determined. It repeats until the number of learning data belonging to or below the predetermined threshold value or until the separation accuracy of the learning data at the separation determination target node falls below the predetermined threshold value (that is, until improvement of the separation accuracy cannot be expected). . The node that cannot be branched becomes a leaf node, and the correct answer or incorrect answer as a discrimination result according to the correct answer or incorrect answer label of the learning data finally remaining in the node, and the convolution filter h if the answer is correct _A discrimination label indicating the type of _m is determined.

Further, the node branching unit 25 also applies all the combinations of a predetermined number of reference coordinate points P _k (correspondingly, individual convolution values g are obtained) among the one or more reference coordinate points P _k. a filter h _m convolution most accurately separated into two learning data, repeat branching based on node threshold for this separation, finally correct or incorrect labels remaining training data to that node depending on the determination result as a correct or incorrect answer, and determines the discriminated label indicating the type of the convolution filter h _m if correct.

Note that a combination of a specific number of reference coordinate points P _k (correspondingly, individual convolution values g can be obtained) among the one or more reference coordinate points P _k can be determined by an external setting by the operator. Automatically based on a predetermined selection criterion (for example, selecting a combination while maintaining the specific number using another reference coordinate point closest to the combination initial value of the specific number of reference coordinate points _Pk ) It is preferable to set to. Note that when there is one reference coordinate point _Pk held in advance in the feature quantity pool unit 21, the specific number used for classification determination by the decision tree is also one, and one decision tree is constructed. Further, even when all the reference coordinate points P _k stored in advance in the feature amount pool unit 21 are set to the specific number, one decision tree is constructed.

In this manner, a number of decision trees corresponding to the number of combinations of a specific number of reference coordinate points P _k (correspondingly, individual convolution values g are obtained) among the one or more reference coordinate points P _k are constructed. .

  Machine learning is performed so that the output label of the decision tree to be constructed (discrimination label of the final result) matches the label of the correct answer or the incorrect answer given in advance to the learning data. Since the final decision tree output label (discriminating label of the final result) is further classified into labels of correct answer (or incorrect answer) and label 1, label 2,... When constructing a decision tree by machine learning, when the simple correct answer or the incorrect answer is classified into two categories, the node branching unit 25 selects only nodes to which a predetermined number or more of learning data is allocated in the plurality of types of labels. Can be used to build a decision tree.

In this example, the node branching unit 25 instructs the filter convolution unit 23 to perform further node branching for each of the branched nodes so that learning data corresponding to each branched node is allocated. Although an example is shown in which loop processing is performed for node branching in a decision tree, convolution values g for all reference coordinate points P _k are obtained in a batch without performing loop processing to avoid duplication processing, and nodes It can also be a process of repeatedly performing branching.

  In addition, a multiscale convolution filter by further convolution with convolution filters of different filter sizes calculates filter coefficients of all types of multiscale convolution filters in advance, and generates a convolution value g without generating a multi-resolution image. It can also be set as the structure which obtains.

  The node branching unit 25 stores the finally constructed decision tree in the learning result storage unit 31.

  On the other hand, the identification processing unit 3 includes a learning result storage unit 31, a multi-resolution image generation unit 33, and a filter convolution unit 33.

  The learning result storage unit 31 stores the decision tree constructed by the node branching unit 25. The decision tree includes one or more reference coordinate points used during machine learning as a feature amount pool, a plurality of types of convolution filters configured with a plurality of types of filter coefficients predetermined for each filter size, and a predetermined tree. In addition, information on a plurality of types of filter sizes and node threshold information for branching of each node are included.

  The multi-resolution image generation unit 33 performs a plurality of resolution conversions on the input image to be subjected to identification processing according to a decision tree (resolution corresponding to a plurality of types of filter sizes) held in the learning result storage unit 31, An image is generated and output to the filter convolution unit 33. That is, the multi-resolution image generation unit 33 converts the image into a plurality of resolution images similar to the multi-resolution image generation unit 22 in the learning processing unit 2 and outputs the converted image to the filter convolution unit 33.

  The filter convolution unit 33 performs convolution filters on a plurality of resolution images corresponding to the input image obtained from the multi-resolution image generation unit 33 according to a decision tree (individual reference coordinate points and individual convolution filters) held in the learning result storage unit 31. Processing is performed to obtain pixel values after the convolution filter processing for each of the plurality of types of filter sizes, and a value obtained by normalizing and combining these pixel values (convolution value g) is obtained.

  Subsequently, the filter convolution unit 33 branches based on each node threshold using the decision tree, and outputs the label assigned to the node as an identification result when the leaf node is reached.

(Example of learning process)
Hereinafter, an example of the learning process performed by the learning processing unit 2 will be described more specifically with reference to FIGS. 2 and 3. In the learning processing example shown in FIG. 2, in configuring multi-scaling by variously changing the filter size of the convolution filter, the size of the target image is reduced instead of increasing the filter size in order to reduce the calculation cost. This is a corresponding example. However, as described above, an example is shown in which loop processing is executed for node branching in a decision tree. However, in order to avoid duplication processing, node branching may be performed without performing loop processing.

The learning processing unit 2 determines whether or not an unbranched node remains for each of the plurality of input learning data f ₁ , f ₂ ,..., F _S (number of data: S). (Step S1) Since an unbranched node naturally remains at the time of the first input (Step S1: Yes), the process proceeds to Step S2.

  Subsequently, the learning processing unit 2 refers to the feature amount pool unit 21 for each of the plurality of pieces of learning data input by the multi-resolution image generation unit 22 (step S2), and the feature amount pool (multiple types of data). A plurality of resolution conversions are performed according to the resolution according to the filter size, and a plurality of resolution images corresponding to each learning data are generated.

For example, as illustrated in FIG. 3, the multi-resolution image generation unit 2 reduces each of a plurality of input learning data f ₁ , f ₂ ,..., F _S (data number: S) to various image sizes. As the filter size N × N, N ₁ × N ₁ (1 time), N ₂ × N ₂ (0.5 time), N ₃ × N ₃ (0.25 time) are included in the feature amount pool unit 21. Are prepared, they are converted into three types of resolution images. It should be noted that it is not necessary to limit to this example such as reducing by 1 / √N ₁ time.

  Subsequently, the learning processing unit 2 performs a predetermined number of convolution filter processes on each learning data belonging to the node, and further convolves (step S3). More specifically, the learning processing unit 2 uses the filter convolution unit 23 to store a feature amount pool (individual reference coordinate points and individual reference points) held in the feature amount pool unit 21 for a plurality of resolution images corresponding to each learning data. The convolution filter processing for each combination of a certain convolution filter having the same filter coefficient with respect to a certain reference coordinate point in the plurality of resolution images is performed with reference to the convolution filter), and A pixel value after the convolution filter processing is obtained, and a value obtained by normalizing and combining these pixel values (convolution value g) is obtained.

For example, as shown in FIG. 3, two pieces of one or more reference coordinate points P ₁ , P ₂ ,..., P _k = (x _k , y _k ) (k is an integer of 1 or more) per learning data. A plurality of convolution filters h ₁ , h ₂ ,..., H _m (m is an integer of 2 or more) corresponding to each reference coordinate point for a combination of _two reference coordinate points P ₁ and P _2. The convolution values g are obtained, and FIG. 3 illustrates the convolution values g ₁ and g ₂ corresponding to the _two reference coordinate points P ₁ and P ₂ , respectively.

Therefore, one learning data f _S is, a plurality of respectively for each reference coordinate point P _k is respectively associated with a convolution filter h _m of plural kinds of convolved with a predetermined number of filter size N × N convolution The value g is obtained. Thus, one or more reference coordinate point P _k, a plurality of kinds of convolution filter h _m, a combination of a plurality of convolution values g respectively associated with these, as a feature vector characterizing define the one learning data expressed.

Then, as shown in FIG. 3, multi-resolution image generation unit 22 and the filter convolution unit 23, all learning data _{f 1,} f 2, ..., the same processing is performed for _{f S.}

Subsequently, the learning processing unit 2, separated by accuracy computing unit 24, and one or more reference coordinate point P _k of all of the training data, a plurality of kinds of convolution filter h _m, the convolution values respectively associated with these acquires combination information of g, determined with highest accuracy convolution separates into two filters h _m all learning data for the reference coordinate point P _k of a specific number, the node threshold for this separation, a node branch The node 25 is branched by the unit 25 as shown in FIG. 3 (step S4). During the node branching, type and node threshold convolution filter h _m is held in association with the node for the construction of decision trees. For the determination of the quality of the separation, the same scale as the conventional technique such as Gini coefficient and information gain is used.

  Subsequently, the learning processing unit 2 determines whether or not further node branching is possible for the branched node by the node branching unit 25 (step S6), and if further node branching is possible (step S6). S6: Yes), the process proceeds to step S2, where the filter convolution unit 23 is instructed to perform further node branching, learning data corresponding to each branched node is allocated, and learning data belonging to the separation determination target node Until the number falls below a predetermined threshold or until the separation accuracy of learning data at the separation determination target node falls below a predetermined threshold (that is, until improvement in separation accuracy cannot be expected) (step S6: No) ),repeat.

  Subsequently, the learning processing unit 2 determines whether or not unbranched nodes remain (step S1), and repeats the processes of steps S2 to S6 until there are no unbranched nodes (step S1: No) ( Step S1: Yes).

  Finally, the learning processing unit 2 repeats the above-described node branching by the node branching unit 25, and the correct answer as the determination result according to the correct answer or incorrect answer label of the learning data belonging to the node that cannot be branched. Alternatively, the label of the incorrect answer and the correct answer (or incorrect answer) is further classified to determine the discrimination label indicating the type.

  That is, as shown in FIG. 4, node branching is performed from the first node 100 to the second node 200 and the third node 300 that are separated depending on whether the feature A for branching the input image is larger or smaller than the threshold value A. Similarly, the second node 200 and the third node 300, and further the fourth node 400 and the fifth node 500 repeat node branching as much as possible at each node, and finally, label 1, label 2,. In this way, a plurality of types of labels are attached.

  Normally, when the decision tree is constructed by machine learning, when the simple correct answer or the incorrect answer is classified into two categories, the node branching unit 25 is a node in which learning data is allocated to a predetermined number or more in the plural types of labels. Build a decision tree using only

  The decision tree constructed in this way can be used for various object detection applications such as vehicle detection, vehicle feature point detection, or a combination thereof, in addition to face detection and face feature point detection. Become.

For example, it can be seen that the feature amounts of the conventional techniques as shown in Non-Patent Documents 1 and 2 can be expressed by the combination of the filter size and the filter coefficient. Note that the reference coordinate point P _k = (x _k , y _k ) to which the filter is applied can be selected in consideration of the feature point position, as in the technique of Non-Patent Document 2. In that case, it is conceivable to perform probabilistic sampling such that selection is made with higher probability as the position is closer to the feature point.

  In particular, since the image data classification apparatus 1 of the present embodiment uses the decision tree constructed in this way, for example, noise or a variety of face orientations (faces) in an input image that is a target of face detection or face feature points. Even when there is image deformation), it is possible to classify image data robustly and accurately because it is a convolution value based on a point having a high-frequency noise removal effect and a reference coordinate point.

  Further, when applying the processing of the image data classification device 1 of the present embodiment to the person recognition processing for the image data, first, after the processing of the image data classification device 1 of the present embodiment, the face based on the technique of Non-Patent Document 1 Even in a configuration in which a plurality of coordinate points (face feature points) are detected from a face image based on the technique of Non-Patent Document 2 for the input image after detection, the processing performance is improved because the classification accuracy is improved. Will improve. However, as will be described below, it is possible to configure the object detection device 10 with higher processing efficiency by using the image data classification device 1 of the present embodiment.

[Object detection device]
Hereinafter, with reference to FIG. 5 to FIG. 9, an example face detection apparatus configured as the object detection apparatus 10 according to an embodiment of the present invention will be described.

(Device configuration)
FIG. 5 is a block diagram illustrating a schematic configuration of a face detection apparatus according to an example configured as the object detection apparatus 10 according to an embodiment of the present invention. Here, an example of a face detection device will be described as a typical example of the object detection device 10, but by selecting learning data as appropriate, in addition to face detection, person detection, person recognition, detection of an object such as a vehicle, etc. Note that it can be widely used for object detection from still images.

  As shown in FIG. 5, the object detection device 10 includes an image data classification device 1 according to the present invention, a scanning window setting unit 11, a classification result determination unit 12, a local coordinate system reference coordinate point update instruction unit 13, Is provided.

The scanning window setting unit 11 is a functional unit that allows an entire input frame image to be scanned with a scanning window of various sizes with respect to an input frame image of a still image such as one frame of a moving image, and has a certain size (scanning window scale) The image at a specific scanning position in the input frame image is cut out from the scanning window and output to the image data classification device 1 according to the present invention. The change of the size of the scanning window and the change of the specific scanning position of the input frame image are instructed by the classification result determination unit 12 described later. For example, FIG. 6 shows three scanning window scales S ₁ , S _2, and S ₃ for the input frame image F. The input frame image F illustrated in the center of the figure has a scanning window scale S ₂ , respectively. A region where the face detection labels 1 and 2 are expected to be distinguished at different scanning positions is indicated by a broken line.

The image data classification device 1 according to the present invention constructs a decision tree learned for face detection, outputs two classification labels of “face” and “not face”, and sets a predetermined average face. the initial value _P 1 of the reference coordinate point of the 4 points as a face feature point (facial feature points) _based, P _2, P 3, and _{P 4,} the initial value _P 1 of the reference coordinate _point, P _2, P 3, _{P 4} The reference coordinates of predetermined neighbors distributed by performing probabilistic sampling that is selected with higher probability as the position is closer (from the initial values P ₁ , P ₂ , P ₃ , P _{4 of the} reference coordinates). It is assumed that a large number of reference coordinate points P ₁ ′, P ₂ ′, P ₃ ′, and P ₄ ′) that are updated are held as a feature amount pool. The update of the set value of the reference coordinate point is instructed by the local coordinate system reference coordinate point update instruction unit 13 described later.

  Further, the filter convolution unit 23 in the image data classification device 1 includes, in addition to the convolution value g as the feature amount of the image feature in the decision tree, specific two coordinate points that can be updated among a plurality of reference coordinate points (face feature points). All combinations of difference values of convolution values (hereinafter referred to as “convolution difference values”) Δg are also calculated.

For example, FIG. 7A corresponds to certain two coordinate points that can be selected from a plurality of reference coordinate points (face feature points) with respect to a certain input image f cut out and input by the scanning window. If convolution values g ₁ and g ₂ , convolution values g ₃ and g ₄ corresponding to another two coordinate points, and convolution values g ₅ and g ₆ corresponding to another two coordinate points are assigned, 2 The difference between the convolution values corresponding to the coordinate points (convolution difference value Δg) is also calculated, and becomes the facial feature quantity used for the two classifications of “being a face” and “not a face” for the input image f.

Further, regarding the update of the reference coordinate point according to the present invention, as shown in three examples of input images f _A , f _B , and f _C in FIG. 7B, the updated reference coordinate points P ₁ ′, P ₂ ′ (same for P ₃ ′ and P ₄ ′) is updated by the local coordinate system with the initial values P ₁ and P ₂ of the reference coordinate points (same for P ₃ and P ₄ ) as origins.

This is because the positional relationship of the reference coordinate points P ₁ ′, P ₂ ′, P ₃ ′, and P ₄ ′ due to individual differences in face shape in the input image to be detected and changes in facial orientation and facial expression. This is to alleviate the problem. For example, as a comparative example, when the reference coordinate point is updated in the absolute coordinate system as shown in three input images f _A , f _B , and f _C in FIG. As a result, the positional deviation of the positional relationship between the reference coordinate points P ₁ ′, P ₂ ′, P ₃ ′, and P ₄ ′ increases depending on the input image. For this reason, the reference coordinate point is updated based on the local coordinate system.

The separation accuracy calculation unit 24 in the image data classification device 1, respectively from filter convolution unit 23, one or more and the reference coordinate point P _k of all of the training data, a plurality of kinds of convolution filter h _m, these Information including all combinations of a plurality of convolution values g attached and convolution difference values Δg corresponding to two specific reference coordinate points P _k among the four reference coordinate points (face feature points) is acquired. The separation accuracy calculation unit 24 obtains the convolution difference value most accurately filter convolution separates into two h _m all learning data for all combinations of Delta] g, the node threshold for this separation. The node branching unit 25 constructs a decision tree so as to output two types of labels “face” and “not face” based on each node threshold.

Therefore, the image data classification device 1 according to the present invention branches the face detection target input image f input by being cut out by the scanning window, using the decision tree, according to each node threshold, Is reached, the face detection label “Face” or “Non-face” assigned to the node is finally updated and assigned to the reference coordinate point P ₁ ′ assigned to the node. , P ₂ ′, P ₃ ′, and P ₄ ′, and outputs them to the classification result determination unit 12 as identification results.

The classification result determination unit 12 finally updates and assigns the assigned four coordinate points from the image data classification apparatus 1 according to the present invention together with the face detection label of “being a face” or “not a face”. Reference coordinate points P ₁ ′, P ₂ ′, P ₃ ′, and P ₄ ′ are input and temporarily stored. One or all of the reference coordinate points P ₁ ′, P ₂ ′, P ₃ ′, and P ₄ ′ of the _four coordinate points includes a case where the value is the same as the initial value of the reference coordinate point.

  Subsequently, if the temporarily held face detection label indicates “not a face”, the classification result determination unit 12 causes the scanning window setting unit 11 to set the scanning window to the next scanning position. Alternatively, if the scanning window is the final scanning position, the initial scanning position of the input frame image is set by the scanning window of the next size (scanning window scale), and the face detection target image is cut out from the input frame image. The image data classification apparatus 1 according to the present invention is instructed to perform classification determination again.

On the other hand, when the temporarily held face detection label indicates “face”, the classification result determination unit 12 determines the reference coordinate points P ₁ ′, P ₂ ′, P ₃ ′, and P ₄ of the four coordinate points. Instruct the local coordinate system reference coordinate point update instruction unit 13 to update '.

The local coordinate system reference coordinate point update instruction unit 13 performs reference coordinate points P ₁ ′, P ₂ ′, P ₃ ′, P of four coordinate points on the input image of the face detection label temporarily stored as “face”. ₄ 'is managed for all possible combinations held in the feature amount pool unit 21 in the image data classification apparatus 1 according to the present invention, and the reference coordinate point setting value is updated according to the present invention. An instruction is given to the image data classification device 1.

  In FIG. 1, the classification result determination unit 12 and the local coordinate system reference coordinate point update instruction unit 13 are illustrated as separate functional units. However, the local coordinate system reference coordinate point update instruction unit 13 includes a classification result determination unit. 12 can be configured as a partial function. That is, the classification result determination unit 12 determines the presence / absence of an object in the image of the scanning window based on the classification result by the image data classification device 1 and the feature points that are the image features when the object exists. The feature point selection process to be selected can be executed in parallel.

Then, the classification result determination unit 12 repeats the classification of “not a face” and “is a face” as much as possible for the input image indicating that the temporarily detected face detection label is “face”. Together with the input image of “Face” that is finally classified with the detection label, the corresponding four coordinate point reference coordinate points P ₁ ′, P ₂ ′, P ₃ ′, P ₄ ′ are used as face feature points. Output to the outside.

  Then, the classification result determination unit 12 performs the following operation on the input frame image indicating “not a face” even if the scanning window scanning position or size (operation window scale) is changed with respect to the scanning window setting unit 11. A face detection label indicating “not a face” is attached and output to the outside.

(Operation example)
FIG. 8 is an explanatory diagram of an operation in the face detection device of one example configured as the object detection device 10 of the present embodiment.

  First, the object detection apparatus 10 uses the scanning window setting unit 11 to input an image f cut out with a scanning window having a predetermined size and a predetermined position with respect to an input input frame image F as an input image. Enter.

Then, as shown in FIG. 8, the image data classification device 1 first sets an initial value P ₁ of four reference coordinate points (face feature points) as face feature points based on a predetermined average face for the input image f. P ₂ , P ₃ , and P ₄ are assigned to classify “not a face” and “is a face” (step S11).

  At this time, the object detection apparatus 10 sets the next scanning window image as a face detection target for the input image f classified as “not a face” by the classification result determination unit 12 with respect to the scanning window setting unit 11. Control as follows.

On the other hand, the classification result determination unit 12 instructs the image data classification device 1 according to the present invention via the local coordinate system reference coordinate point update instruction unit 13 for the input image f classified as “face”. Then, the reference coordinate points P ₁ ′, P ₂ ′, P ₃ ′, and P ₄ ′ of the four coordinate points are updated (step S12). As described above, when the result of classification by the image data classifying apparatus 1 is “not a face”, the process is regressed to input an image of the next scanning window.

Then, the classification result determination unit 12 repeats the classification of “not a face” and “is a face” as much as possible for the input image indicating that the temporarily detected face detection label is “face”. Together with the input image of “Face” that is finally classified with the detection label, the corresponding four coordinate point reference coordinate points P ₁ ′, P ₂ ′, P ₃ ′, P ₄ ′ are used as face feature points. Output to the outside.

As described above, the classification result determination unit 12 repeatedly updates the reference coordinate points P ₁ ′, P ₂ ′, P ₃ ′, and P ₄ ′ for the input image of “Face” that is finally classified. By causing the image data classification device 1 to perform classification, it is gradually difficult to classify “not a face” and “is a face”, and eventually converge to a state where classification is achieved. Then, the updated reference coordinate points P ₁ ′, P ₂ ′, P ₃ ′, and P ₄ ′ for the final “face” input image are highly accurate.

  Therefore, the object detection apparatus 10 according to the present embodiment performs a classification problem of “not a face” and “is a face”, updates of reference coordinate points (face feature points) of four coordinate points (dispersion of displacement of face feature points). Since the image data classification device 1 can solve the regression problem for performing (minimization of the image) in parallel, the processing efficiency and the face detection accuracy can be improved.

  That is, serial processing is performed such that face detection based on the technique of Non-Patent Document 1 is performed, and then a plurality of coordinate points (face feature points) are detected from the face image based on the technique of Non-Patent Document 2 for the input image. Rather, the object detection device 10 of the present embodiment improves the processing efficiency.

  Further, in the above-described example, the example in which the reference coordinate points of the four coordinate points are updated has been described. However, the number of reference coordinate points is further reduced, or the reference coordinate points of more nine coordinate points are updated. Can be set arbitrarily.

(Verification by experiment)
If the accuracy of face detection is not improved, the accuracy of detection of face feature points useful for human recognition cannot be expected. In order to improve the accuracy of face detection, it is effective to improve the accuracy of face classification. Therefore, a comparison experiment of face detection performance was performed between the object detection apparatus 10 of the present embodiment configured to update the reference coordinate points of the nine coordinate points and the technique of Non-Patent Document 1 configured under the same conditions.

  As learning data, 20,000 face images were extracted from TV images for two months, and the decision tree was constructed in the image data classification device 1 in the object detection device 10 of the present embodiment. The maximum number of regressions of the object detection apparatus 10 is limited to 5 times, and the number of labels at the time of learning is limited so that the maximum number of nodes is 600, and the decision tree is constructed.

  The image to be tested is a plurality of frame images in a certain day of broadcast video as input frame images for face detection, and the face detection performance of the object detection device 10 of this embodiment and the technique of Non-Patent Document 1 is as follows. As a result of comparison, the results shown in FIG. 9 were obtained.

  In FIG. 9, “detection rate” is the proportion of faces that have been detected in the input frame image. The “false detection rate” indicates the ratio of errors included in the detection result. It was confirmed that the object detection apparatus 10 of the present embodiment improved performance by 29.3% as “detection rate” and improved by 21.1% as “false detection rate”.

  When these detection results are analyzed, in the technique of Non-Patent Document 1, undetected due to a change in face orientation and facial expression, and false detection due to a complicated background are detected with the object detection apparatus 10 of the present embodiment. It has been confirmed that the image data classification device 1 according to the present invention is particularly effective for the object detection device 10 in order to perform face detection from a camera image that is confirmed as a difference.

(Summary)
As described above, the image data classification device 1 according to the present invention can more accurately capture the shape and characteristics of an object shown in a video than the conventional technique by using a multiscale convolution filter. Data classification accuracy can be improved.

  The image data classification apparatus 1 according to the present invention can be widely used for object detection from still images such as face detection, person detection, person recognition, and object detection such as a vehicle.

  In addition, the image data classification device 1 according to the present invention is not only used as an object detection device 10 based on a decision tree, but also uses other techniques based on a decision tree such as regression using a decision tree or a random forest. Can also be used. That is, the random forest is one group learning technique using a decision tree, and uses a large number of decision trees to estimate the label of the data and finally determine the estimated label by majority voting. It is. For this reason, the classifier constituting the decision tree in the random forest can be used in place of the image data classification device 1 according to the present invention.

  In addition to the object detection device 10 based on a decision tree, it can also be used for various boosting algorithms such as AdaBoost and Real AdaBoost. That is, AdaBoost and Real AdaBoost are techniques for classifying data by connecting a large number of classifiers, and the image data classification apparatus 1 according to the present invention can be used as the classifier.

  The image data classification device 1 and the object detection device 10 can each function as a computer, and a program for causing the computer to realize each component is stored in the memory of the computer. Under the control of a central processing unit (CPU) provided in the computer, a program in which processing content for realizing the function of each component is described is appropriately read from the memory, and the function of each component is It can be realized on a computer.

  The image data classification device 1 and the object detection device 10 according to the present invention and these programs are not limited to the above-described embodiments, but are limited only by the description of the scope of claims.

  According to the present invention, it is possible to classify image data with more versatility and more robustly and accurately, and it is possible to accurately estimate the label of the image data. Useful for detecting applications.

DESCRIPTION OF SYMBOLS 1 Image data classification device 2 Learning process part 3 Identification process part 10 Object detection apparatus 11 Scanning window setting part 12 Classification result determination part 13 Local coordinate system reference | standard coordinate point update instruction part 21 Feature-value pool part 22 Multi-resolution image generation part 23 Filter Convolution unit 24 Separation accuracy calculation unit 25 Node branching unit 31 Learning result storage unit 32 Multi-resolution image generation unit 33 Filter convolution unit

Claims (9)

An image data classification device for classifying image data,
A learning processing unit that learns and constructs a decision tree from learning data prepared in advance using a multi-scale convolution filter;
An identification processing unit that classifies an input image to be identified using the multiscale convolution filter according to the learned decision tree;
An image data classification device comprising: The learning processing unit
Features of one or more predetermined reference coordinate points, a plurality of types of convolution filters composed of a plurality of types of filter coefficients predetermined for each filter size, and a plurality of types of predetermined filter sizes A feature amount pool means to hold as a pool;
For each of the plurality of input learning data, the multi-scale convolution filter processing is performed by the plurality of types of convolution filters according to the plurality of types of filter sizes according to the feature amount pool, First convolution filter processing means for obtaining a plurality of convolution values corresponding to the number of plural kinds of convolution filters for each of the one or more reference coordinate points;
About one or more reference coordinate points based on the combination information of the one or more reference coordinate points for each of all learning data, the plurality of types of convolution filters, and the convolution values associated with each of the plurality of convolution filters. Separation accuracy calculation means for obtaining a type of convolution filter that most accurately separates all learning data to be node-branched into two and a node threshold for this separation,
Based on the node threshold, all learning data is separated into two as node branches, the type of convolution filter related to the node branch and the node threshold related to the node branch are held in association with the node, Node branching means for constructing the decision tree by performing repetitive control to perform further node branching for all learning data after node branching;
The image data classification device according to claim 1, further comprising:   The node branching means repeats until the number of learning data belonging to the separation determination target node is equal to or less than a predetermined threshold or the separation accuracy of learning data in the separation determination target node is equal to or less than a predetermined threshold. The image data classification apparatus according to claim 2, wherein the decision tree is learned and constructed by performing repetitive control. The identification processing unit
Learning result storage means for storing the decision tree constructed by the node branching means;
Second convolution filter processing means for classifying the input image to be identified using the multiscale convolution filter according to the learned decision tree;
The image data classification device according to any one of claims 1 to 3, further comprising: An object detection device for detecting a predetermined object from an input frame image,
An image data classification device according to any one of claims 1 to 4,
Based on the classification result by the image data classification device, a determination process for determining the presence / absence of an object in an image of a predetermined scanning window with respect to the input frame image, and a feature point serving as an image feature when the object is present are selected. Classification result judging means for executing feature point selection processing in parallel;
An object detection apparatus comprising: An image data classification device according to any one of claims 2 to 4, comprising:
The image data classification device includes:
The feature amount pool means includes a plurality of reference coordinate points, a plurality of types of convolution filters configured with a plurality of types of filter coefficients predetermined for each filter size, and a plurality of types of filter sizes determined in advance. Having means for storing as a feature amount pool;
The convolution filter processing means has means for further obtaining a difference value of a convolution value between specific two coordinate points that can be updated among the plurality of reference coordinate points,
The separation accuracy calculation means is configured to calculate all of the node branching targets for the plurality of reference coordinate points based on all combinations of difference values of convolution values between specific two coordinate points that can be updated among the plurality of reference coordinate points. Means for determining the type of convolution filter that most accurately separates learning data into two and the node threshold for this separation,
The node branching means separates all learning data into two as node branches based on the node threshold, and corresponds the type of convolution filter related to the node branch and the node threshold related to the node branch to the node 6. The information processing apparatus according to claim 5, further comprising means for learning and constructing the decision tree by performing repetitive control to perform further node branching for all learning data after the node branching. The object detection device described in 1.   The classification result determining unit determines an initial value of a predetermined number of reference coordinate points among the plurality of reference coordinate points in the feature amount pool unit, and each of the local values having the initial value of the predetermined number of reference coordinate points as an origin. 7. The object according to claim 6, further comprising reference coordinate point updating means for updating the image data classification device so as to correct a positional deviation of the predetermined number of reference coordinate points by a coordinate system. Detection device.   A program for causing a computer to function as the image data classification device according to any one of claims 1 to 4.   The program for functioning a computer as an object detection apparatus as described in any one of Claim 5 to 7.