JP2017162098A - Learning method, information processing device and learning program - Google Patents

Abstract

[PROBLEMS] To improve classification accuracy. A calculation unit 1b acquires information indicating a first region G11 and information indicating a classification destination of the image G1 in the image G1 and the image G1. The computing unit 1b performs first detection results of a plurality of types of feature amounts in the first region G11 and second detection results of a plurality of types of feature amounts in the second region G12 other than the first region G11 in the image G1. Compare The calculation unit 1b determines the weights of the plurality of types of feature amounts when classifying the image G1 to the acquired classification destination according to the comparison. The computing unit 1b learns the classification of the image G1 based on the result of weighting each of a plurality of types of feature amounts with the weights. [Selection] Figure 1

Description

  The present invention relates to a learning method, an information processing apparatus, and a learning program.

  Currently, image processing techniques are used in various fields. For example, there is a proposal of an automatic inspection device that performs quality determination by inspecting an image showing the appearance of a product. The automatic inspection apparatus repeats an operation called learning for an image to be inspected, and then performs pass / fail determination for an arbitrary image.

  As a learning method for classifying images, for example, there is a method called supervised learning. In supervised learning, a predetermined number of image sets are prepared in advance, and the user labels each image as correct / incorrect (for example, good / bad in pass / fail judgment). Then, a component of the feature quantity that can separate the correct / incorrect answer from the set of feature quantities (for example, brightness and hue in the image) used for image classification is analyzed by a computer, and learning is performed using the components. I do.

  In addition, in a system used in the field of process control and the like, there is also a proposal for classifying a situation in order to determine whether it is an abnormal situation or a situation where a predetermined operation should be performed. In this proposal, in the feature space defined by the feature quantity used for the situation classification, the situation is classified by classifying abnormal / normal or action A / action B, and abnormality determination or action determination is performed.

JP 2006-293528 A JP 2001-250101 A

  As described above, it is also conceivable to prepare an image set in advance and perform learning using the image set. However, when learning is started, an image set for learning may not be at hand. If a learning image set is not available, learning may be performed using sequentially generated image data. However, in this case, it is difficult to determine which feature amount should be used for learning. Although it is conceivable to perform learning by treating all types of feature quantities equally, feature quantities that do not contribute to the original classification become noise, and the accuracy of classification can be reduced.

  In one aspect, the present invention aims to increase classification accuracy.

  In one aspect, in the learning method, the computer acquires information indicating a first area and information indicating a classification destination of the image among the images, and performs first detection of a plurality of types of feature amounts in the first area. According to the result and the comparison of the second detection result of the plurality of types of feature amounts in the second region other than the first region of the image, each of the plurality of types of feature amounts at the time of classifying the image to the classification destination , And learning of image classification is performed based on the result of weighting each of a plurality of types of feature amounts by the weight.

  In one aspect, classification accuracy can be increased.

It is a figure which shows the information processing apparatus of 1st Embodiment. It is a figure which shows the hardware example of the image processing server of 2nd Embodiment. It is a figure which shows the example of a function of an image processing server. It is a figure which shows the example of an attention area | region determination rule. It is a figure which shows the example of a feature-value table. It is a figure which shows the example of a feature-value priority table. It is a figure which shows the example of feature space. It is a figure which shows the example of an update of a priority. It is a figure which shows the example of the weighting of a feature-value. It is a flowchart which shows the example of learning. It is a flowchart which shows the example of a feature-value priority table update. It is a flowchart which shows the example of attention area determination. It is a figure which shows the relationship between a classification | category and a feature-value. It is a flowchart which shows the other example of a feature-value priority table update. It is a figure which shows the other example of the update of a priority.

Hereinafter, the present embodiment will be described with reference to the drawings.
[First Embodiment]
FIG. 1 is a diagram illustrating the information processing apparatus according to the first embodiment. The information processing apparatus 1 classifies the input image into two categories. The two classifications are, for example, correct answers and incorrect answers. Alternatively, the two classifications may be represented as “good and bad”, “true and false”, “positive and negative”, or the like. Such classification can be applied to, for example, determining whether or not a product shown in an image is good, whether the weather is good or bad in a landscape shown in the image, and the presence or absence of a human face in the image.

  The information processing apparatus 1 performs supervised learning in order to determine the classification destination of the input image. However, in the example of the first embodiment, the learning image set does not exist in advance. For this reason, the information processing apparatus 1 performs learning using sequentially input images. When the information processing apparatus 1 performs learning, the information processing apparatus 1 narrows down feature quantities to be learned (that is, used for classification) from among a plurality of types of feature quantities in the input image. As the feature amount in the image, for example, an amount obtained by quantifying the feature relating to the image, such as a brightness distribution, a color histogram, and a texture, can be considered.

  The information processing apparatus 1 includes a storage unit 1a and a calculation unit 1b. The storage unit 1a may be a volatile storage device such as a RAM (Random Access Memory) or a non-volatile storage device such as an HDD (Hard Disk Drive) or a flash memory. The calculation unit 1b may include a CPU (Central Processing Unit), a DSP (Digital Signal Processor), an ASIC (Application Specific Integrated Circuit), an FPGA (Field Programmable Gate Array), and the like. The calculation unit 1b may be a processor that executes a program. As used herein, the “processor” may include a set of multiple processors (multiprocessor).

  The storage unit 1a stores an image, information indicating a first area of the image, and information indicating a classification destination of the image. The first area is an area that has attracted attention in order to determine the classification destination. It can also be said that the first region is a characteristic region in order to determine the classification destination in the image. For example, the user U1 indicates the image G1, information indicating the first region G11 in the image G1, and information indicating the classification destination C1 of the image G1 (for example, information indicating whether the image G1 is classified as a correct answer or an incorrect answer). Can be input to the information processing apparatus 1 using the terminal device 2. That is, the user U1 can input the information on the classification destination C1 of the image G1 and the region noticed when the image G1 is classified into the classification destination C1 to the information processing apparatus 1 using the terminal device 2. For example, the calculation unit 1b may store the image G1 acquired from the terminal device 2, information indicating the first region G11, and information indicating the classification destination C1 in the storage unit 1a.

  Here, the information indicating the first region G11 may be a marker M1 for specifying the first region G11 in the image G1. For example, the marker M1 may be information representing a polygon or an ellipse surrounding the first region G11 (in this case, the inner region surrounded by the marker M1 is referred to as the first region G11). Alternatively, the marker M1 may be information representing a line segment or two line segments that intersect each other. For example, when the marker M1 is a line segment, the calculation unit 1b can specify the first area G11 according to a predetermined rule such as setting the first area G11 inside a rectangle surrounding a predetermined area including the line segment. Alternatively, when the marker M1 is two line segments intersecting each other, the calculation unit 1b performs the first rule according to a predetermined rule such that the inside of a predetermined range circle including the intersection of the two line segments is the first region G11. The region G11 can be specified. For example, the user U1 can input the marker M1 to be superimposed on the image G1 by operating the terminal device 2. In this case, the marker M1 is input to the information processing apparatus 1 as information on a locus input to the image G1 by the user U1.

  The computing unit 1b acquires the image, information indicating the first area of the image, and information indicating the classification destination of the image from the terminal device 2, and stores the acquired information in the storage unit 1a. Then, the computing unit 1b obtains the first detection result of the plurality of types of feature amounts in the first region and the second detection result of the plurality of types of feature amounts in the second region other than the first region in the image. Compare. According to the comparison, the calculation unit 1b determines the weight of each of the plurality of types of feature amounts when classifying the image to the acquired classification destination (the degree of influence on the classification, such as the degree of influence and the priority). Good). For example, the weight is represented by a numerical value. The greater the weight value, the greater the influence on the classification, and the smaller the weight value, the smaller the influence on the classification. Furthermore, the calculation unit 1b performs weighting of each of the plurality of types of feature amounts according to the determined weight, and learns image classification based on the weighting result.

  For example, as described above, the calculation unit 1b allows the user U1 to input the image G1, the marker M1 (trajectory information) indicating the first region G11, and the classification destination information of the image G1. Then, the calculation unit 1b obtains first detection results of a plurality of types of feature amounts in the first region G11 of the image G1. Further, the calculation unit 1b obtains second detection results of a plurality of types of feature amounts in the second region G12 other than the first region G11 of the image G1. Here, as an example, three types of feature amounts X, Y, and Z are considered. A space having dimensions of a plurality of types of feature values may be referred to as a feature value space. When considering three types of feature amounts X, Y, and Z, a three-dimensional feature amount space having three axes corresponding to the feature amounts X, Y, and Z can be considered. In this case, the detection result of the three types of feature amounts may be considered as a combination of three coordinates with respect to the three axes.

  For example, the computing unit 1b obtains feature amounts X = x1, Y = y1, and Z = z1 in the first region G11 as the first detection result. In addition, the calculation unit 1b obtains feature amounts X = x2, Y = y2, and Z = z2 in the second region G12 as the second detection result. Then, the calculation unit 1b compares the first detection result with the second detection result. Specifically, the computing unit 1b compares the same type of feature quantities and identifies the feature quantity having the largest difference. In this case, it is assumed that the feature amount having the largest difference is the feature amount X. Then, the calculation unit 1b makes the weight of the feature amount X when classifying the image G1 to the classification destination C1 larger than the weights of the feature amounts Y and Z. This is because the feature quantity having the largest difference in the first detection result and the second detection result is highly likely to be the feature quantity focused on when the classification destination is determined as the classification destination C1.

  For example, the calculation unit 1b may use the ratio of the number of times determined to have the largest difference as the weight of each of the plurality of types of feature amounts. Specifically, when the total number of input images is one and the feature amount having the largest difference between the attention region and the non-attention region is determined as the feature amount X (when voting for the feature amount X), The calculation unit 1b sets the weight of the feature quantity X to “1”, the weight of the feature quantity Y to “0”, and the weight of the feature quantity Z to “0”. Alternatively, the total number of images input up to this time, and the number of times that the feature amount having the largest difference between the attention region and the non-attention region is determined to be the feature amount X (the number of votes for the feature amount X) ) Is 6 times, the calculation unit 1b sets the weight of the feature amount X to “6/10 = 0.6”. In addition, the total number of images input up to this time is 10. The number of times that the feature amount having the largest difference between the attention region and the non-attention region is determined to be the feature amount Y (the number of votes for the feature amount Y). ) Is one time, the calculation unit 1b sets the weight of the feature amount Y to “1/10 = 0.1”. Furthermore, the total number of images input up to this time is 10. The number of times that the feature amount having the largest difference between the attention region and the non-attention region is determined to be the feature amount Z (the number of votes for the feature amount Z). ) Is three times, the calculation unit 1b sets the weight of the feature amount Z to “3/10 = 0.3”. The calculation unit 1b registers the weights of the feature amounts X, Y, and Z thus obtained, for example, in the weight table T1 stored in the storage unit 1a.

  In this way, the arithmetic unit 1b receives the information of the classification destination and the information of the area focused on the determination of the classification destination in the image together with the sequentially arriving images, and the plurality of types of feature amounts X, Y, and Z are received. Among them, the weights of the feature amounts X, Y, and Z for the determination of the classification destination are updated. For example, when the weight is updated with a predetermined number of images, the calculation unit 1b weights each of the plurality of types of feature amounts according to the determined weight. For example, consider a case where the calculation unit 1b determines that the weights of the feature amounts X, Y, and Z are “0.6”, “0.1”, and “0.3”, respectively. In this case, the calculation unit 1b obtains a weighting result obtained by correcting the feature amount X by 6 times, the feature amount Y by 1 time, and the feature amount Z by 3 times for each of the images input so far (however, You may multiply by the value itself).

  The computing unit 1b learns image classification based on the weighting result. More specifically, the computing unit 1b generates an identification model for classifying images using each weighted feature amount. An existing machine learning method (for example, a support vector machine (SVM)) can be used to generate the identification model. When classifying an image using the created identification model, each feature amount extracted from the image to be classified is weighted by the weight registered in the weight table T1, and classification is performed based on the identification model. .

  Here, for example, when it is desired to recognize that “a person's face is reflected in this image” for an image, an image labeled with / without a person's face is displayed. An identification model for classification can be created by preparing a large amount and using supervised learning. However, when there is no data set to be learned at hand, it is unclear what feature quantity should be used for learning. As a simple method, it is conceivable to prepare a feature amount set that is often used for image classification, and to perform learning by treating all feature amounts equally. However, in this method, the dimension of the feature quantity that does not contribute to the original classification becomes noise, and the classification accuracy can be reduced.

  Therefore, the information processing apparatus 1 allows the input of the correct answer or incorrect answer label and the information of the attention area in the image at the time of labeling, and compares the feature amount between the attention area and the non-attention area, Efficiently narrow down the effective features for labeling from the features. This is because it is considered that a feature value having a large value in the vicinity of the attention area (first area) and the other area (second area) has a relatively large influence on the determination of correct / incorrect labeling. Also, for example, if the shape specified as the region of interest is uneven, it is labeled with attention to the features of the edge system, and if the shape is rectangular or elliptical, the color, brightness, etc. with the spreading direction component of the shape It can also be estimated that the features are labeled with attention to features that have sudden changes.

  Thus, according to the information processing apparatus 1, learning can be made efficient. In particular, since learning can be performed by narrowing down to feature quantities that are estimated to have a large contribution to the original classification, classification accuracy can be improved as compared to learning by treating all types of feature quantities equally. Furthermore, since it is expected that the classification model converges faster than when all types of feature values are handled equally, the number of images required for learning can be reduced, and the learning costs (time for learning) can be reduced. And user work costs).

  In particular, in the case of online learning (a method in which image data is generated every moment and learning is performed using sequentially generated image data), since there is no set of image data for learning at hand, which feature quantity is classified Trial and error can not be done. On the other hand, according to the information processing apparatus 1, feature quantities that are highly likely to be effective for classification can be narrowed down using sequentially arrived image data. For this reason, the information processing apparatus 1 is useful when performing online learning.

  Next, the function of the information processing apparatus that performs learning by the method described in the first embodiment will be described more specifically. For example, the information processing apparatus 1 may be realized by a computer (including a server computer and a client computer).

[Second Embodiment]
FIG. 2 is a diagram illustrating an example of hardware of the image processing server according to the second embodiment. The image processing server 100 includes a processor 101, a RAM 102, an HDD 103, an image signal processing unit 104, an input signal processing unit 105, a medium reader 106, and a communication interface 107. Each unit is connected to the bus of the image processing server 100.

  The processor 101 controls information processing of the image processing server 100. The processor 101 may be a multiprocessor. The processor 101 is, for example, a CPU, DSP, ASIC, or FPGA. The processor 101 may be a combination of two or more elements of CPU, DSP, ASIC, FPGA, and the like.

  The RAM 102 is a main storage device of the image processing server 100. The RAM 102 temporarily stores at least part of an OS (Operating System) program and application programs to be executed by the processor 101. The RAM 102 stores various data used for processing by the processor 101.

  The HDD 103 is an auxiliary storage device of the image processing server 100. The HDD 103 magnetically writes and reads data to and from the built-in magnetic disk. The HDD 103 stores an OS program, application programs, and various data. The image processing server 100 may include other types of auxiliary storage devices such as flash memory and SSD (Solid State Drive), or may include a plurality of auxiliary storage devices.

  The image signal processing unit 104 outputs an image to the display 11 connected to the image processing server 100 in accordance with an instruction from the processor 101. As the display 11, a CRT (Cathode Ray Tube) display, a liquid crystal display, or the like can be used.

  The input signal processing unit 105 acquires an input signal from the input device 12 connected to the image processing server 100 and outputs it to the processor 101. As the input device 12, for example, a pointing device such as a mouse or a touch panel, a keyboard, or the like can be used.

  The medium reader 106 is a device that reads programs and data recorded on the recording medium 13. As the recording medium 13, for example, a magnetic disk such as a flexible disk (FD) or an HDD, an optical disk such as a CD (Compact Disc) or a DVD (Digital Versatile Disc), or a magneto-optical disk (MO) is used. Can be used. Further, as the recording medium 13, for example, a non-volatile semiconductor memory such as a flash memory card can be used. For example, the medium reader 106 stores the program and data read from the recording medium 13 in the RAM 102 or the HDD 103 in accordance with an instruction from the processor 101.

  The communication interface 107 communicates with other devices via the network 10. The communication interface 107 may be a wired communication interface or a wireless communication interface.

Here, the image processing server 100 is an example of the information processing apparatus 1 according to the first embodiment.
FIG. 3 is a diagram illustrating an example of functions of the image processing server. The image processing server 100 includes a storage unit 110, a label reception unit 120, a feature extraction unit 130, a priority feature amount determination unit 140, a feature weighting unit 150, and an identification model generation unit 160. The storage unit 110 is realized as a storage area secured in the RAM 102 or the HDD 103. The label receiving unit 120, the feature extracting unit 130, the priority feature amount determining unit 140, the feature weighting unit 150, and the identification model generating unit 160 are realized by the processor 101 executing a program stored in the RAM 102.

  The storage unit 110 stores an attention area determination rule, a feature amount table, and a feature amount priority table. The attention area determination rule is information defining a determination rule for an attention area in an image according to the type of marker (information different from image information) given to the image. Here, the attention area indicates an area that is noticed for determining a label when the user labels a certain image as correct / incorrect. The type of marker is classified according to the shape of the marker, for example. Specifically, various types are conceivable, such as a closed line, one line segment that is not closed, and two line segments that intersect each other.

  The feature amount table is information indicating the correspondence relationship between an image, a label for the image, and a plurality of types of feature amounts in the image. The feature amount priority table is information indicating priorities used for classifying images of a plurality of types of feature amounts. It can be said that the feature amount priority table is information indicating weights for the image classification of each of a plurality of types of feature amounts. Note that a plurality of types of feature quantities to be analyzed are set in the image processing server 100 in advance.

  The label accepting unit 120 accepts input of an image, a label for the image, and information indicating a region of interest that has been noted at the time of labeling. For example, the user can confirm the image displayed on the display 11, determine a label to be given to the image, and input the label to the image processing server 100 using the input device 12. At that time, the user can input information (marker) indicating the region of interest focused on for the label determination to the image processing server 100 using the input device 12. The marker is information for specifying a certain area in the corresponding image. The marker information is input to the display 11 as information separate from the image information. A user can also input an image, a label for the image, and information indicating a region of interest into the image processing server 100 using a client computer connected to the network 10. The label receiving unit 120 stores the received image and marker information in the storage unit 110. The label receiving unit 120 registers the received label in the feature amount table stored in the storage unit 110 in association with the identification information of the image.

  The feature extraction unit 130 extracts a plurality of types of feature amounts to be analyzed from the input image. The feature extraction unit 130 registers the extracted feature amount in the feature amount table stored in the storage unit 110 in association with the identification information of the image.

  The priority feature amount determination unit 140 identifies a region of interest in the image based on the received image and marker information. Based on the attention area determination rules stored in the storage unit 110, the priority feature amount determination unit 140 identifies an attention area corresponding to the shape of the marker from the image. The priority feature amount determination unit 140 extracts a plurality of types of feature amounts in the attention area. The priority feature amount determination unit 140 extracts a plurality of types of feature amounts in the non-attention area. The priority feature amount determination unit 140 compares feature amounts of the same type extracted for the attention area and the non-attention area. The priority feature amount determination unit 140 determines priorities for labeling (classification) of the plurality of types of feature amounts according to the comparison. The priority feature amount determination unit 140 registers the determined priority in the feature amount priority table stored in the storage unit 110.

The feature weighting unit 150 weights each feature amount registered in the feature amount table using the priorities of the plurality of types of feature amounts in the feature amount priority table.
The identification model generation unit 160 generates an identification model for classifying images using each feature amount weighted by the feature weighting unit 150. An existing method such as SVM can be used to generate the identification model.

  FIG. 4 is a diagram illustrating an example of the attention area determination rule. The attention area determination rule 111 is stored in the storage unit 110 in advance. The attention area determination rule 111 includes items of a marker type and an attention area.

In the marker type item, the type of marker classified by the shape of the marker is registered. In the attention area item, a method of determining the attention area according to the type of marker is registered.
For example, in the attention area determination rule 111, information that the marker type is “closed line” and the attention area is “area inside the closed line” is registered. This indicates that when the marker type is “closed line”, the region of interest is “an area inside the closed line”. The “closed line” is, for example, a line that borders the outer periphery of a certain region. The “closed line” can also be referred to as a line connecting the start point and the end point. The marker M11 is an example of a “closed line”. When the marker M11 is designated, the attention area is an area R11 inside the marker M11.

  Further, for example, in the attention area determination rule 111, information that the marker type is “one line segment” and the attention area is “an area inside a rectangle of a predetermined size including the periphery of the line segment” is registered. This indicates that when the marker type is “one line segment”, the region of interest is “a region inside a rectangle of a predetermined size including the periphery of the line segment”. The “rectangle having a predetermined size including the periphery of the line segment” may be, for example, a rectangle having a predetermined area including the periphery of the line segment. “A rectangle of a predetermined size including the periphery of a line segment” means two parallel sides having the same length as the line segment extending in the length direction of the line segment and a predetermined length perpendicular to the two sides. The rectangle which consists of these two sides may be sufficient. The “line segment” may be a line obtained by approximating a straight line to a line represented by an “unclosed line” (a line having a different start point and end point). The marker M12 is an example of “one line segment”. When the marker M12 is designated, the attention area is, for example, an area R12 inside a rectangle having a predetermined size including the marker M12.

  Also, for example, in the attention area determination rule 111, information that the marker type is “a plurality of intersecting line segments” and the attention area is “area around the intersection” is registered. This indicates that when the marker type is “a plurality of intersecting line segments”, the region of interest is “region around the intersection”. The “region around the intersection” may be, for example, a region inside a circle with a predetermined radius centered on the intersection. The marker M13 is an example of “a plurality of intersecting line segments”. The marker M13 is a marker representing a shape in which two line segments intersect. When the marker M13 is designated, the attention area is, for example, an area R13 inside a circle with a predetermined radius centered on the intersection of the markers M13.

  As described above, the attention area determination rule 111 registers the correspondence between the marker type and the attention area specifying method. For the method of specifying the marker type and the region of interest, other types and specifying methods than those described above can be used.

  FIG. 5 is a diagram illustrating an example of the feature amount table. The feature amount table 112 is stored in the storage unit 110. In the feature amount table 112, extraction results of a plurality of predetermined types of feature amounts are registered for each input image. In the feature amount table 112, the label value input for each image is also registered. For example, the value of the label is either “true” or “false”. In the example of the feature quantity table 112, the feature quantity names such as “feature quantity A”, “feature quantity B”, “feature quantity C”,... The bottom row of the “Item” column is “Label”. In the example of the feature quantity table 112, the top row represents “image name”. The image names are, for example, “image P1”, “image P2”,. Each value in the column of “image P1” is a feature value and a label value corresponding to the image P1.

  In the example of the feature amount table 112, for the image P1, the feature amount A is “0.20”, the feature amount B is “0.56”, the feature amount C is “0.11”, and the label is “true”. ". For the image P2, the feature amount A is “0.30”, the feature amount B is “0.14”, the feature amount C is “0.67”, and the label is “false”. In the feature amount table 112, the extraction result of each feature amount and the value of the label are registered for each sequentially arrived image.

  FIG. 6 is a diagram illustrating an example of a feature amount priority table. The feature amount priority table 113 is stored in the storage unit 110. The feature amount priority table 113 is an example of the weight table T1 according to the first embodiment. The feature quantity priority table 113 includes items of feature quantity names and priorities.

  The feature quantity name is registered in the feature quantity name item. A priority value is registered in the priority item. The priority value is a value between 0 and 1. The higher the value, the higher the priority. The total priority of each feature quantity is normalized to “1”. Specifically, the priority is the ratio of the number of times that each of the images has been voted as a feature amount that is the largest difference between the attention area and the non-attention area to the total number of each image. Note that the initial value of the priority of each feature quantity in the feature quantity priority table 113 is “0”.

  For example, in the feature quantity priority table 113, information that the feature quantity name is “feature quantity A” and the priority is “0.3” is registered. This indicates that the priority of the feature amount A is “0.3”. In the feature quantity priority table 113, information that the feature quantity name is “feature quantity B” and the priority is “0.6” is registered. This indicates that the priority of the feature amount B is “0.6”. In the feature quantity priority table 113, information that the feature quantity name is “feature quantity C” and the priority is “0.1” is registered. This indicates that the priority of the feature amount C is “0.1”. In this case, the feature quantities other than the feature quantities A, B, and C have the priority “0”. Among the feature quantities A, B, and C, the feature quantity B has the highest priority, the feature quantity A has the second highest priority, and the feature quantity C has the lowest priority.

  FIG. 7 is a diagram illustrating an example of the feature space. For example, the feature space is represented by a plurality of axes corresponding to a plurality of types of feature amounts used for image classification. FIG. 7 illustrates a feature space represented by three axes of a color histogram, brightness distribution, and texture. For example, the color histogram is a bar graph representing how many pixels of each color exist in the image. For example, the feature extraction unit 130 normalizes the image color histogram to a value between 0 and 1 as one feature amount (however, an axis may be provided for each of a plurality of colors). Similarly, the feature extraction unit 130 normalizes the brightness distribution and texture to values of 0 to 1 as feature amounts. In the example of FIG. 7, the feature space is represented by three types of feature amounts. However, other types of feature amounts may be used instead of or in addition to the illustrated types of feature amounts. .

  FIG. 8 is a diagram illustrating an example of updating priority. For example, when receiving an input of a certain image to which a marker is attached, the priority feature amount determination unit 140 extracts a color histogram, a brightness distribution, and a texture feature amount for each of the attention area and the non-attention area. The priority feature amount determination unit 140 votes the feature amount having the largest difference between the attention area and the non-attention area, and determines the priority. In the example of FIG. 8, the priority feature amount determination unit 140 illustrates a case where each feature amount is extracted for the first input image in a state where the priority of each feature amount is the initial value “0”. . At this time, for the color histogram, it is assumed that the feature amount difference between the attention area and the non-attention area is the largest. In this case, the priority feature amount determination unit 140 votes for the color histogram. Since this is the first determination, the priority of the color histogram in the feature amount priority table 113 is updated to “1”. In this way, the priority feature amount determination unit 140 updates the priority of each feature amount based on the ratio of the number of votes to the number of determinations.

  FIG. 9 is a diagram illustrating an example of weighting feature amounts. The feature weighting unit 150 weights the feature amount registered in the feature amount table 112 with the priority registered in the feature amount priority table 113. For example, the feature weighting unit 150 sets c × (a × 10) as a weighted feature amount for the feature amount c and the priority a (a is a real number between 0 and 1). More specifically, the feature amount of the brightness distribution of the image P1 in the feature amount table 112 is c = “0.20”, and the priority of the brightness distribution of the feature amount priority table 113 is a = “0.2”. Consider the case. In this case, the feature weighting unit 150 weights the feature value of the brightness distribution of the image P1 as 0.20 × (0.2 × 10) = 0.20 × 2 = 0.40. Similarly, the feature weighting unit 150 weights each feature quantity in the feature quantity table 112. As a result, the feature weighting unit 150 creates a weighted feature value table 112 a based on the feature value table 112. The identification model generation unit 160 generates an identification model based on the weighted feature quantity table 112a. That is, the image processing server 100 weights each of the plurality of types of feature amounts according to the priority, and generates an identification model used for image classification based on the weighting result.

  The use of the weighted feature quantity table 112a when generating the identification model means that the scale of the axis corresponding to each feature quantity in the feature space is used using the weight according to the priority of the feature quantity priority table 113. You may think that it corresponds to converting. In this way, the identification model generation unit 160 can generate an identification model by making feature quantities that have a large influence on classification (labeling) more prominent.

FIG. 10 is a flowchart illustrating an example of learning. In the following, the process illustrated in FIG. 10 will be described in order of step number.
(S1) The label receiving unit 120 determines whether an image with a marker (image information and information indicating coordinates corresponding to a marker region in the image) is received together with a labeling result. When an image with a marker is received, the feature extraction unit 130 extracts a feature amount from the received image, registers it with the label in the feature amount table 112, and advances the process to step S2. If an image with a marker has not been received, the process proceeds to step S1 (waiting for the arrival of an image with a marker). As described above, the user can input information (marker) indicating a region of interest focused on for label determination into the image processing server 100 using the input device 12 such as a mouse, a touch pad, a touch panel, and a keyboard. Alternatively, the label receiving unit 120 may receive a labeling result and an image with a marker from a client computer connected to the network 10.

  (S2) The priority feature amount determination unit 140 updates the feature amount priority table 113 based on the feature amounts of the attention area and the non-attention area specified by the markers in the image received in step S1. Details of the processing will be described later.

  (S3) The feature weighting unit 150 creates a weighted feature value table 112a based on the feature value table 112 and the feature value priority table 113. The identification model generation unit 160 generates an identification model for classifying images using the weighted feature amount table 112a. Then, the process proceeds to step S1.

  For example, after inputting a plurality of images, the image processing server 100 performs steps until the labeling of “true” and “false” of an existing image can be reproduced with a predetermined accuracy using the generated identification model. It is conceivable to repeat the processing of S1 to S3.

  More specifically, the image processing server 100 registers the feature value extraction result for the image input for the Nth time (for example, N is an integer of 2 or more) in the feature value table 112. In addition, the priority feature amount determination unit 140 updates the feature amount priority table 113 based on the marker for the image. The feature weighting unit 150 creates a weighted feature quantity table 112 a based on the feature quantity table 112 and the feature quantity priority table 113. The identification model generation unit 160 generates an identification model based on the weighted feature quantity table 112a. If the identification model reproduces the labeling of the N images at a predetermined rate (for example, 95%), the image processing server 100 ends the above learning procedure, and finally generates it thereafter. The arrived image is classified using the identified identification model. On the other hand, if the identification model does not reproduce the labeling for the N images with a predetermined accuracy, the image processing server 100 executes the above learning procedure for the image input for the N + 1th time, The identification model is updated and the labeling reproduction rate is evaluated for N + 1 images. In this way, the image processing server 100 updates the identification model and increases the classification accuracy.

  FIG. 11 is a flowchart illustrating an example of updating the feature amount priority table. In the following, the process illustrated in FIG. 11 will be described in order of step number. The procedure shown below corresponds to step S2 in FIG.

  (S11) Based on the attention area determination rule 111 stored in the storage section 110, the priority feature amount determination section 140 determines the attention area according to the marker assigned to the received image. Details of the processing will be described later.

(S12) The priority feature amount determination unit 140 calculates a plurality of types of feature amounts in each of the attention area and the non-attention area.
(S13) The priority feature amount determination unit 140 determines the feature amount having the largest difference between the feature amounts in the attention area and the non-attention area.

  (S14) The priority feature amount determination unit 140 votes for the feature amount determined in step S13, and updates the priority of each feature amount registered in the feature amount priority table 113. The total number of votes for a plurality of types of feature values is N. Let n (i) (n is an integer greater than or equal to 0) be the number of votes for a certain feature value i (i indicates the type of feature value). In this case, the total of n (i) for each feature amount is N. Then, the priority feature amount determination unit 140 sets the priority of the feature amount to n (i) / N. The priority feature amount determination unit 140 may hold the values of N and n (i) for each feature amount in the storage unit 110.

  FIG. 12 is a flowchart showing an example of attention area determination. In the following, the process illustrated in FIG. 12 will be described in order of step number. The procedure shown below corresponds to step S11 in FIG.

  (S21) The priority feature amount determination unit 140 determines whether or not there is a closed region indicated by the marker with respect to the marker and image received in step S1. If there is a closed region, the process proceeds to step S22. If there is no closed area, the process proceeds to step S23. The “closed region” corresponds to the marker being a closed line. Therefore, the priority feature amount determination unit 140 may determine the determination in step S21 based on “whether or not the marker is a closed line”. In this case, if the marker is a closed line, the process proceeds to step S22. On the other hand, if the marker is not a closed line, the process proceeds to step S23.

  (S22) Based on the attention area determination rule 111, the priority feature amount determination unit 140 determines the inside of the closed area (that is, the marker represented by the closed line) as the attention area. The priority feature amount determination unit 140 determines an area other than the attention area in the image received in step S1 as a non-attention area. Then, the process ends.

  (S23) The priority feature amount determination unit 140 determines whether or not there is an intersection on the locus indicated by the marker. If there is an intersection, the process proceeds to step S24. If there is no intersection, the process proceeds to step S25. “There are intersections in the trajectory” means, for example, the intersections of a plurality of line segments as described above when the markers intersect with each other.

  (S24) Based on the attention area determination rule 111, the priority feature amount determination unit 140 determines the area around the intersection as the attention area. Specifically, the priority feature amount determination unit 140 determines an area inside a circle with a predetermined radius centering on the intersection as the attention area. Then, the process ends.

  (S25) Based on the attention area determination rule 111, the priority feature amount determination unit 140 determines a rectangle that covers the locus portion indicated by the marker as the attention area. Here, if No in step S21 and No in step S23, the marker corresponds to “one line segment” (not necessarily a completely straight line). In this case, the priority feature amount determination unit 140 determines an area inside a rectangle having a predetermined size including the periphery of the line segment as the attention area. Alternatively, the priority feature amount determination unit 140 includes two parallel sides having the same length as the line segment extending in the length direction of the line segment, and two sides having a predetermined length orthogonal to the two sides. The area inside the rectangle may be determined as the attention area. Then, the process ends.

  A plurality of markers may be specified in one image. The priority feature amount determination unit 140 may extract a plurality of attention areas from one image according to the shapes of the plurality of markers. In that case, the priority feature amount determination unit 140 may consider a plurality of attention regions as one attention region, obtain each feature amount, and compare it with each feature amount extracted from the non-attention region.

  By the way, for example, when it is desired to classify a person's face as “reflected” or “not reflected” for an image, an image with a correct answer (shown) / incorrect answer (not shown) is attached. An identification model for classification can be created by preparing a large amount and using supervised learning. However, when starting supervised learning, a learning image set (a set of training data) may not be at hand. If a learning image set is not available, learning may be performed using sequentially generated image data. However, in this case, it is difficult to determine which feature amount should be used for learning. In this case, it is conceivable to perform learning by treating all types of feature quantities equally. However, if all types of feature quantities are treated equally, the feature quantities that do not contribute to the original classification become noise, and the classification accuracy can be reduced. Specifically, it is as follows.

  FIG. 13 is a diagram illustrating the relationship between the classification and the feature amount. FIG. 13A shows an example of feature quantities effective for classification. For example, by using the feature amounts A and B, a plurality of images can be classified into appropriate classification destinations by a certain identification model (for example, a hyperplane). On the other hand, FIG. 13B shows an example of feature amounts that are not effective for classification. For example, when the feature amounts C and D are used, it becomes difficult to search for an identification model that can appropriately classify a plurality of images. For this reason, when learning is performed by equally treating the four types of feature quantities A, B, C, and D, the feature quantities that do not contribute to the original classification may become noise.

  Therefore, the image processing server 100 allows the input of the correct answer or incorrect answer label and the information on the attention area in the image at the time of labeling, compares the feature amount between the attention area and the non-attention area, Efficiently narrow down the effective features for labeling from the features. This is because it is considered that a feature quantity having a large value in the attention area and the other non-attention areas has a relatively large influence on determination of correct / incorrect labeling. Thus, according to the image processing server 100, learning can be made efficient. In particular, since learning can be performed by narrowing down to feature quantities that contribute to the original classification, classification accuracy can be improved as compared to learning by treating all types of feature quantities equally. Furthermore, since it is expected that the classification model converges faster than when all types of feature values are handled equally, the number of images required for learning can be reduced, and the learning costs (time for learning) can be reduced. And user work costs).

  In particular, in the case of online learning (a method in which image data is generated every moment and learning is performed using sequentially generated image data), since there is no set of image data for learning at hand, which feature quantity is classified Trial and error can not be done. On the other hand, according to the image processing server 100, priorities can be given to feature quantities that are likely to be effective for classification in order using image data that arrives in sequence, which can be used to generate an identification model. For this reason, the image processing server 100 is particularly useful when performing online learning using sequentially arrived image data.

  Next, a modification of the second embodiment will be described. First, the image processing server 100 may update the feature amount priority table 113 by the following procedure instead of the procedure of FIG.

  FIG. 14 is a flowchart illustrating another example of the feature amount priority table update. In the following, the process illustrated in FIG. 14 will be described in order of step number. The procedure shown below corresponds to step S2 in FIG. Here, the procedure of FIG. 14 is different from the procedure of FIG. 11 in that step S11a is executed next to step S11, and step S12 is executed next to step S11a. Therefore, in the following, step S11a will be mainly described, and description of other steps will be omitted.

  (S11a) The priority feature amount determination unit 140 determines the weighting to each feature amount according to the marker received in step S1 of FIG. For example, if the shape of the marker is a “closed line”, the weight of the feature amount classified into the global feature amount is set larger than the weights of the other feature amounts. More specifically, it is conceivable to weight the feature quantities classified into the global feature quantities twice and weight other feature quantities one time. Here, the global feature value is a feature value representing the entire image, and for example, a color histogram can be considered. As another example, when the shape of the marker is “a plurality of intersecting line segments”, the weight of the feature amount classified into the local feature amount is set larger than the weight of the other feature amount. Here, the local feature amount is a feature amount expressing a certain point, and, for example, SIFT (Scale-Invariant Feature Transform) can be considered. Then, the process proceeds to step S12.

  The image processing server 100 weights each of a plurality of types of feature amounts in accordance with the shape of the marker (that is, trajectory information). For example, the priority feature amount determination unit 140 may use the weighting value determined in step S11a when obtaining the difference in feature amount in step S13. Specifically, if the global feature value is weighted more than other feature values, the priority feature value determination unit 140 performs step S11a on the global feature value extraction result in the attention area and the non-attention area. Multiplying the weights obtained in step (3), the difference between the feature amounts is obtained. Then, the priority feature amount determination unit 140 can determine the voting rate (weight for classification) for each feature amount according to the shape of the marker. In addition, the user can easily designate the type of feature amount to be focused on to the image processing server 100 according to the shape of the marker.

  Alternatively, the feature weighting unit 150 may further consider the weight obtained in step S11a when weighting the feature amount in step S3 of FIG. 10 executed after the update of the feature amount priority table 113. For example, it is conceivable that the feature weighting unit 150 obtains a weighted feature amount by multiplying a certain feature amount by both the weight according to the vote rate and the weight obtained in step S11a. Also in this case, the user can easily specify the type of feature amount to be focused on to the image processing server 100 by the shape of the marker.

  In the example of the second embodiment, the identification model is updated every time one image arrives. However, after a certain number of images are accumulated, the feature amount priority table 113 and the identification model are updated. May be updated. For example, the label receiving unit 120 counts the number of times determined as Yes in step S1 of FIG. 10, and when the number of times reaches a predetermined number, resets the count of the number of times determined as Yes in step S1, The process proceeds to step S2. If the number of times determined Yes in step S1 in FIG. 10 has not reached the predetermined number, the label receiving unit 120 waits for the arrival of an image in the case of No in step S1. The feature extraction unit 130 extracts each feature amount from each image and registers it in the feature amount table 112 together with the received label. Further, the label receiving unit 120 stores the marker information input together with the image in the storage unit 110 in association with the image.

It is conceivable that the priority feature amount determination unit 140 updates the feature amount priority table 113 based on the information of the plurality of images, labels, and markers thus held.
FIG. 15 is a diagram illustrating another example of priority update. Here, it is assumed that the priority feature amount determination unit 140 determines the feature amount of the vote destination for the three images and updates the feature amount priority table 113. In this case, the priority feature amount determination unit 140 extracts a plurality of types of feature amounts determined in advance for each region of interest of the three images, and coordinates on the feature space represented by the plurality of types of feature amounts. Ask. That is, the priority feature amount determination unit 140 obtains three coordinates on the feature space for the attention area of each of the three images. Then, the priority feature amount determination unit 140 obtains the center of gravity of the obtained three coordinates. Then, each component of the obtained center of gravity can be regarded as various feature amounts related to the attention area of the three images.

  Similarly, the priority feature amount determination unit 140 extracts a plurality of types of feature amounts for the non-attention areas of the three images, and obtains three coordinates on the feature space. Then, the priority feature amount determination unit 140 obtains the center of gravity of the obtained three coordinates. Then, each component of the obtained center of gravity can be regarded as various feature amounts related to the non-attention areas of the three images.

  The priority feature quantity determination unit 140 compares the components of the two centroids obtained for the attention area and the non-attention area, and identifies the feature quantity having the largest difference. Then, the priority feature amount determination unit 140 votes for the identified feature amount and updates the feature amount priority table 113. When the feature amount priority table 113 is updated, the feature weighting unit 150 creates a weighted feature amount table 112 a based on the feature amount table 112 and the feature amount priority table 113. When the weighted feature quantity table 112a is newly created, the identification model generation unit 160 updates the identification model for image classification based on the weighted feature quantity table 112a.

  As described above, the image processing server 100 may update the feature amount priority table 113 and the identification model every time a plurality of images arrive. In this case, it is possible to reduce the influence of noise (for example, the possibility of voting for a feature amount having a relatively small contribution to the classification) and further improve the classification accuracy, rather than updating every time an image arrives. .

  Note that the information processing of the first embodiment can be realized by causing the computing unit 1b to execute a program. The information processing according to the second embodiment can be realized by causing the processor 101 to execute a program. The program can be recorded on a computer-readable recording medium 13.

  For example, the program can be distributed by distributing the recording medium 13 on which the program is recorded. Alternatively, the program may be stored in another computer and distributed via a network. For example, the computer stores (installs) a program recorded in the recording medium 13 or a program received from another computer in a storage device such as the RAM 102 or the HDD 103, and reads and executes the program from the storage device. Good.

DESCRIPTION OF SYMBOLS 1 Information processing apparatus 1a Memory | storage part 1b Operation part 2 Terminal device G1 Image G11 1st area | region G12 2nd area | region M1 Marker T1 Weight table U1 User

Claims (8)

Computer
Obtaining an image, information indicating a first region of the image, and information indicating a classification destination of the image;
According to a comparison of a first detection result of a plurality of types of feature amounts in the first region and a second detection result of the plurality of types of feature amounts in a second region other than the first region of the image. , Determining the weight of each of the plurality of types of feature amounts when classifying the image to the classification destination,
Based on the result of weighting each of the plurality of types of feature amounts by the weight, learning of the classification of the image is performed.
Learning method.   2. In the determination, among the plurality of types of feature amounts, a weight of a feature amount having the largest difference between the first detection result and the second detection result is set larger than the weights of other feature amounts. The learning method described.   In the determination, the weight of each of the plurality of types of feature amounts determined based on the second image acquired last time according to the comparison of the first detection result and the second detection result regarding the first image acquired this time The learning method according to claim 1, wherein the learning method is updated.   The learning method according to any one of claims 1 to 3, wherein, in the acquisition, a method for acquiring the first region is selected according to trajectory information input to the image.   The learning method according to claim 4, wherein in the determination, the weight of each of the plurality of types of feature amounts is determined according to the information of the trajectory.   The learning method according to claim 4, wherein in the acquisition, the user is allowed to input the classification destination of the image and the locus. A storage unit that stores an image, information indicating a first area of the image, and information indicating a classification destination of the image;
According to a comparison of a first detection result of a plurality of types of feature amounts in the first region and a second detection result of the plurality of types of feature amounts in a second region other than the first region of the image. Determining the weight of each of the plurality of types of feature amounts when classifying the image to the classification destination, and based on the result of weighting each of the plurality of types of feature amounts by the weight, An arithmetic unit for learning;
An information processing apparatus. On the computer,
Obtaining an image, information indicating a first region of the image, and information indicating a classification destination of the image;
According to a comparison of a first detection result of a plurality of types of feature amounts in the first region and a second detection result of the plurality of types of feature amounts in a second region other than the first region of the image. , Determining the weight of each of the plurality of types of feature amounts when classifying the image to the classification destination,
Based on the result of weighting each of the plurality of types of feature amounts by the weight, learning of the classification of the image is performed.
A learning program that executes processing.

Images