JP2011070408A - Method of acquiring sample image, sample image acquisition device, and image classification device - Google Patents

Abstract

An object of the present invention is to obtain an image classification device that can improve the efficiency of collecting sample images and can be adapted to the diversity of classification categories and image search keywords.
Web image search means 2 for specifying a keyword and collecting images in a plurality of HTML pages on which the keyword is posted, and a low-dimensional color, shape, pattern, etc. from the collected Web search image or input image Feature quantity conversion means 3 that measures signal values and outputs them as image feature quantities; sample image identification means 4 that identifies image feature quantities generated from a plurality of Web search images as sample images related to keywords; and sample images The image feature quantity corresponding to is extracted as an image feature quantity representative of the classification category whose name is the designated keyword, while the image feature quantity generated from the input image whose classification category is unknown, and the classification category Classification category determining means 5 is provided for calculating a similarity with a representative image feature and classifying an input image into a predetermined classification destination based on the similarity.
[Selection] Figure 1

Description

  The present invention relates to an image classification apparatus that classifies a large number of images into a predetermined classification destination, and adaptively generates image feature amounts corresponding to a specified keyword, and a user desires using the generated image feature amounts. The present invention relates to an image classification apparatus for realizing classification from the viewpoint of performing image classification, and an image classification apparatus having a function of assigning a keyword for image search for searching for an image desired by a user from a large number of images.

  The present invention also relates to a sample image acquisition method and apparatus used in an image classification apparatus, and more particularly to a sample image acquisition method and sample image acquisition apparatus that automatically acquire a sample image from a Web image by specifying a keyword.

With the recent spread of digital cameras, anyone can easily take a large number of photos. Therefore, a method for efficiently browsing an enormous number of images (photos) stored in the storage is required.
In the past, photographs printed on paper have been generally improved in browsing efficiency by sorting and storing the photographs by date, person, and scene. On the other hand, in digital images recorded on a recording medium, keywords that accurately represent the contents of the images are assigned to the images and stored, and when browsing, the desired images can be searched directly by specifying the keywords. It became possible to browse well.

  In this case, the keywords are things associated with the image, such as a person name, object name, scene, date, place, scene, situation, scene, event, impression, meaning, intention, etc. Everything is true.

However, since keyword assignment is performed manually, it takes time to visually check the contents of the images one by one, and consider the appropriate keyword to assign to the image. The number of images to be processed As the number increased, it was a great effort that could not be ignored.
Therefore, in order to reduce the work burden required for assigning keywords to images, image classification devices and image search keyword assignment devices as disclosed in Patent Documents 1 to 3 have been proposed.

  The devices described in Patent Document 1 and Patent Document 2 are provided with a color, shape, or pattern extracted from an input image indicating which of the classification categories the input image belongs to. The image classification device automatically determines the image feature amount based on the similarity between the image feature amount extracted in advance from the sample image representing each classification category.

  In particular, the image classification device described in Patent Literature 2 classifies significant feature amounts that are significantly different from sample images of other categories among the feature amounts extracted from sample images that represent the classification categories. By using it as an image feature amount for, there is an effect of greatly improving the accuracy of category classification.

  Patent Document 3 is characterized in that, using the result of category classification by the image classification device described in Patent Document 1 or Patent Document 2, the name of the classified category is assigned to the image as a keyword. An image search keyword assigning device has been proposed.

JP-A-11-328422 JP 2008-33705 A JP 2002-32751 A

  According to the above-described image classification device and image search keyword assignment device, it is possible to automatically classify an input image and assign a keyword. However, in order to perform image classification, it is necessary to generate an image feature amount of each classification category in advance. For this purpose, it is necessary to collect a large amount of sample images representing each classification category in advance.

As for the sample image collection method in the apparatus described in Patent Literature 1 to Patent Literature 3, since there is no special description in any of the examples, in order to realize automatic classification and keyword assignment, a sample image must be stored in advance. There is a problem that it is necessary to collect manually, and the collection work takes a lot of time and effort.
Therefore, in order to save the trouble of collecting sample images, it has been desired to realize a sample image acquisition method that efficiently collects sample images representing each category as a classification reference, for example, automatically or semi-automatically.

In addition, since the viewpoint of classification / search varies depending on the person, it is difficult to set a classification category and an image search keyword common to all. Even if image feature quantities are generated for as many types of classification categories as possible, there are almost infinite number of classification category and image search keyword candidates, so it is not practical to prepare sample images for all of them. there were.
Therefore, in order to deal with the diversity of classification categories and image search keywords, it has been desired to adaptively set the classification categories according to the user's request and to create the image feature amounts.

  The present invention has been proposed in view of the above circumstances, and provides a sample image acquisition method and apparatus that improves the efficiency of sample image collection, and image classification that can be adapted to the diversity of classification categories and image search keywords. The object is to provide a device.

  In order to achieve the above object, the present invention uses text information and image information stored on the Web, and automatically generates a sample image related to a specified keyword without human intervention or in response to a user request. The main structure is to collect adaptively.

  That is, in the sample image acquisition method of claim 1, in a system that enables connection to an HTML page via a network, a keyword is specified, and images in a plurality of HTML pages on which the keyword is posted are collected from the Web, Low-dimensional signal values such as color, shape, and pattern are measured from a plurality of Web search images collected from the Web and output as image feature amounts, respectively, and the plurality of image feature amounts are separated from distribution characteristics in a feature space. The feature space is clustered with image feature quantities close to each other, and an image belonging to a cluster having the largest distribution number or a cluster having a small variance is specified as a sample image related to a designated keyword.

According to a second aspect of the present invention, there is provided a sample image acquisition apparatus including a following configuration in a system that can be connected to an HTML page via a network.
Web image search means for specifying a keyword and collecting images in a plurality of HTML pages on which the keyword is posted from the Web.
Feature quantity conversion means for measuring low-dimensional signal values such as color, shape, and pattern from the Web search image collected from the Web image search means and outputting them as image feature quantities.
For the image feature amounts respectively generated from the plurality of Web search images by the feature amount conversion means, the feature space is clustered between image feature amounts that are close to each other from the distribution characteristics in the feature space, and the cluster having the largest distribution number or Sample image specifying means for specifying an image belonging to a cluster with small variance as a sample image related to a specified keyword.

  In addition to the configuration of the sample image acquisition device according to claim 2, the image classification device according to claim 3 extracts an image feature amount corresponding to the sample image specified by the sample image specifying means, and extracts the specified keyword. An image feature amount representing a classification category as a name, and an image feature amount generated by the feature amount conversion unit from the input image whose classification category is unknown, and an image feature amount representing the classification category A classification category determining unit that calculates a similarity and classifies an input image into a predetermined classification destination based on the similarity is provided.

  According to a fourth aspect of the present invention, in the image classification apparatus according to the third aspect, the classification category determination unit determines whether the input image belongs to the classification category based on the similarity, and determines that the input image belongs to the classification category. In this case, the image search keyword adding means for adding the name of the category to the input image as a keyword is provided.

A fifth aspect of the present invention relates to the image classification device according to the third or fourth aspect,
The feature amount conversion means includes:
A feature vector extracting means for extracting a plurality of points having large signal changes such as edges and irregularities from an image as key points, and outputting a feature vector calculated from a color, shape, pattern, etc. near each key point;
Measure the distribution characteristics in the feature space of the feature vector group extracted from a wide variety of different images by the feature vector extraction means, cluster the feature space between feature vectors that are close to each other, and select representative vectors for each cluster CodeBook generation means to output as CodeBook,
An accumulator for storing the CodeBook generated by the CodeBook generating means;
The feature vector extracted by the feature vector extraction means is vector quantized to the nearest representative vector in the CodeBook stored in the storage unit, and at the same time, the appearance frequency of each representative vector is obtained, and the histogram is obtained. It is characterized by comprising vector quantization means for outputting as an image feature quantity of the input image.

A sixth aspect of the present invention relates to the image classification device according to the third or fourth aspect,
The classification category determination means includes
An accumulation unit for storing the feature amount of the sample image specified by the sample image specifying means as an image feature amount of a classification category whose name is a specified keyword;
The difference between the image feature amount of the classification category whose name is the designated keyword and the image feature amount of other various category categories already stored in the storage unit is extracted by machine learning as a criterion for classification determination. A category classifier generating means for outputting a classification judgment criterion as a classifier;
Whether the image feature quantity generated by the feature quantity conversion means from the input image whose classification category is unknown belongs to the classification category whose name is the designated keyword is generated by the category classifier generation means. It is characterized by comprising a category classification means for classifying an input image into a predetermined classification destination when it is determined that the input image belongs.

  According to the first and second aspects of the present invention, it is possible to automatically collect images related to a designated keyword from the Web without requiring manual visual confirmation. Therefore, it is possible to save the labor of manually collecting the sample images necessary for generating the image feature amount, and it is possible to automatically generate the image feature amount related to the designated keyword.

  According to the invention of claim 3, in addition to the preset classification category, the user can specify a keyword and set a new classification category to adaptively classify the images. At this time, since there is no restriction on the keyword to be specified, the user can perform image classification based on a desired keyword. For example, an image related to a keyword desired by the user can be classified from among images possessed by the user.

  According to the fourth aspect of the present invention, an image search keyword can be assigned to an image determined to be related to a keyword designated by the user. For example, an image search keyword can be assigned to an image determined to be related to a keyword designated by the user from among images possessed by the user.

According to the fifth and sixth aspects of the invention, it is possible to extract a more stable feature amount from an image and further improve the accuracy of image classification and image search keyword assignment.
Furthermore, in the keyword assignment for image search through human hands, even when the same image is targeted, the keyword assigned may differ depending on the subjectivity of the person. However, according to the image classification device according to the present invention, it is possible to determine which keyword is appropriate for the image and the similarity of the image feature amount. Since the classification category names determined to be similar based on the objective criteria and given as keywords are assigned as keywords, the keywords for image search are always assigned based on unified criteria without depending on people. be able to.

It is a block diagram which shows the structure of the image classification device of this invention. (A)-(c) is a graph for demonstrating the process which calculates the image feature-value in the feature-value conversion means of the image classification device of this invention. It is a graph for demonstrating "tomato cluster creation" as an example in the sample image specific means of the image classification device of this invention.

An example of an embodiment of an image classification device of the present invention will be described with reference to the drawings.
The image classification apparatus according to the present invention collects and classifies sample images from the Web using the contents of an HTML file as a clue by enabling connection to an HTML page via a network.
A large number of HTML files are stored on a server on the Web, and they are connected via a network. HTML is composed of text and images, and in many cases, keywords in the same HTML contain keywords that accurately represent the contents of the images. The present invention pays attention to this, collects and classifies a plurality of sample images from the Web by specifying a keyword and using a normal Web image search method.

  An image classification apparatus according to the present invention includes a transmission / reception unit for accessing text information and images on the Web, an input unit for inputting a keyword by a user, and a main storage unit for temporarily developing image data being processed. The storage unit stores images and various text information, the central processing unit controls the calculation and various processes, and the display unit displays the processing results.

  That is, as shown in FIG. 1, the image classification apparatus of the present invention is connected to a network 1 for keywords input from an input unit, collects images in an HTML page, and collects images. Alternatively, a feature amount conversion unit 3 that calculates an image feature amount from an input image, a sample image specification unit 4 that specifies a sample image related to a keyword, a classification category determination unit 5 that classifies the input image into a predetermined classification destination, It has an image search keyword assigning means 6 for assigning the name of the classification category to the input image as a search keyword, and an accumulation unit 7 for recording various information. The sample image acquisition device of the present invention used for the image classification device is composed of a Web image search means 2, a feature amount conversion means 3, and a sample image specifying means 4.

  The image classification device includes (1) a function for collecting an image for a keyword (designated keyword) designated by a user and generating an image feature quantity (at the time of feature quantity creation), and (2) an unknown classification category (keyword A function (at the time of classification / search) for assigning an image search keyword to an input image.

  The Web image search means 2 collects a large number of images in an HTML page on which a keyword (specified keyword) specified by the user is posted from the Web, and outputs it to the feature amount conversion means 3. Since the image information in the HTML page is often provided with character information as text information, an image having text information that matches the specified keyword is collected and captured.

The feature amount conversion unit 3 measures low-dimensional signal values such as color, shape, and pattern from the image (collected image) collected by the Web image search unit 2 and the input image that is directly input, and outputs it as an image feature amount.
That is, the feature amount conversion unit 3 includes a feature vector extraction unit 31 that calculates a feature vector from a collected image, a CodeBook generation unit 32 that generates a CodeBook from the feature vector, and a vector that calculates an image feature amount by quantizing the feature vector. A quantization means 33 is provided.

  The feature vector extraction means 31 extracts a plurality of points with large signal changes such as edges and irregularities from the image as key points, and outputs a feature vector calculated from the color, shape, pattern, etc. near each key point. The calculation of the feature vector performed by the feature vector extracting unit 31 is realized by a known technique. Specific implementation methods include, for example, SIFT (Scale-Invariant Feature Transform, Lowe, D. "Distinctive Image Features from. Scale-invariant keypoints," International journal of Computer Vision, Vol. 60, proposed by Lowe et al. No. 2, pp. 91-110, 2004) can be used. In this case, the feature vector is output as a 128-dimensional vector.

By using SIFT, the same key points are extracted and the same feature vectors are extracted even if the images are different in view of rotation, size, etc., if they are the same subject and the same contents. In SIFT, each process of (A) key point detection and (B) feature vector extraction is performed in the following flow.
(A) Key point detection (a) Key point candidate point detection (b) Key point localization (B) Feature vector extraction (c) Orientation calculation (d) Feature quantity extraction

In the detection of key point candidate points in (a), a plurality of points having large signal changes such as edges and irregularities are detected from the image by DoG (Difference-of-Gaussian) processing. Change the scale of the Gaussian function in several stages, create multiple smoothed images convoluted with the Gaussian function and the input image, and key points are the points that become extreme values in the difference image (DoG image) of those smoothed images Detect as candidate points.
In the localization of key points in (b), key points that can be stably extracted from the key point candidate points detected in (a) are narrowed down. That is, a point having a small contrast and a point having a large main curvature are deleted from the key point candidate points as points affected by noise and points not suitable for stable extraction.
In the orientation calculation of (c), in order to extract the same feature vector even if the image is rotated at the same key point, the direction characterizing each key point is determined from the gradient of each point in the smoothed image. calculate. Specifically, a histogram relating to the gradient direction and gradient intensity is measured from a rectangular area around the key point.
First, classification is performed by a class quantized to 36 with respect to the gradient direction.
Next, the gradient strength is added to the classified class, and the direction of the class showing the mode in the histogram is calculated as the orientation.
In the description of the feature quantity in (d), the feature vector extraction target area at each key point is normalized based on the orientation obtained in (c), and the feature vector extraction target around the key point cut out by normalization is extracted. A feature vector is calculated from the region.

Next, generation of CodeBook by the CodeBook generation unit 32 and calculation of image feature amounts by the vector quantization unit 33 will be described with reference to FIG.
The CodeBook generating unit 32 plots the feature vectors (each x point in FIG. 2A) extracted from various different images by the feature vector extracting unit 31 on the feature space, and the feature space of the feature vector group The distribution characteristics above are measured, the feature space is clustered between feature vectors that are close to each other, and the representative vector of each cluster (the point XX in FIG. 2A is the feature vector) is the vector quantization means 33 in the subsequent stage. Is output as CodeBook 71 for

As a specific means for realizing clustering, k-means, which is a known technique, can be used.
Clustering by k-means is performed by the following procedures (1) to (4).
(1) The data is divided into k clusters, which are an arbitrary number.
(2) Calculate the centroid for each cluster.
(3) A cluster that minimizes the distance from the center of gravity is obtained for all data, and each data is assigned to the smallest cluster.
(4) End if there is no change from the previous cluster. If there is a change, return to (2).

  In this case, the number of divided clusters k can be arbitrarily set, and the generated image feature amount is k-dimensional. The CodeBook 71 is generated by the CodeBook generating unit 32 by inputting various different images in advance to the feature vector extracting unit 31 and stored in the storage unit 7.

  The vector quantization means 33 vector-quantizes the feature vector extracted by the feature vector extraction means 31 for the input image to the nearest representative vector (CodeBook) in the CodeBook 71 stored in the storage unit 7. Simultaneously with (FIG. 2B), the appearance frequency of each representative vector is obtained, and the histogram is output as the feature quantity of the input image (FIG. 2C). Therefore, the image feature amount calculated by the vector quantization unit 33 is generated as a k-dimensional histogram.

When collecting images for a keyword (designated keyword) specified by the user, the Web image search means 2 searches for a Web image for the keyword by a search method currently in practical use. For example, a search method using an image search service of Google (trade name) is an extension of a conventional method for searching text information by specifying a keyword. In other words, an image in HTML including the specified keyword is only presented as a search result. Therefore, the text in the HTML file contains many noise-causing components other than the keywords that accurately represent the image contents, so the image contents retrieved by Web image search do not always match the specified keywords. There are also many.
For example, even if the keyword “tomato” is specified, not only the image showing the appearance of “tomato” but also “tomato juice”, “tomato ketchup”, and even an image unrelated to tomato may be searched. is there.

In the image classification device of the present invention, the sample image specifying means 4 for removing a component that becomes noise from each image feature amount output from the vector quantization means 33 is provided.
The sample image specifying unit 4 converts a plurality of images (collected images) collected by the Web image search unit 2 into image feature amounts (k dimensions) by the feature amount conversion unit 3, respectively, and distribution characteristics in the feature space The feature space is clustered with image feature quantities that are close to each other, and an image belonging to the cluster having the largest distribution number or an image belonging to the cluster having a small variance is specified as a sample image related to the designated keyword.

For example, if you enter the keyword "tomato" and you want to get an image of "tomato", the image searched from the web image is the appearance image of tomato, "tomato juice", "tomato ketchup", "tomato""Irrelevantimage". When these images are converted into image feature amounts by the feature amount conversion means 3, they are as shown in FIG.
Then, the feature space is clustered between image feature quantities that are close to each other based on the distribution characteristics in the feature space, and an image belonging to the cluster having the largest number of distributions (X region in the case of FIG. 3) is designated as the keyword “tomato As a sample image related to "." As another embodiment, an image belonging to a cluster with small variance may be specified as a sample image related to the designated keyword “tomato”.

  As a specific means for realizing clustering, x-means, which is a known technique, can be used. Unlike the k-means described above, the x-means need not specify the number k of clusters to be divided, and the cluster number k is automatically determined and clustering is performed.

  The classification category determination unit 5 extracts the image feature amount corresponding to the sample image specified by the sample image specification unit 4 from the image feature amount generated by the feature amount conversion unit 3 and classifies the category of the designated keyword. When an input image whose classification category is unknown is input, the similarity with the image feature generated by the feature conversion unit 3 is calculated and input based on the similarity The image is classified into a predetermined classification destination.

  That is, the classification category determination unit 5 includes a category classifier generation unit 51 for realizing a function (at the time of generating a feature amount) of performing image classification with an image feature amount representing a classification category whose name is a designated keyword, A category classification means 52 for classifying an input image whose classification category is unknown (no keyword is assigned) in order to realize a function for assigning a search keyword (during classification / search); Yes. In addition, the storage unit 7 stores in advance the image feature quantities 72 of other various classification categories, and classifies the feature quantities of the sample images specified by the sample image specifying means 4 as classification categories having the designated keywords as names. Is stored as a classifier 73 that becomes the image feature amount of the image.

The category classifier generating means 51 determines that the image feature quantity of the category classified by the designated keyword is significantly different from the image feature quantities 72 of other various category categories stored in the storage unit 7 in advance. Extraction is performed by machine learning as a criterion for classification determination, and the criterion for classification determination is output as a classifier 73.
As a specific means for realizing machine learning, SVM (Support Vector Machine), which is a known technique, can be used.

SVM is a classification method that determines which of two given classes a given data belongs to, and was proposed in 1995 by Corinna Cortes and V.Vapnik of AT & T. In SVM, criteria for classification are extracted by learning from sample data collected in advance.
For example, in order to classify whether a given image is “a face image or not”, not only a face image but also many images other than a face are collected and used as a sample image for learning. Prior knowledge such as “this image is a face” or “not a face” is given, and a determination criterion for classification is generated based on this prior knowledge.

Specifically, each sample image is converted into an image feature amount, and a boundary (hyperplane) for classifying the face and the non-face from the distribution state in the feature space is obtained and used as a classification reference. Actually, when classifying a newly given image, the face is determined depending on which side of the boundary (hyperplane) the image feature amount extracted from the newly given image is included in the feature space. Whether it is an image or not is determined.
Therefore, in order to actually perform classification by SVM, a procedure of (1) collecting a large number of sample images and (2) generating a criterion (classifier) for classification by learning is required in advance. Become. At this time, each sample image is required to be provided with information for clarifying which of the two classes it belongs. In this example, this condition is satisfied by the sample image specifying means 4 (a sample image to which a keyword is assigned can be obtained).

  The category classification unit 52 determines whether the image feature amount generated by the feature amount conversion unit 3 from an input image whose classification category is unknown belongs to a classification category whose name is a designated keyword. The determination is made using the classifier 73 generated by the generation means 51. If the category classification means 52 determines that the specified keyword belongs to the classification category whose name is the specified keyword, the input image is classified as the image 74 of the storage unit 7 which is a predetermined classification destination, or the image Output to the search keyword assigning means 6.

  When the classification category determination unit 5 determines that the input image belongs to a classification category whose name is the designated keyword, the image search keyword assignment unit 6 uses the name of the classification category as a keyword in the input image. And the image 74 is classified into a predetermined classification destination in the storage unit 7.

Next, a procedure for creating an image feature amount and assigning an image search keyword using the image classification apparatus described above will be described.
“Creating image features for various images”
First, the category classifier generation means 51 performs processing for previously recording the image feature quantity 72 serving as a reference for creating the classifier 73 for the image feature quantity of each image in the storage unit 7. This process is performed by calculating an image feature amount in the feature amount conversion means 3 from arbitrary various images input via the network 1 or directly, and recording it in the storage unit 7 as an image feature amount 72 in advance.

That is, the feature vector extracting unit 31 of the feature amount converting unit 3 inputs a feature vector group extracted from a wide variety of different images to the CodeBook generating unit 32, measures the distribution characteristics in the feature space, and calculates the distance. The feature space is clustered with feature vectors that are close to each other. Then, the representative vector of each cluster is recorded in the storage unit 7 as a CodeBook 71. Then, the vector quantization means 33 performs quantization processing for each CodeBook, creates each image feature amount 72 composed of histograms with 100 to 200 representative vectors, and records it in the storage unit 7.
These image feature quantities (image feature quantities output by the vector quantization means 33) 72 are necessary for creating the classifier 73 by the category classifier generation means 51, and are output by the category classifier generation means 51. The classifier 73 accumulates in the accumulating unit 7 and is used as a criterion for classifying the input image whose classification category is unknown by the category classification unit 52.

“Create image features for images searched by keywords”
The user inputs a keyword from the input unit and creates an image feature amount of an image that matches the keyword. That is, the Web image search unit 2 performs a Web image search based on the keyword input from the input unit, and an image obtained through the network 1 is output to the feature amount conversion unit 3.
The feature vector extraction means 31 converts the input image into a feature vector and outputs it by the method of extracting the feature vector from the detected key point and the detected key point, for example, as in SIFT.

  The vector quantization means 33 performs vector quantization of the input feature vector using the CodeBook 71 stored in the storage unit 7, and quantizes the input feature vector to the nearest representative vector in the CodeBook 71. At the same time, the frequency quantized into each representative vector is obtained, and the histogram is output as an image feature amount.

  The sample image specifying means 4 clusters the input image feature values by image feature values that are close in distance from the distribution characteristics in the feature space, and designates an image that belongs to a cluster with a large number of distributions or a cluster with a small variance. Specified as a sample image related to the keyword.

  In the category classifier generating means 51, both are inputted from the image feature quantity of the classification category whose name is the inputted designated keyword and the image feature quantities 72 of other various classification categories stored in the storage unit 7 in advance. Differences that are significantly different are extracted by machine learning as a criterion for classification determination and output as a classifier 73. The output classifier 73 is stored in the storage unit 7 so that it can be used as a judgment criterion for classification by the category classification means 52 in the subsequent stage.

“Category classification and image search keyword assignment for input images with unknown classification category”
The feature vector extraction means 31 converts an input image whose classification category is unknown into a feature vector and outputs it by a method of extracting a feature vector from the detected key points and the detected key points, for example, as in SIFT. .
The vector quantization means 33 performs vector quantization of the input feature vector using the CodeBook 71 stored in the storage unit 7, and quantizes the input feature vector to the nearest representative vector in the CodeBook 71. At the same time, the frequency quantized into each representative vector is obtained, and the histogram is output as an image feature amount.

The category classification means 52 uses the classifier 73 stored in the storage unit 7 to determine whether or not the newly input image belongs to the category of the classifier 73 and determines that it belongs. In this case, the input image is classified into an image 74 that is a predetermined classification destination.
If it is determined as a result of the category classification means 52 that the input image belongs to the category of the specified classifier 73, the image search keyword assigning means 6 images the name indicated by the classifier 73 in the input image. It is given as a keyword for search. The image file name and the keyword may be linked and recorded in a database or the like without being directly assigned to the image.

A specific application example using the above-described image classification apparatus will be described.
The present invention can be used for a service for classifying and searching a large number of images possessed by a user.
For example, when the user has taken and stored a large number of images of the child in the storage unit 7 (when the image is simply recorded in the storage unit 7), the feature amount conversion means 3 (or direct input) these images from the storage unit 7 As image). If you want to search for “sitting and eating” images from a large number of images of the child, the user enters “sitting and eating” as a keyword in the input section. The above-described processes are performed by collecting images based on the HTML from the HTML, and the image feature amount of the image “sitting and eating” is recorded in the storage unit 7 as the classifier 73.
Next, in the category classification means 52, the image feature amount of each image possessed by the user (stored in the storage unit 7 or directly input) and the “sitting and cooking rice” recorded in the classifier 73. Compared with the image feature value of the “eat” image, if the degree of similarity is high, it is recognized as a “sitting and eating” image, and the result is displayed on the display unit. Efficient images can be extracted efficiently.

If the user wants to classify a large number of children's images, the user designates and inputs a keyword (for example, “sitting and eating”), and the Web image search means 2 uses the keyword. By collecting the images based on the HTML from the HTML, the image feature amount of the image “sitting and eating” is recorded in the classifier 73 by the above-described processes.
Next, the category classification means 52 compares the image feature amount of each image possessed by the user with the image feature amount of the “sitting and eating” image recorded in the classifier 73, and the degree of similarity. If the image is high, the image is recognized as “sitting and eating”, and the keyword “sitting and eating” is assigned to the image by the image retrieval providing unit 6 and the image is stored in the storage unit 7 together with the image. Recorded as 74.

According to the above-described image classification device, by adding a search keyword to an image, it is possible to directly extract the image by inputting the keyword when searching for an image next time.
Conventionally, the classification category and the keyword for image search are limited to those set in advance. However, in the above example, the keyword can be freely set by the user, so that the keyword according to the user's preference can be set. Classification is possible.

  DESCRIPTION OF SYMBOLS 1 ... Network, 2 ... Web image search means, 3 ... Feature-value conversion means, 4 ... Sample image specification means, 5 ... Classification category determination means, 6 ... Image search keyword provision means, 7 ... Accumulation part, 31 ... Feature vector Extraction means, 32 ... CodeBook generation means, 33 ... Vector quantization means, 51 ... Category classifier generation means, 52 ... Category classification means, 71 ... CodeBook, 72 ... Image feature amount, 73 ... Classifier, 74 ... Image.

Claims (6)

In a system that can connect to an HTML page via a network,
Specify keywords and collect images from multiple HTML pages on which the keywords are posted from the Web.
Measure low-dimensional signal values such as color, shape, and pattern from multiple web search images collected from the web, and output them as image features.
For the plurality of image feature amounts, the feature space is clustered between image feature amounts that are close in distance from the distribution characteristics in the feature space, and an image belonging to a cluster with the largest distribution number or a cluster with a small variance is designated as a specified keyword. A sample image acquisition method characterized by specifying as a related sample image. In a system that can connect to an HTML page via a network,
Web image search means for specifying a keyword and collecting images from a plurality of HTML pages on which the keyword is posted from the Web;
Feature quantity conversion means for measuring low-dimensional signal values such as colors, shapes and patterns from the web search images collected from the web image search means and outputting them as image feature quantities;
For the image feature amounts respectively generated from the plurality of Web search images by the feature amount conversion means, the feature spaces are clustered between image feature amounts that are close in distance from the distribution characteristics in the feature space, and the cluster having the largest number of distributions or Sample image specifying means for specifying an image belonging to a cluster with small variance as a sample image related to a specified keyword;
A sample image acquisition apparatus comprising: In a system that can connect to an HTML page via a network,
Web image search means for specifying a keyword and collecting images from a plurality of HTML pages on which the keyword is posted from the Web;
Feature quantity conversion means for measuring low-dimensional signal values such as colors, shapes and patterns from the web search image or input image collected from the web image search means and outputting them as image feature quantities;
For the image feature amounts respectively generated from the plurality of Web search images by the feature amount conversion means, the feature spaces are clustered between image feature amounts that are close in distance from the distribution characteristics in the feature space, and the cluster having the largest number of distributions or Sample image specifying means for specifying an image belonging to a cluster with small variance as a sample image related to a specified keyword;
The image feature quantity corresponding to the sample image specified by the sample image specifying means is extracted and used as an image feature quantity representative of a classification category whose name is the designated keyword, while the classification category is unknown A classification for calculating the similarity between the image feature quantity generated by the feature quantity conversion unit from the image and the image feature quantity representing the classification category, and classifying the input image into a predetermined classification destination based on the similarity degree An image classification apparatus comprising a category determination unit.   Based on the similarity, the classification category determination means determines whether it belongs to the classification category, and if it is determined that the input image belongs to a classification category, the name of the classification category is used as a keyword in the input image. The image classification device according to claim 3, further comprising an image search keyword assigning unit for assigning. The feature amount conversion means includes:
A feature vector extracting means for extracting a plurality of points having large signal changes such as edges and irregularities from an image as key points, and outputting a feature vector calculated from a color, shape, pattern, etc. near each key point;
Measure the distribution characteristics in the feature space of the feature vector group extracted from a wide variety of different images by the feature vector extraction means, cluster the feature space between feature vectors that are close to each other, and select representative vectors for each cluster CodeBook generation means to output as CodeBook,
An accumulator for storing the CodeBook generated by the CodeBook generating means;
The feature vector extracted by the feature vector extraction means is vector quantized to the nearest representative vector in the CodeBook stored in the storage unit, and at the same time, the appearance frequency of each representative vector is obtained, and the histogram is obtained. The image classification apparatus according to claim 3, further comprising a vector quantization unit that outputs an image feature amount of the input image. The classification category determination means includes
An accumulation unit for storing the feature amount of the sample image specified by the sample image specifying means as an image feature amount of a classification category whose name is a specified keyword;
The difference between the image feature amount of the classification category whose name is the designated keyword and the image feature amount of other various category categories already stored in the storage unit is extracted by machine learning as a criterion for classification determination. A category classifier generating means for outputting a classification judgment criterion as a classifier;
Whether the image feature quantity generated by the feature quantity conversion means from the input image whose classification category is unknown belongs to the classification category whose name is the designated keyword is generated by the category classifier generation means. The image classification apparatus according to claim 3 or 4, further comprising: a category classification unit that classifies the input image into a predetermined classification destination when it is determined that the input image belongs.

Images