JP2014178941A - Image collation system - Google Patents

Abstract

When recognizing a general object included in an image, multiple contours are obtained with a small amount of calculation, and feature extraction of the object is performed step by step without requiring an enormous amount of calculation. It is possible to perform stepwise object matching.
As an image matching system, contour information extracting means for extracting one or a plurality of contour lines represented by a set of points as contour information, and multi-resolution contour information expressed by a plurality of resolutions from the contour information are multiplexed. Generated by combining contours that are formed according to the weighting based on the arrangement relationship between predetermined points and adjacent points when different reference ranges are applied to each resolution for each resolution. Multi-resolution contour information generating means for processing, multi-resolution feature extracting means for extracting features included in the multi-resolution contour information, and matching means for performing matching processing using the extracted features are provided.
[Selection] Figure 1

Description

  The present invention relates to a system for recognizing or searching a general object or shape included in image information, and more particularly, a system for processing object or shape information expressed by multi-resolution information at high speed. About.

  In recent years, with the rapid spread of digital video equipment such as digital cameras, general object recognition for recognizing what objects are included in captured images and videos. Expectations are growing. The general object recognition is a technique for recognizing an object included in an unrestricted real-world scene image with a general name (category names). General object recognition is regarded as one of the most difficult tasks in image recognition research. General object recognition technology can be applied to various applications. For example, appropriate classification of image data stored without being classified in a database, search for necessary image data, extraction of a desired scene from a moving image, or cropping of only a desired scene It can be applied to re-editing.

  As a technique for recognizing an object from an image, various recognition techniques such as face recognition and fingerprint recognition have been developed so far. However, these are all technologies for recognizing images taken under specific restrictions, and are limited to specific applications. Development of technology that recognizes general objects has been pointed out that recognition technology specialized for one target has been pointed out as the recognition rate drops as soon as it is used for another purpose. Is expected.

  In order to accurately recognize a general object included in an image, it is necessary to accurately grasp the characteristics of the image to be recognized or the object in the image. In order to grasp the characteristics of the identification target, the image collation system performs detailed analysis of the image. The inventors are working on technical development that can recognize a general object by performing a detailed analysis on the entire image. For this reason, many different approaches need to be considered.

In order to recognize a general object included in an image, it is useful to extract the features of the image to be recognized in stages. That is, first extract the minimum features that can recognize that there is an object, then extract the minimum features that can describe the difference between the object and other objects, and This is a stepwise feature extraction, such as extraction of detailed features.
Without such stepwise feature extraction, we generally face the problem of not knowing where an object is. Next, even if it is possible to specify where an object is, whether the feature identified from the object is the minimum feature appropriate for describing it as a difference between the object and another object, or happens to be It is likely to be impossible to distinguish between the generated characteristics (noise). An example of such stepwise feature identification is shown in FIG.
In the image analysis of the image showing the bat shown in FIG. 12, the object position is identified from the image by the low resolution analysis, and the high resolution analysis is performed on the object position in three stages. In each stage, an idea is presented that matching should be performed based on the identification features identified in the previous stage.

  In this way, if image matching is performed in a manner similar to the object recognition method recognized by humans, many advantages can be obtained from the viewpoints of reduction in calculation amount, matching accuracy, and matching time. On the other hand, an image matching system generally needs to appropriately extract machine-readable image features such as feature points and feature quantities from an image.

In the technique used for image collation, the characteristics of an image to be recognized are converted and acquired into a format that can be effectively used by a machine by various methods.
For example, as a method for extracting features used to describe differences from other objects with respect to an object in an image, there are widely used methods for extracting common terms of geometric features included in an image by machine learning. It has been adopted. This method is disclosed in Patent Document 1 and Patent Document 2, for example. However, most of the feature quantities extracted in this way cannot be extracted by arithmetic processing unless parameters such as threshold values are set in advance by statistical learning or user experience. It is unsuitable for describing objects that can be seen and low-resolution objects. In other words, it has an aspect unsuitable for performing general object recognition and stepwise extraction on an image to be recognized.

Non-Patent Document 1 and Non-Patent Document 2 disclose a method using multi-resolution images as a method of using features of general objects in stages without requiring such statistical learning or user experience. ing. The inventor has also disclosed a technique using a multi-resolution image in Non-Patent Document 3.
According to these methods, an image including an object to be recognized is subjected to filter processing using a plurality of frequencies to create a multi-resolution image (a group of multi-resolution information), and each of the images included in the multi-resolution image. A feature amount in a resolution image, that is, a curvature change point, a luminance gradient, and the like are extracted. As a result, global features extracted even at low resolution and detailed features extracted only at high resolution can be extracted in stages. Since feature-specific information such as the curvature change point and the brightness gradient appears in the object, it is possible to extract a desired target feature in the identification image without requiring statistical learning or user experience.

  By the way, when global features and detailed features are described step by step using a multi-resolution image, it is necessary to extract the features using a filter of a size corresponding to each resolution used for the description. become.

  For example, when a two-dimensional DOG (Difference of Gaussian) filter used in the image processing example described in Non-Patent Document 1 is used, a DOG filter having a radius proportional to the resolution used for description is used. Become. Then, the amount of calculation processing is proportional to the square of the desired resolution, and a large amount of calculation processing is required to describe global features.

In order to reduce such a large amount of arithmetic processing, a multi-resolution contour expression that expresses the contour of an object used in Non-Patent Document 2 in a multi-resolution manner, instead of using a two-dimensional multi-resolution image, is useful. This multi-resolution contour representation has already been proposed in many documents including Non-Patent Document 2 and Non-Patent Document 3.
Multi-resolution contour representation is converted to two-dimensional analysis of the object image itself, and after replacing the object with the contour representation once, it is multiplexed by applying a one-dimensional Gaussian filter in the direction along the contour. A resolution contour (multi-resolution information) is generated. By this processing, it is possible to significantly reduce the amount of calculation processing compared to the case where a two-dimensional DOG filter is applied.

JP 2002-352252 A JP 2005-346391 A

Lowe, D.C. G. , Object recognition from local scale independent features, Proc. of IEEE International Conference on Computer Vision, pp. 1150-1157 FARZIN MOKHTARIAN AND ALAN MACK WORKTH, Scale-based description and Recognition of Planar curves and two-dimensional competition in Matters, and IEEE Transactions on Patters. 8, no. 1 pp. 34-43 YUMA MATSUDA, MASATSUGU OGAWA, MASAFUMI YANO Visible shape representation with geometrically controlled partitions. Biological Cybernetics 106 (4-5), 295-305 (2012)

  In order to recognize a general object included in an image satisfactorily, it is effective to extract the features of the image to be recognized in stages. As an effective technique for this purpose, a technique that describes the feature of an object step by step using multi-resolution information is effective.

  In the case where global features and detailed features are described step by step by performing description using a multi-resolution image, it has already been proposed to perform feature extraction of an identification target using a two-dimensional filter. On the other hand, there is a feature that the calculation processing amount increases in proportion to the square of the desired resolution.

In addition, a method for generating a multi-resolution contour by applying a one-dimensional filter along the contour of an object has already been proposed. This method has a feature that can reduce a large amount of arithmetic processing as compared to performing feature extraction of an identification target using a two-dimensional filter.
However, when trying to describe stepwise features using multi-resolution contours, there are cases where stepwise feature extraction is performed unsuitably for image matching depending on the shape to be identified. The three images shown in FIG. 13 are examples of images in which it is difficult to satisfactorily identify global features when the existing stepwise feature extraction method using a one-dimensional filter is used.
In an image like the shape shown in FIG. 13, when a feature is described by a contour, a design portion characteristic as the image is identified as a global (low resolution) feature in the middle of the contour. More specifically, the image processing system is capable of forming a shape such as a cut (and a cut shape derived from the shape) drawn so as to cross over an object to be described as high resolution in two dimensions. The shape like the diagonal line is described in the same row as the outer shell of the object.

In light of the above, improvements can be found in the stepwise feature extraction method that describes features with contours.
In short, when extracting multiple contours, it is necessary to accurately collate the shape implemented by the system in the same way as other shapes in the description of the stepwise features different from the features included in the actual object. An image processing technique for acquiring multiple contours with a low amount of computation is desired.

  An object of the present invention is to provide an image processing system, a method, and a program for obtaining a multi-resolution contour used for stepwise feature extraction of an object without requiring a large amount of calculation.

  It is another object of the present invention to provide an image processing system, method and program for extracting multi-resolution contour features used for stepwise feature extraction of an object without requiring a large amount of calculation.

  Another object of the present invention is to perform image matching that captures stepwise object features using multi-resolution contours without requiring a large amount of calculation when recognizing a general object included in an image. It is to provide an image matching system.

  An image matching system for matching an object or a shape included in an image according to the present invention includes contour information indicating one or a plurality of contour lines represented by a set of points from the object or the shape included in the image or the image. Contour information extracting means for extracting the multi-resolution contour information expressed by a plurality of resolutions from the contour information, and a predetermined point when a different reference range is applied to each resolution for each resolution to be multiplexed Multi-resolution contour information generating means for forming a contour in accordance with weighting based on the arrangement relationship between and adjacent points, and combining the formed individual contours, and an object included in the multi-resolution contour information or Multi-resolution feature extraction means for extracting shape features, and object or shape matching processing using the features extracted from the multi-resolution outline information And collating means for performing the above.

  An image processing system for generating multi-resolution contour information used for collation processing according to the present invention uses, as contour information, one or more contour lines represented by a set of points from the image or an object or shape included in the image. Contour information extracting means for extracting, and a predetermined point when the reference range different for each resolution for each resolution to be multiplexed is applied to the multi-resolution contour information expressed by a plurality of resolutions from the contour information; And multi-resolution contour information generating means for generating a contour according to weighting based on an arrangement relationship with an adjacent point, and generating and combining the formed individual contours.

  An image processing system for extracting a multi-resolution contour feature used for collation processing according to the present invention uses, as contour information, one or more contour lines represented by a set of points from the image or an object or shape included in the image. Contour information extracting means for extracting, and a predetermined point when the reference range different for each resolution for each resolution to be multiplexed is applied to the multi-resolution contour information expressed by a plurality of resolutions from the contour information; Multi-resolution contour information generating means for generating a contour by weighting based on an arrangement relationship with an adjacent point and combining the formed individual contours, and an object or shape included in the multi-resolution contour information And multi-resolution feature extraction means for extracting the features of the above.

  According to the present invention, it is possible to provide an image processing system, a method, and a program for obtaining a multi-resolution contour used for stepwise feature extraction of an object without requiring a large amount of calculation. Further, according to the present invention, it is possible to provide an image processing system, a method, and a program for extracting features of a multi-resolution contour used for stepwise feature extraction of an object without requiring a large amount of calculation. Moreover, when recognizing a general object included in an image, it is possible to provide an image matching system that operates well without requiring a huge amount of calculation.

It is a block diagram of the image collation system in embodiment. It is explanatory drawing which shows an example of an input image. It is explanatory drawing which shows an example of an outline vector. It is explanatory drawing which shows an example of an adjacent point. It is explanatory drawing which shows an example of a point sequence. It is explanatory drawing which shows an example of the 1st conditions which implement weighting with respect to a point sequence. It is explanatory drawing which shows an example of the 2nd conditions which implement weighting with respect to a point sequence. It is explanatory drawing which shows an example of the 3rd conditions which implement weighting with respect to a point sequence. It is explanatory drawing which shows an example of the 4th condition which implements weighting with respect to a point sequence. It is a flowchart which shows an example of operation | movement of the image collation system in embodiment. It is explanatory drawing which showed the image processing flow image by an image collation system. It is explanatory drawing which showed the usage example of the stepwise characteristic. It is explanatory drawing which shows an example of the image in which a stepwise feature extraction is difficult when a multi-resolution outline is used.

  In order for an information processing apparatus to recognize a general object included in an image with a low amount of computation, an appropriate technique for extracting object features in stages is desired. For this reason, it is difficult to extract spatial resolution features accurately and stepwise for some images using the method of extracting multi-resolution contours using a one-dimensional filter proposed as an effective method. .

In response to these problems, the inventor proposes a system for performing image processing as follows.
The processing operations included in the system can be shown in three steps.
In the first step, sample data representing a contour feature of an image or an object shape included in the image as a set of two-dimensional points is extracted (sampled). Preferably, a set of points is sampled as a point sequence (contour vector).
In the second step, the set of points (contour vector) extracted in the first step is weighted based on the arrangement relationship of each point in consideration of the spatial resolution (two-dimensional resolution) for each resolution to be multiplexed. A filtering process for generating a contour for each resolution is performed by selecting and connecting a part of the contour, and a multi-resolution contour is generated by collecting the proliferated contours. The outline of each resolution is extracted based on the positional relationship between an arbitrary point included in the set range of points determined according to each resolution and a point adjacent to the arbitrary point.
Preferably, each resolution contour is resampled as a contour vector to generate a multi-resolution contour vector.

  In the third step, features are extracted from the contours (contour vectors) of the respective resolutions generated in the second step. This extracted feature is a stepwise feature of the identification target included in the original image sampled in the first step.

  By performing the above three steps, multiple contours obtained by extracting two-dimensional (spatial) resolution features in stages can be obtained with a low amount of computation, even for a part of the illustrated images. It becomes possible. By using the multi-resolution contour extracted in this way, image collation processing using the multi-resolution contour for various images can be realized with a low calculation amount and high accuracy.

  Hereinafter, an example of an image matching system that performs image matching after performing the above-described steps and generating multiple contours will be described using embodiments.

  An embodiment of the present invention will be described with reference to FIGS.

[Embodiment]
By adopting this embodiment, it is possible to obtain a multi-contour that makes it possible to extract the spatial features of the identification target step by step without requiring a large amount of calculation, and to extract good features. It is possible to provide a system that performs good image matching by executing processing and image matching processing.

[Description of configuration]
FIG. 1 is a block diagram illustrating a configuration of an image matching system according to an embodiment. In addition, about the structure with little connection with this invention, description is simplified or abbreviate | omitted.
The image matching system includes a control unit 10 that manages the entire information processing, a memory 20, an image information acquisition unit 101 that acquires image data to be processed, and contour vector extraction that extracts contour vector information from the acquired image information. Means 102, multi-resolution contour vector generating means 103 for extracting multi-resolution contour information vector from the extracted contour, multi-resolution feature extracting means 104 for extracting feature quantity from each generated resolution contour vector, extracted multi-resolution Object collation means 105 for comparing features and performing object collation processing, object collation result output means 106 for outputting collation results, image information storage unit 201 for storing acquired image data, etc., extracted multi-resolution The multi-resolution feature information storage unit 202 stores features. The contour vector extracting unit 102, the multi-resolution contour vector generating unit 103, and the multi-resolution feature extracting unit 104 work together to function as an image feature extracting unit.

  The control unit 10 manages the overall operation of information processing related to image processing.

  In the image information acquisition unit 101, image data designated by the user such as a moving image or a photograph is taken into the system and stored in the image information storage unit 201. As the capturing method, the image information specified by the user may be acquired as it is, or may be acquired after performing preprocessing such as filtering processing in order to assist the following processing. An example of the image information acquired by the image information acquisition unit 101 is illustrated in FIG. Hereinafter, the processing operation for extracting the feature points from the image information will be described using the image shown in FIG. This image is difficult for a machine to perform stepwise feature extraction as a multi-resolution contour.

The contour vector extraction unit 102 reads the image information acquired by the image information acquisition unit 101 from the image information storage unit 201, and converts only the contour information included in the image information into a point sequence having two-dimensional coordinates ( (Contour vector). An example of the contour information extracted by the contour vector extraction means 102 is shown in FIG.
The contour vector extraction unit 102 first extracts a pixel corresponding to the contour from the image information acquired by the image information acquisition unit 101 or extracts a pixel corresponding to the object.

When extracting the pixel corresponding to the contour, for example, a place where the hue, saturation, brightness, etc. change rapidly (that is, a pixel in the image) may be extracted using a Laplacian / Gaussian filter or the like. Each extracted location is expressed as a set of points using an orthogonal coordinate system (x, y) or the like. Naturally, the method of contour extraction is not limited to the above.
In addition, when extracting pixels corresponding to an object, all those whose hue, saturation, brightness, etc. are included in a predetermined range are extracted. At this time, the image information can be converted into black and white and then binarized, or can be extracted using a clustering method.

  Contour tracking is performed on the pixels corresponding to the contour or the object extracted by such a method to extract the ordered contours. Any method can be used for the contour tracking here as long as the order of the pixels can be added.

  The interval between the pixels corresponding to the extracted contour is equal to or smaller than the size of the smallest reference range (that is, the reference range corresponding to the highest resolution) among the reference ranges referenced in the following multi-resolution contour vector generation means 103. It is desirable to ensure that there is. Further, it may be selected from a range satisfying ½ times or more of the pixel interval of the original image. As an example of specific intervals, a 1/10 interval of the reference range corresponding to the highest resolution, or a 1/20 interval in detail, may be adopted based on the above range. Here, it is possible to shorten the following calculation time by widening the pixel interval (that is, erasing pixels less than a certain interval). In consideration of this, it is also possible to perform processing for automatically adjusting the pixel interval extracted as a set of points. This process for adjusting the pixel spacing employs a wider interval calculated backward from factors such as the resolution of the original image (pixel spacing), the resolution included in the multiple contours to be generated later (reference range to be used), and the accuracy required by the user. The system should be constructed to do so. A processing unit may be provided that accepts a desired value by presenting the user with an estimate of accuracy, time required for the entire processing, the amount of consumed resources, and the like, and reversely calculates the interval from the value. However, this processing may degrade the accuracy of the following processing, so it is desirable to use different pixel intervals depending on the nature of the object to be recognized.

The multi-resolution contour vector generation unit 103 considers the spatial resolution (two-dimensional resolution) for a plurality of ordered resolutions (multiplexed resolutions) for the point sequence (contour vector) extracted by the contour vector extraction unit 102. A process of repeatedly selecting an appropriate point as a part of the contour to be generated while referring to the positional relationship of each point within the reference range of each resolution having a predetermined size and connecting the points By performing the re-contouring process for each resolution by forming a contour composed of points from which points unnecessary for the resolution obtained from the set of points are thinned out, the re-contoured multi-resolution contour vector Are combined to generate a multi-resolution contour vector. In this processing, the multi-resolution contour vector generation means 103 serves as a filter for each resolution, and also serves to integrate the contours of the respective resolutions increased by passing through the respective filters.
The outline of each resolution is extracted based on the positional relationship between an arbitrary point included in the two-dimensional reference range of the point determined according to each resolution and a point adjacent to the arbitrary point.
As described above, the multi-resolution contour vector generating unit 103 selects an appropriate contour element from a reference range having a size set in advance for each resolution in accordance with weighting based on the arrangement relationship; The control unit is operated as a contour reproducing unit that forms a contour corresponding to each resolution by joining selected contour elements. By this processing operation, the contour included in the multi-resolution contour information is reproduced from the set of points for each resolution.

  The contour element selection means includes contour element point sequence forming means for forming a plurality of combinations of point sequences consisting of a series of three points centered on a predetermined point as contour element candidates within a reference range centered on the predetermined point. It is desirable to operate as point sequence selection means for selecting an appropriate contour element from the formed point sequence group based on the adaptability of the arrangement relationship. By performing the processing operation in this way, a re-contouring process considering the spatial resolution can be performed.

  The re-contouring process in consideration of the spatial resolution is performed, for example, in the following series of four steps. Note that the re-contouring process for each resolution, the processing operation of each step, and the processing operation in each step may be performed simultaneously and in parallel as appropriate, and the order may be changed.

In the first step, first, the size of the adjacent range corresponding to the desired resolution (resolution included in the multi-resolution contour vector) is acquired. The adjacent range is a two-dimensional reference range that defines a point adjacent to the predetermined point as an adjacent point with respect to the predetermined point of interest.
Next, an arbitrary point constituting the contour vector is selected as a point of interest, and a point included in the adjacent range is extracted as an adjacent point with respect to the point of interest. For example, assuming a circle centered on the point of interest and having a radius defined by the resolution as a radius, a point (group) included in this circle is associated with the point of interest as an adjacent point. This process may be performed using all points included in the contour vector as the target point, but the connection point for the target point selected in Step 3 below is reduced by proceeding with the process as the next target point. It is also possible to achieve a calculation amount.
An example of adjacent points extracted with respect to the point of interest by this method is shown in FIG. As shown in the enlarged portion of FIG. 4, points (c10, c11, c12, c18, c19, c20, c22, c23) included in the circle (adjacent range) centered on the point of interest c21 are adjacent points. And is associated with the point of interest c21.
The adjacent point extraction process described above is performed on the points forming the contour vector at all desired resolutions.

In the second step, a point sequence consisting of three points centered on the point of interest is generated for each desired resolution, and all combinations are generated for each point of interest using the adjacent points extracted in the first step. Then, weighting is performed on the point sequence composed of the three points. This weighting result for the sequence of three points varies depending on the size of the adjacent range. That is, through the setting of the size of the adjacent range, a point sequence composed of three points adapted to the resolution being processed obtains a high weight. Here, the point sequence consisting of three points having high weight is adopted as a part of the contour in the subsequent contouring process.
An example of a sequence of three points is shown in FIG. This sequence of three points is composed of a point of interest c21, an adjacent point c10, and an adjacent point c23. A point sequence composed of many three points is generated by such a set of the point of interest and adjacent points.
It is desirable that the weighting of the three points generated for each point of interest is performed in consideration of the following four conditions.
The first condition is a condition in which the weight is increased as the distance between the point of interest and the adjacent point is larger (illustrated in FIG. 6).
The second condition is a condition (illustrated in FIG. 7) in which the weight is increased as the inner angle generated by the three point sequence is larger.
The third condition is a condition that the order of the point sequence of the three points is not inconsistent with the order of the point sequence in the original contour vector (illustrated in FIG. 8). If there is a contradiction in order, it is desirable to set the weight to zero or a small value.
The fourth condition is a condition that the weight is increased as the variation in “azimuth” of each point included in the adjacent range defined in the first step is smaller (illustrated in FIG. 9). When a set of points having no vector quantity is used (when a point sequence is not used), it is also possible to use the continuity of an nth-order curve or an arc formed by the set of points.
In the example in which the variation in orientation shown in FIG. 9 is small, a linear curve is formed and there is no variation in orientation. On the other hand, in the example in which the azimuth variation is large, the azimuth included in the adjacent range is divided into three rows, and the collection of individual azimuths is also bent. Thus, the weight determined from the variation in the azimuth in the adjacent range is used as the weight of the point sequence consisting of three points.
If each point is not collected as a vector in advance, if the points with adjacent numbers on the contour are computerized, the same condition can be used even if an orientation is given between the points when performing this processing. Can do.
The conditions for weighting the point sequence are not limited to these four conditions, and many conditions can be combined as long as the conditions satisfy the conditions of adjacent points to be selected.
Generation of a point sequence group consisting of three points based on the above adjacent range and weighting processing for each point sequence are performed for all the points of the contour vector for each resolution to be multiplexed.
In order to reduce the amount of calculation, the first step and the second step may be performed as the next point of interest for the connection point to the point of interest selected in step 3.

In the third step, a contour (network) is generated by combining the individually weighted three point sequences generated in the second step. In this generation process, the generated point sequence of three points is regarded as a “link”, and both end points of the point sequence are regarded as connection points, that is, “connection nodes”. ”In order based on“ ”to form a network. Since each link in the network is weighted in the second step, contour element candidates having a large weight can be extracted continuously. For example, in the point sequence shown in FIG. 5, c10-c21-c23 is a “link”, and c10 and c23 are “connection nodes”. This “link” is connected to another “link” having c10 or c23 having a large weight as a “connection node”.
If the amount of computation is reduced, the “connection node” becomes the next point of interest, and the contour (network) is completed while the contour elements (links) are connected in order according to the vector direction.
The processing of the final point may be performed when a certain amount of links are connected from the start point (start point) and then when the start point appears again in the adjacent range. If there is a link that can be continued, the link may be given priority. Moreover, the range used for terminating the network may be prepared in advance as a range smaller than the adjacent range separately from the adjacent range. Further, the final connection point candidate may be stored in advance and used in the first step performed on the start point.
The network generated in this step is not limited to one network, and a plurality of networks having different starting points may be generated as contour candidates. A contour to be finally adopted from these contour candidates may be selected according to the weight.

In the third step, contours having large weights are continuously extracted by sequentially selecting links having large weights. Hereinafter, an extraction processing method with higher accuracy will be described.
The method of extracting the contour for each resolution is, for example, first extracting all the contours (a series of nodes) included in the network candidate, and then summing or summing the weights of the above point sequences has the largest value. The method of selecting is conceivable.
For example, the start of this calculation may be performed by selecting a point sequence having a large weight as the start position of one row and starting from that point sequence, or may be performed simultaneously using a plurality of point sequences as start positions. On the other hand, it is desirable to use a point sequence that falls outside the contour as the starting position. Whether it is outside or inside the contour may be calculated by adding the outside angle formed by the point sequence to the discrimination condition. Further, after performing each calculation, the validity as the start position may be determined by verifying whether it is a link or a link according to the coordinate relationship with other links. In addition, when performing computations from a plurality of positions simultaneously in parallel, the probability of being outside may be obtained by comparing networks obtained by individual computations. These may be appropriately reflected in the weighting.
For example, when the closed contour is extracted, the calculation is terminated when the extracted contour circulates. When the open contour is extracted, the calculation is terminated when the extracted contour reaches both ends. It should be noted that when the closed contour is extracted, it is desirable to delete a portion protruding from the closed contour (a node and a link existing before the node that closed the network) to improve the accuracy of the contour with respect to the resolution.
At this time, if all the links are calculated in order, the amount of calculation becomes enormous. Therefore, a threshold for the weight of each link is set in advance, and a link below a predetermined threshold is not calculated. It is desirable to save calculation amount.
Furthermore, considering that the amount of calculation becomes enormous as the network size increases, the integrated value of the weight of the point sequence included in the contour (a series of nodes) in the middle of the calculation falls below a predetermined threshold value. It is desirable to save the calculation amount by not calculating.
In addition, the probabilities of contour candidates may be compared among candidates using other methods, and a contour with high accuracy may be selected.
Further, in the method for increasing the accuracy of the contour, the point sequence consisting of the above three points may be changed to a point sequence consisting of five points. Moreover, after obtaining the point sequence consisting of the above three points, the number of points may be increased to five. Using the intermediate point (intermediate node) between the point of interest (target node) and the connection point (connection node) increased here, the vectors from the intermediate point to the connection point are overlapped and connected between the links. As a result, the accuracy of the contour is improved.

Using the above three steps, the multi-resolution contour vector generation unit 103 extracts an appropriate contour from the contour vectors extracted by the contour vector extraction unit 102 according to each resolution to be multiplexed, and multiplexes by combining the contour groups. A resolution contour vector is generated.
In addition, even if it does not depend on the method by the above three steps, a part of the contour is appropriately selected and connected by weighting using an appropriate condition based on the arrangement relation of each point in consideration of the spatial resolution (two-dimensional resolution). As long as it is a method for re-contouring, the multi-resolution contour may be generated using any method.

特徴量としてユークリッド曲率情報だけでは十分でない場合は、輪郭点の位置関係情報や、法線の方位角や、孤長などを特徴量として加えることを行っても構わない。 The multi-resolution feature extraction unit 104 performs a feature amount extraction process on each resolution contour generated by the multi-resolution contour vector generation unit 103. The feature amount used here may be any amount as long as it is a feature of the contour, but it is desirable that the feature amount include an amount capable of expressing a geometric feature unique to the contour. For example, the value of the Euclidean curvature shown in the following formula (1) or the position of the point (inflection point) at which the Euclidean curvature becomes zero is used. The Euclidean curvature is an amount represented by K (t) in the following equation (1), and the contour coordinate t that is taken around the contour starting from an arbitrary point on the contour, and the contour is an orthogonal coordinate system ( It is defined by the first and second derivative values of t of x, y when expressed in (x, y coordinate system).

If Euclidean curvature information alone is not sufficient as a feature quantity, contour point positional relationship information, normal azimuth angle, or arc length may be added as a feature quantity.

  In addition, it is not always necessary to extract features for all contours at once for contours of all resolutions of multiple contours, and the features of multiple contours used for matching processing are used in parallel based on object matching processing and the amount of computation. It is good also as collecting sequentially.

The object matching unit 105 compares the features extracted by the multi-resolution feature extraction unit 104 with respect to two or more objects to be compared, and performs object matching. The comparison target (comparison target) can be compared with a specific image specified by the user, or the type of identification target can be recognized by referring to a general image recognition database. The collation target is not particularly limited.
As a result, the stepwise collation processing as shown in FIG. 12 can be performed while maintaining accuracy. In addition, since a multi-resolution contour with high accuracy can be generated with a low calculation processing amount, it is possible to perform collation with high accuracy including feature extraction.

The object collation result output unit 106 outputs the collation result performed by the object collation unit 105 to a recording device or a monitor.
The collation result can display the classification to which the identification target belongs using a classification prepared in advance, embed it in an image by tagging it, or link it between similar images.

[Description of operation]
Next, an operation example of the collation system of the embodiment will be described. FIG. 10 is a flowchart showing an example of the operation of the present embodiment.

  First, the image information acquisition unit 101 acquires target image data designated by the user and records it in the image information storage unit 201 (S1001). The acquisition of image information is not limited to the one specified by the user, and the system may acquire it automatically or semi-automatically.

  Next, the contour vector information extraction unit 102 extracts only the contour vector information represented by the point sequence from the image information acquired by the image information acquisition unit 101 (S1002).

  Next, the multi-resolution contour vector generation unit 103 performs a re-contouring process in consideration of the spatial resolution (two-dimensional resolution) for each resolution to be multiplexed with respect to the contour vector extracted by the contour vector extraction unit 102, Multi-resolution outline vector information is generated (S1003).

  Next, the multi-resolution feature extraction unit 104 performs feature extraction processing on each resolution contour generated by the multi-resolution contour vector generation unit 103 (S1004).

  Finally, the object matching unit 105 compares the features extracted by the multi-resolution feature extraction unit 104 with respect to two or more objects to be compared, and performs object matching (S1005). The collation result may be output as appropriate by the object collation result output means 106.

  By operating the image matching system in this way, it is possible to perform image matching processing with a low calculation amount with higher accuracy from the image.

  Note that each unit of the image collation system may be realized using a combination of hardware and software. In a form realized by hardware, all or a part of each part may be constructed on an LSI. In a form in which hardware and software are combined, an image processing program is developed in the RAM, and hardware such as a control unit (CPU) is operated based on the program, thereby causing each unit to function as various means. The program may be recorded in a fixed manner on a recording medium and distributed. The program recorded on the recording medium is read into a memory via a wired, wireless, or recording medium itself, and operates a control unit or the like. Examples of the recording medium include an optical disk, a magnetic disk, a semiconductor memory device, and a hard disk.

  The image collation system can be constructed on a computer or server alone. Moreover, you may construct | assemble as a cloud system or a mobile terminal. Also, an image collation system that performs distributed information processing associated with collation on the terminal and the cloud may be constructed by combining the cloud system and the information processing terminal via a network. For example, the terminal side may be responsible for obtaining the identification target image and outputting the collation result, and performing other processing on the cloud. Further, a system or a program may be constructed so as to perform contour extraction and multiple contour information generation on the terminal side. Further, the terminal may generate multiple contour information and store it in association with the photograph together with the position information. Also, the terminal may extract the features of the multiple contour information and store them in association with the photograph. As described above, since the image has multiple contours and multiple contour features, it is possible to easily classify the image and specify the subject afterwards. Moreover, various selectivity can be secured for the feature extraction process and the collation process.

  To describe the above embodiment in another expression, an information processing system that operates as an image matching system is based on an image matching program, such as image information acquisition means, image feature extraction means, object matching means, object matching result output means, etc. It can be realized by operating as. The image feature extraction means can be constructed by contour information extraction means, multi-resolution contour generation means, and multi-resolution feature extraction means.

  As described above, an image processing system to which the present invention is applied can generate multiple contours with high accuracy with a low amount of computation in order to extract image features used for collation processing step by step. As a result, the image matching system can also obtain a good matching result with a low amount of computation.

  That is, according to the present invention, it is possible to provide an image processing system, a method, and a program for obtaining a multi-resolution contour used for stepwise feature extraction of an object without requiring a large amount of calculation. Moreover, when recognizing a general object included in an image, it is possible to provide an image matching system that operates well without requiring a huge amount of calculation.

  The specific configuration of the present invention is not limited to the above-described embodiment, and changes in a range that does not depart from the gist of the present invention, such as separation / merging of block configurations and replacement of procedures, are also included in the present invention. .

In addition, a part or all of the above-described embodiments can be described as follows. Note that the following supplementary notes do not limit the present invention.
[Appendix 1]
An image matching system for matching an object or shape contained in an image,
Contour information extracting means for extracting one or a plurality of contour lines represented by a set of points as contour information from the image or the object or shape included in the image;
Based on the arrangement relationship between predetermined points and adjacent points when multi-resolution outline information expressed by a plurality of resolutions from the outline information is applied to different reference ranges for each resolution for each resolution to be multiplexed. Multi-resolution contour information generating means for generating a contour by forming the contour according to the weighting, and combining the formed individual contours;
Multi-resolution feature extracting means for extracting features of an object or shape included in the multi-resolution outline information;
Collating means for performing object or shape matching processing using features extracted from the multi-resolution contour information;
An image matching system comprising:

[Appendix 2]
The contour information extracting means extracts a set of points representing individual contour lines as the contour information by a point sequence,
The multi-resolution feature extraction unit uses the feature quantity to be extracted as an extraction requirement for each resolution to be applied, and uses the sequence of extraction of point sequences in accordance with other conditions.

[Appendix 3]
The multi-resolution contour information generating means includes:
A contour element selecting means for selecting an appropriate contour element from a reference range having a size set in advance for each resolution according to a weight based on the arrangement relationship;
Contour reproducing means for forming a contour corresponding to each resolution by joining the selected contour elements;
With
The image collating system according to the above-mentioned supplementary note, wherein the contour included in the multi-resolution contour information is reproduced for each resolution.

[Appendix 4]
The contour element selection means includes
Contour element point sequence forming means for forming a plurality of combinations of point sequences consisting of a series of three points centered on the predetermined point as contour element candidates within a reference range corresponding to each resolution centered on the predetermined point;
Point sequence selection means for selecting an appropriate contour element from the formed point sequence group based on the adaptability of the arrangement relationship;
The image collating system according to the above supplementary note, comprising:

[Appendix 5]
The point sequence selection means includes:
As the adaptability of the arrangement relationship,
The distance formed by the point sequence is large;
The angle formed by the point sequence is large;
The order of the point sequences formed by the point sequence forming means does not contradict the order of the point sequences when the point sequences are extracted by the contour information extracting unit;
The dispersion value of the orientation of the point sequence included in the adjacent range is small;
The image collating system according to the above-mentioned supplementary note, wherein the point sequence selection process is performed by considering any one or a combination or all of the four items.

[Appendix 6]
The contour information extracting means is configured such that an interval between points forming a contour line to be extracted as contour information is equal to or smaller than the size of a reference range used for generating the highest resolution recontour used in the multi-resolution contour information generating means. The image collating system according to the above-mentioned supplementary note, wherein an interval adopted from a range satisfying 1/2 or more of an image pixel interval is used.

[Appendix 7]
Storage means for storing image information to be processed and features of an object extracted from the image information;
The contour line of the object described in the image obtained by image analysis of the image information is converted into data as a set of points as contour information,
Identifying an arbitrary point and a plurality of points included in a two-dimensional range having different sizes for each resolution centered on the arbitrary point from the set of points included in the contour information, In accordance with weighting using an appropriate condition for linking each point based on an arrangement relationship between an arbitrary point and a plurality of points included in the two-dimensional range, a partial network to be a part of the contour is selected and the selected part Re-contouring the entire network that joins the network as a contour for each resolution,
Generate and process multi-resolution contour information by combining the contours proliferated corresponding to individual resolutions,
Image feature extraction means for extracting and recording features of an object or shape included in the generated multi-resolution contour information;
An image processing system comprising:

[Appendix 8]
The image feature extraction means includes
As the contour information, a set of points representing individual contour lines is extracted with a point sequence,
The image processing system according to the above supplementary note, wherein the feature quantity to be extracted is used as an extraction requirement for each resolution to be applied, and the sequence of extraction of point sequences is used in accordance with other conditions.

[Appendix 9]
The image feature extraction means includes
A contour element selecting means for selecting an appropriate contour element from within a two-dimensional range having a size set in advance for each resolution according to a weight based on the arrangement relationship;
Contour reproducing means for forming a contour corresponding to each resolution by joining the selected contour elements;
With
The image processing system according to the above remark, wherein the contour included in the multi-resolution contour information is reproduced for each resolution.

[Appendix 10]
The contour element selection means includes
Contour element point sequence forming means for forming a plurality of combinations of point sequences consisting of a series of three points centered on the predetermined point as contour element candidates within a two-dimensional range corresponding to individual resolutions centered on the predetermined point; ,
Point sequence selection means for selecting an appropriate contour element from the formed point sequence group based on the adaptability of the arrangement relationship;
The image processing system according to the above supplementary note, comprising:

[Appendix 11]
The point sequence selection means includes:
As the adaptability of the arrangement relationship,
The distance formed by the point sequence is large;
The angle formed by the point sequence is large;
The order of the point sequences formed by the point sequence forming means does not contradict the order of the point sequences when the point sequences are extracted by the contour information extracting unit;
The dispersion value of the orientation of the point sequence included in the adjacent range is small;
The image processing system according to the above-described supplementary note, wherein the point sequence selection process is performed by considering any one, a combination, or all of the four items.

[Appendix 12]
The image feature extraction unit is configured such that an interval between points forming a contour line to be extracted as contour information is equal to or less than a reference range size used for generating the highest resolution re-contour used in the multi-resolution contour information generation unit, and The image processing system according to the above remark, wherein an interval adopted from a range satisfying ½ times or more of the pixel interval of the image is used.

[Appendix 13]
An image matching method by an information processing system for matching an object or a shape included in an image,
From the image or the object or shape included in the image, one or more contour lines represented by a set of points are extracted and processed as contour information,
Based on the arrangement relationship between predetermined points and adjacent points when multi-resolution outline information expressed by a plurality of resolutions from the outline information is applied to different reference ranges for each resolution for each resolution to be multiplexed. A contour is formed according to the weighting, and the individual contours thus formed are combined and generated,
An object or shape feature included in the multi-resolution contour information is extracted;
An image collating method characterized in that an object or shape is collated using a feature extracted from the multi-resolution outline information.

[Appendix 14]
An image processing method for obtaining multiple contours used when performing image matching for matching an object or shape contained in an image,
From the image or the object or shape included in the image, one or more contour lines represented by a set of points are extracted and processed as contour information,
Based on the arrangement relationship between predetermined points and adjacent points when multi-resolution outline information expressed by a plurality of resolutions from the outline information is applied to different reference ranges for each resolution for each resolution to be multiplexed. An image processing method characterized in that a contour is formed according to the weighting, and the generation processing is performed by combining the formed individual contours.

[Appendix 15]
An image processing method for obtaining a multiple contour feature used when performing image matching for matching an object or a shape included in an image,
The contour line of the object described in the image obtained by image analysis of the recorded image information read from the storage unit is converted into data as a set of points as contour information,
Identifying an arbitrary point and a plurality of points included in a two-dimensional range having different sizes for each resolution centered on the arbitrary point from the set of points included in the contour information, In accordance with weighting using an appropriate condition for linking each point based on an arrangement relationship between an arbitrary point and a plurality of points included in the two-dimensional range, a partial network to be a part of the contour is selected and the selected part Re-contouring the entire network that joins the network as a contour for each resolution,
Generate and process multi-resolution contour information by combining the contours proliferated corresponding to individual resolutions,
An image processing method, wherein an object or shape feature included in the generated multi-resolution contour information is extracted and recorded in a storage unit.

[Appendix 16]
An information processing system that performs image matching for matching an object or shape included in an image,
Contour information extracting means for extracting one or a plurality of contour lines represented by a set of points as contour information from the image or the object or shape included in the image;
Based on the arrangement relationship between predetermined points and adjacent points when multi-resolution outline information expressed by a plurality of resolutions from the outline information is applied to different reference ranges for each resolution for each resolution to be multiplexed. Multi-resolution contour information generating means for generating a contour by forming the contour according to the weighting, and combining the formed individual contours;
Multi-resolution feature extracting means for extracting features of an object or shape included in the multi-resolution outline information;
Collating means for performing collation processing of an object or a shape using features extracted from the multi-resolution contour information;
A program characterized by operating as

[Appendix 17]
An information processing system that generates multiple contours used for image matching,
Contour information extracting means for extracting one or a plurality of contour lines represented by a set of points as contour information from the image or the object or shape included in the image;
Based on the arrangement relationship between predetermined points and adjacent points when multi-resolution outline information expressed by a plurality of resolutions from the outline information is applied to different reference ranges for each resolution for each resolution to be multiplexed. Multi-resolution contour information generating means for generating a contour by forming the contour according to the weighting and combining the formed individual contours;
A program characterized by operating as

[Appendix 18]
An information processing system that extracts multiple contour features used for image matching,
The contour line of the object described in the image obtained by image analysis of the image information read from the storage unit is converted into data as a set of points as contour information,
Identifying an arbitrary point and a plurality of points included in a two-dimensional range having different sizes for each resolution centered on the arbitrary point from the set of points included in the contour information, In accordance with weighting using an appropriate condition for linking each point based on an arrangement relationship between an arbitrary point and a plurality of points included in the two-dimensional range, a partial network to be a part of the contour is selected and the selected part Re-contouring the entire network that joins the network as a contour for each resolution,
Generate and process multi-resolution contour information by combining the contours proliferated corresponding to individual resolutions,
A program characterized in that an object or shape feature included in generated multi-resolution outline information is extracted and recorded in a storage unit.

  According to the present invention, when recognizing a general object included in an image, it is possible to perform stepwise object feature extraction using multiple contour information without requiring an enormous amount of calculation. The present invention can be applied to many uses such as image search using multiple contours and image classification.

10 Control unit (control means)
20 memory (storage means)
101 Image information acquisition means 102 Contour vector extraction means (contour information extraction means)
103 multi-resolution contour vector generating means (multi-resolution contour generating means)
104 Multi-resolution feature extraction unit 105 Object collation unit 106 Object collation result output unit 201 Image information storage unit (storage unit)
202 Multi-resolution feature information storage unit (storage means)

Claims (10)

An image matching system for matching an object or shape contained in an image,
Contour information extracting means for extracting one or a plurality of contour lines represented by a set of points as contour information from the image or the object or shape included in the image;
Based on the arrangement relationship between predetermined points and adjacent points when multi-resolution outline information expressed by a plurality of resolutions from the outline information is applied to different reference ranges for each resolution for each resolution to be multiplexed. Multi-resolution contour information generating means for generating a contour by forming the contour according to the weighting, and combining the formed individual contours;
Multi-resolution feature extracting means for extracting features of an object or shape included in the multi-resolution outline information;
Collating means for performing object or shape matching processing using features extracted from the multi-resolution contour information;
An image matching system comprising: The contour information extracting means extracts a set of points representing individual contour lines as the contour information by a point sequence,
2. The image collation according to claim 1, wherein the multi-resolution feature extraction unit uses the feature quantity to be extracted as an extraction requirement for each resolution to be applied, and uses the extraction sequence of the point sequence in accordance with other conditions. system. The multi-resolution contour information generating means includes:
A contour element selecting means for selecting an appropriate contour element from a reference range having a size set in advance for each resolution according to a weight based on the arrangement relationship;
Contour reproducing means for forming a contour corresponding to each resolution by joining the selected contour elements;
With
The image matching system according to claim 2, wherein a contour included in the multi-resolution contour information is reproduced for each resolution. The contour element selection means includes
Contour element point sequence forming means for forming a plurality of combinations of point sequences consisting of a series of three points centered on the predetermined point as contour element candidates within a reference range corresponding to each resolution centered on the predetermined point;
Point sequence selection means for selecting an appropriate contour element from the formed point sequence group based on the adaptability of the arrangement relationship;
The image collating system according to claim 3, further comprising: The point sequence selection means includes:
As the adaptability of the arrangement relationship,
The distance formed by the point sequence is large;
The angle formed by the point sequence is large;
The order of the point sequences formed by the point sequence forming means does not contradict the order of the point sequences when the point sequences are extracted by the contour information extracting unit;
The dispersion value of the orientation of the point sequence included in the adjacent range is small;
The image collation system according to claim 4, wherein the point sequence selection process is performed by considering any one, a combination, or all of the four items.   The contour information extracting means is configured such that an interval between points forming a contour line to be extracted as contour information is equal to or smaller than the size of a reference range used for generating the highest resolution recontour used in the multi-resolution contour information generating means. 6. The image collating system according to claim 1, wherein an interval adopted from a range satisfying 1/2 or more of an image pixel interval is used. Storage means for storing image information to be processed and features of an object extracted from the image information;
The contour line of the object described in the image obtained by image analysis of the image information is converted into data as a set of points as contour information,
Identifying an arbitrary point and a plurality of points included in a two-dimensional range having different sizes for each resolution centered on the arbitrary point from the set of points included in the contour information, In accordance with weighting using an appropriate condition for linking each point based on an arrangement relationship between an arbitrary point and a plurality of points included in the two-dimensional range, a partial network to be a part of the contour is selected and the selected part Re-contouring the entire network that joins the network as a contour for each resolution,
Generate and process multi-resolution contour information by combining the contours proliferated corresponding to individual resolutions,
Image feature extraction means for extracting and recording features of an object or shape included in the generated multi-resolution contour information;
An image processing system comprising: An image matching method by an information processing system for matching an object or a shape included in an image,
From the image or the object or shape included in the image, one or more contour lines represented by a set of points are extracted and processed as contour information,
Based on the arrangement relationship between predetermined points and adjacent points when multi-resolution outline information expressed by a plurality of resolutions from the outline information is applied to different reference ranges for each resolution for each resolution to be multiplexed. A contour is formed according to the weighting, and the individual contours thus formed are combined and generated,
An object or shape feature included in the multi-resolution contour information is extracted;
An image collating method characterized in that an object or shape is collated using a feature extracted from the multi-resolution outline information. An image processing method for obtaining multiple contours used when performing image matching for matching an object or shape contained in an image,
From the image or the object or shape included in the image, one or more contour lines represented by a set of points are extracted and processed as contour information,
Based on the arrangement relationship between predetermined points and adjacent points when multi-resolution outline information expressed by a plurality of resolutions from the outline information is applied to different reference ranges for each resolution for each resolution to be multiplexed. An image processing method characterized in that a contour is formed according to the weighting, and the generation processing is performed by combining the formed individual contours. An information processing system that performs image matching for matching an object or shape included in an image,
Contour information extracting means for extracting one or a plurality of contour lines represented by a set of points as contour information from the image or the object or shape included in the image;
Based on the arrangement relationship between predetermined points and adjacent points when multi-resolution outline information expressed by a plurality of resolutions from the outline information is applied to different reference ranges for each resolution for each resolution to be multiplexed. Multi-resolution contour information generating means for generating a contour by forming the contour according to the weighting, and combining the formed individual contours;
Multi-resolution feature extracting means for extracting features of an object or shape included in the multi-resolution outline information;
Collating means for performing collation processing of an object or a shape using features extracted from the multi-resolution contour information;
A program characterized by operating as

Images