JP2014225168A - Program, device, and method for calculating similarity between images represented by feature point set - Google Patents

Abstract

A program for calculating a similarity between a query image and a reference image as accurately as possible when markers based on the reference image are arranged on a plane and the reference image is present in the query image. An apparatus and method are provided. A feature point p ′ of a query image, a feature point p of a reference image, and a Homography matrix H that projects from the reference image to the query image are input, and a rotation parallel progression sequence from the reference image to the planar image with respect to the query image A similarity calculation means for calculating T and calculating the similarity between a point obtained by projective transformation using the Homography matrix H and the rotation parallel progression T for a plurality of other feature points of the reference image and a feature point of the query image Have. [Selection] Figure 1

Description

  The present invention relates to a technique for calculating the similarity between images represented by feature point sets. Further, by calculating the similarity between images, a reference image (image to be searched) represented by a set of feature points is changed to a query image (image serving as a search key) also represented by a set of feature points. The present invention relates to a technique for retrieving a similar reference image with high accuracy.

  In recent years, image recognition and search techniques based on local feature points (feature vectors) have attracted attention. In particular, algorithms such as SIFT (Scale-Invariant Feature Transform) and SURF (Speeded Up Robust Features) are robust to changes in rotation and scale, and are widely used.

  Conventionally, a similar image retrieval framework using local features is called “Bag-of-Visual Words” (or Bag-of-Features, Bag-of-Keypoints) (for example, Patent Document 1 and Non-Patent Documents). 1). According to this technique, a sentence retrieval method using a Bag-of-Words model and a transposed index is applied to retrieval of similar images. Bag-of-Words expresses a sentence as a feature vector defined by the frequency of one word, and uses IDF (Inverse Document Frequency) derived in advance based on the sentence set to determine the similarity between sentences. It is a framework to derive. On the other hand, Bag-of-Visual Words quantizes the local feature quantity of an image, regards the local feature quantity after quantization as a word, and expresses it as one feature vector similarly defined by the frequency. The same analogy method can be applied using the weighting used.

  In some cases, it may be desirable to further improve the accuracy of the list of reference images arranged in descending order of similarity to the query image. Further, there is a case where it is desired to determine whether or not the same object is included in the query image and the reference image using a threshold value based on the similarity. For these cases, there is a reranking technique that calculates a more accurate score for the top N reference images obtained from the first search result (see, for example, Non-Patent Documents 2 and 3). According to the technique described in Non-Patent Document 2, the Homography matrix between the reference image and the query image is estimated for each of the top N reference images based on the neighborhood point correspondence with the query image. By using the number of point correspondences (inliers) that satisfy the constraint conditions defined by the Homography matrix as a new score, high accuracy of the score is achieved.

  On the other hand, from the viewpoint of services, conventionally, for example, trading card games (abbreviated as “trading cards”) have used augmented reality (AR) technology. For example, there is a card in which an AR marker is described (see, for example, Non-Patent Document 4). This is a smartphone that has downloaded a predetermined application, and reads the AR marker written on the card. As a result, the character appears on the display of the smartphone so as to emerge, and the game proceeds.

JP 2010-282581 A

J. Sivic et al., "Video Google: A Text Retrieval Approach to Object Matching in Videos," in Proc. ICCV, 2003. J. Philbin, O. Chum, M. Isard, J. Sivic, and A. Zisserman, "Object Retrieval with Large Vocabularies and Fast Spatial Matching," in Proc of CVPR, 2007. O. Chum and J. Matas, "Matching with PROSAC-ProgressiveSample Consensus," in Proc. Of CVPR, 2005. Bandai, “Carddass equipped with AR content will be released on July 30,” [online], [April 19, 2013 search], Internet <URL: http://gamebiz.jp/?p=15875> D. G. Lowe, "Distinctive Image Features from Scale-InvariantKeypoints," International Journal of Computer Vision, vol. 60, no. 2, pp.91-110, 2004. Y. Uchida, K. Takagi, and S. Sakazawa, "An Alternative to IDF: Effective Scoring for Accurate Image Retrieval with Non-Parametric DensityRatio Estimation," in Proc. Of ICPR, 2012. H. Jegou, M. Douze, and C. Schmid, "Product quantization fornearest neighbor search," in IEEE Trans. On PAMI, vol. 33, no. 1, pp117-128, 2011. H. Jegou, M. Douze, and C. Schmid, "Improving bag-offeatures for large scale image search," in IJCV, vol.87, no.3, pp.316-336, 2010.

However, according to the existing reranking technique, the Homography matrix is individually estimated and the score is calculated for the top N images. Therefore, the following problems arise.
(1) The estimation of the homography matrix requires at least four pairs of points. For this reason, a correct Homography matrix cannot be obtained unless all four points correspond to inliers. The probability that all four points correspond to inliers decreases exponentially as the outlier ratio increases, and the recognition accuracy also decreases.
(2) The reliability of the Homography matrix is calculated using point correspondence other than the point correspondence used for the calculation. Therefore, the verification of the Homography matrix cannot be performed unless at least four point correspondences are obtained. As a result, non-detection occurs.

  Such a technique of local feature points of an image can also be applied to a marker recognition technique using a card. For example, a natural image can be recognized as a marker, not an artificial marker such as an AR marker or a QR code (Quick Response Code) (registered trademark). In particular, a card or the like is generally arranged on a plane such as a table.

  Therefore, according to the present invention, the similarity between the query image and the reference image is calculated as accurately as possible with respect to the query image captured in a state where a marker (for example, a card) based on the reference image is arranged on a plane. A program, an apparatus, and a method are provided.

According to the present invention, in order to calculate the similarity between the query image represented by the feature point set and the reference image, a program for causing a computer mounted on the apparatus to function,
Input a point correspondence set P of the feature point p ′ of the query image and the feature point p of the reference image, and a Homography matrix H that projects from the reference image to the query image,
A first step of arbitrarily selecting two point correspondences (p1, p1 ′) (p2, p2 ′);
A second step of calculating a rotation parallel sequence T from the reference image to the planar image for the query image using the two point correspondence;
A third step of counting the number of neighbors in which the error between the points transformed by using the Homography matrix H and the rotational parallel progression T and the feature points of the query image are within a predetermined range; Repeat steps 1 to 3 for all points in the set P;
A fourth step of deriving a highly aligned rotating parallel train T having the largest number of neighbors;
Causing the computer to function as a similarity calculation means for executing a fifth step in which the number of neighboring points corresponding to the highly aligned rotation parallel progression T is within the predetermined range is set as the similarity between the reference image and the query image. It is characterized by.

According to another embodiment of the program of the present invention,
Local feature point extraction means for extracting a feature point set of local feature points from a query image and a reference image;
A computer is used as a matching unit that matches a feature point set of a query image with a feature point set of a reference image, and outputs a feature point p ′ of a query image that is similar in a predetermined range and a feature point p of the reference image to a similarity calculation unit. It is also preferable to make it function.

According to another embodiment of the program of the present invention,
The query image contains multiple reference images,
It is also preferable to cause the computer to function so that the Homography matrix H is projected onto a query image from a reference image different from the reference image.

According to another embodiment of the program of the present invention,
Reference feature point storage means for storing a set of local feature points extracted from each of a plurality of reference images;
Image search means for searching for top N reference images similar to the feature point set of the query image using reference feature point storage means;
For the top N reference images, the Homography matrix H between the query images is calculated in order from the top of the similarity, and the Homography matrix H that is similar to an inliner at a predetermined threshold or more is output. It is also preferable to further cause the computer to function as matrix estimation means.

According to another embodiment of the program of the present invention,
When the Homography matrix estimation means calculates the Homography matrix H between the query images for the top N reference images in order from the top of the similarity, first, the neighborhood point correspondence (inliner) is a predetermined number or more. It is also preferable to cause the computer to function so as to output the obtained Homography matrix.

According to another embodiment of the program of the present invention,
The matching means preferably causes the computer to function so that the most similar reference feature point p is output in association with each query feature point p ′.

According to another embodiment of the program of the present invention,
The matching means associates the most similar reference feature point p with each query feature point p ′, and when a plurality of query feature points p ′ are associated with the reference feature point p, the matching feature point p is the most. It is also preferable to make the computer function so that only similar query feature points p ′ are output in association with each other.

ようにコンピュータを機能させることも好ましい。 According to another embodiment of the program of the present invention,
The rotational translation sequence T has one rotational parameter θ and two translation parameters tx, ty,
The feature point p ′ of the query image, the feature point p of the reference image, the Homography matrix H, and the rotation parallel sequence T have a relationship of p ′ = H · T · p.

It is also preferable to make the computer function.

According to another embodiment of the program of the present invention,
It is also preferable to cause the computer to function so that the rotation parallel progression T is estimated from the two pairs of point correspondences using the nonlinear least square method, the Levenberg-Marquardt method, or the Newton method.

According to the present invention, an apparatus for calculating a similarity between a query image represented by a feature point set and a reference image,
Input a point correspondence set P of the feature point p ′ of the query image and the feature point p of the reference image, and a Homography matrix H that projects from the reference image to the query image,
A first step of arbitrarily selecting two point correspondences (p1, p1 ′) (p2, p2 ′);
A second step of calculating a rotation parallel sequence T from the reference image to the planar image for the query image using the two point correspondence;
A third step of counting the number of neighbors in which the error between the points transformed by using the Homography matrix H and the rotational parallel progression T and the feature points of the query image are within a predetermined range; Repeat steps 1 to 3 for all points in the set P;
A fourth step of deriving a highly aligned rotating parallel train T having the largest number of neighbors;
It has a similarity calculation means for executing a fifth step in which the correspondence number of neighboring points within a predetermined range for the highly aligned rotation parallel progression T is set as the similarity between the reference image and the query image. To do.

According to the present invention, using a device, a method for calculating a similarity between a query image represented by a feature point set and a reference image,
Input a point correspondence set P of the feature point p ′ of the query image and the feature point p of the reference image, and a Homography matrix H that projects from the reference image to the query image,
A first step of arbitrarily selecting two point correspondences (p1, p1 ′) (p2, p2 ′);
A second step of calculating a rotation parallel sequence T from the reference image to the planar image for the query image using the two point correspondence;
A third step of counting the number of neighbors in which the error between the points transformed by using the Homography matrix H and the rotational parallel progression T and the feature points of the query image are within a predetermined range; Repeat steps 1 to 3 for all points in the set P;
A fourth step of deriving a highly aligned rotating parallel train T having the largest number of neighbors;
The fifth step is characterized in that the number of neighboring points that fall within a predetermined range with respect to the highly aligned rotational parallel progression T is a similarity between the reference image and the query image.

  According to the program, apparatus, and method of the present invention, the similarity between a query image and a reference image is calculated as accurately as possible for a query image that is captured in a state where markers based on the reference image are arranged on a plane. be able to.

It is a functional block diagram of the search device in this invention. It is an image correspondence figure showing inlier and outlier based on a Homography matrix. It is explanatory drawing showing conversion between a query image coordinate system and a reference image coordinate system. It is explanatory drawing which matched one card | curd which appears in a query image, and the reference image. It is explanatory drawing which matched the other card | curd which appears in a query image, and the reference image based on FIG. It is explanatory drawing showing the usage pattern using the smart phone incorporating the search device of this invention.

  Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings.

According to the present invention, it can be divided into two embodiments as follows.
(1) Similarity calculation function: calculates a similarity between a query image represented by a feature point set and a reference image.
(2) Search device: Using a similarity calculation function, one or more reference images appearing in a query image are searched from a large number of reference images.
Hereinafter, the present invention will be described as a search device including a similarity calculation unit.

  FIG. 1 is a functional configuration diagram of a search device according to the present invention.

  According to FIG. 1, the search device 1 inputs a large amount of “reference images” in advance. In the reference image, at least one instance (object, marker) belonging to the same object or the same category is shown. In the search, the search device 1 inputs a “query image” serving as a search key, and searches for one or more reference images that appear in the query image.

  1 includes a local feature point extraction unit 11, a reference feature point storage unit 12, an image search unit 13, a matching / reranking unit 14, a homography estimation unit 15, and a similarity calculation unit 16. Have These functional components can be realized by executing a program that causes a computer installed in the search device to function. In particular, the similarity calculation unit 16 which is a characteristic part of the present invention can be understood as a similarity calculation method from the processing flow.

[Local feature point extraction unit 11]
The local feature point extraction unit 11 extracts a feature point set of local feature points from each of the query image and the reference image. The feature point set of the reference image is output to the reference feature point storage unit 12, and the feature point set of the query image is output to the image search unit 13.

  For example, SIFT, SURF, or ORB (Oriented FAST and Rotated BRIEF) is used as the local feature point extraction algorithm. These local feature points are described by coordinates p = (x, y) and feature quantity vector f. Note that the local feature points of the query image and the local feature points of the reference image have the same number of dimensions for the feature vector.

  For example, in the case of SIFT, a 128-dimensional feature point set is extracted from one image (see, for example, Non-Patent Document 5). SIFT is a technique for analyzing a characteristic local region using a scale space and describing a feature vector that is invariant to scale change and rotation. On the other hand, in the case of SURF, higher-speed processing is possible than in SIFT, and a 64-dimensional feature point set is extracted from one image. In addition, the ORB extracts a set of 256-bit binary feature vectors from one content by using BRIEF (Binary Robust Independent Elementary Features) as a feature description by a binary code. In particular, according to the ORB, it is possible to maintain an accuracy equal to or higher than that of SIFT or SURF and realize a speed increase of several hundred times.

[Reference Feature Point Accumulator 12]
The reference feature point accumulation unit 12 accumulates a feature point set of local feature points for each reference image (for each reference image identifier).

[Image Search Unit 13]
The image search unit 13 uses the reference feature point storage unit 12 to search for top N (> 1) reference images similar to the feature point set of the query image (for example, see Non-Patent Document 6). The shorter the distance between the feature point of the query image and the feature point of the reference image, the higher the similarity. Specifically, a method using Cartesian product quantization (for example, Non-Patent Document 7) or a method using Hamming Embedding (for example, Non-Patent Document 8), which is one of the Approximate Nearest Neighbor algorithms, LSH (Locality-Sensitive
It is also preferable to use (Hashing). The feature point set of the top N reference images is output to the matching / reranking unit 14, and the similarity ranking of the reference images is output to the Homography estimation unit 15.

[Matching / Reranking Unit 14]
The matching / reranking unit 14 has a matching function and a reranking function.
<Matching function>
The matching function associates the most similar reference feature point p with each query feature point p ′. When a plurality of query feature points p ′ are associated with the reference feature point p, only the query feature point p ′ most similar to the reference feature point p is associated. The point correspondence P = {(p, p ′)} set between the associated reference image and query image is output to the Homography estimation unit 15 and the similarity calculation unit 16.

<Reranking function>
The list of reference images output from the image search unit 13 is reranked in descending order according to the similarity output from the similarity calculation unit 16. For example, IDF (Inverse Document Frequency) can be used for the re-ranking. Finally, a reference image obtained with a higher score above a predetermined threshold is output as a search result.

[Homography estimation unit 15]
The Homography estimation unit 15 calculates a Homography matrix H between the top N reference images from the image search unit 13 and the query images in descending order of similarity. Here, when a Homography matrix H that is similar to a neighboring point (inliner) at a predetermined threshold or more is detected, the Homography matrix H is output to the similarity calculation unit 16.

  FIG. 2 is an image correspondence diagram showing inliers and outliers based on the Homography matrix. outlier is represented by a broken line.

In the query image and the target reference image, as the degree of similarity is higher, the feature vectors are projected geometrically linear. Therefore, the coordinates can be replaced by the Homography matrix H which is a planar projective transformation matrix. The Homography matrix H is expressed as follows.

  For the calculation of the Homography matrix, a set of corresponding points of feature points input from the matching / reranking unit 14 is used. The number of unknown parameters of the Homography matrix H is 8 (h11 to h33, h33 = 1), and a set of corresponding points gives two constraint equations. Therefore, this matrix H can be calculated by the least square method if there are four or more pairs of corresponding points.

Therefore, the Homography matrix estimation unit 15 randomly selects four sets from the set of corresponding points of the input feature points, and calculates a Homography matrix H from the four sets. Then, when each query feature point is projected using the Homography matrix H, it is determined as follows.
(1) If the reference feature point is projected close to a predetermined threshold value or less, it is determined as inlier.
(2) Conversely, if it is projected farther than the predetermined threshold, it is determined as outlier.
After this process is executed a plurality of times, only the Homography matrix H having the largest number of inliers is employed.

[Similarity calculation unit 16]
The similarity calculation unit 16 inputs the point correspondence P = {(p, p ′)} set of the feature point p ′ of the query image and the feature point p of the reference image, and the Homography matrix H. Then, the following steps are executed.
(S1) Two point correspondences (p1, p1 ′) (p2, p2 ′) are arbitrarily (randomly) selected.
(S2) The rotation parallel progression T from the reference image to the planar image with respect to the query image is calculated using the two point correspondence.
(S3) Count the number of neighbors in which the error between the projection transformation point using the Homography matrix H and the rotation parallel progression T and the feature point of the query image falls within a predetermined range. The error means the similarity (the smaller the error, the higher the similarity), and is expressed as follows.
Error: || p'-H · T · p || ²
Then, it is determined that the correspondence of neighboring points whose error is equal to or less than a predetermined threshold is inlier. Finally, count the number of inliers.

  Here, S1 to S3 are repeated for all point correspondences of the point correspondence set P.

(S4) A highly aligned rotational parallel train T having the largest number of neighbors is derived. That is, the rotation parallel progression T having the largest number of inliers is employed.
(S5) The number of neighbor points corresponding to the high-alignment rotation parallel progression T within the predetermined range is set as the similarity between the reference image and the query image.

  FIG. 3 is an explanatory diagram illustrating conversion between the query image coordinate system and the reference image coordinate system.

The reference image can be converted into the intermediate coordinate system by the rotation parallel progression T, and further converted into the query image by the Homography matrix H. The rotation translation sequence T has one rotation parameter θ and two translation parameters tx, ty. The feature point p ′ of the query image, the feature point p of the reference image, the Homography matrix H, and the rotation parallel progression sequence T have the following relationship.
p '= H ・ T ・ p

  Since the rotation parallel progression T is three unknown variables (tx, ty, θ), it can be calculated by the least square method using at least two pairs of point correspondences (p, p ′). As another method, the rotational parallel progression T can be calculated from the two pairs of point correspondences using a nonlinear least square method such as the Levenberg-Marquardt method or the Newton method.

According to the above-described Homography estimation unit 15, in order to estimate eight parameters, at least four sets of point correspondences are required. On the other hand, according to the similarity calculation unit 16, it is possible to estimate the rotational translation parameter only by correspondence between two sets of points. This is because it is only necessary to estimate three parameters based on translation and rotation on the assumption that the plane on which the reference image exists is known. As a result, the following effects can be obtained.
(1) The probability that all the two sets of point correspondences are inliers is overwhelmingly higher than the probability that all the four sets of point correspondences obtained by normal Homography estimation are inliers. Therefore, the possibility that the correct parameter can be estimated increases and the recognition accuracy also increases.
(2) Since only two sets of point correspondences are used for parameter estimation, the other point correspondences can be used for verification, and the undetected rate becomes low.

  FIG. 4 is an explanatory diagram in which one card shown in the query image is associated with the reference image.

  According to FIG. 4, it is assumed that the query image includes a state in which two cards (a plurality of reference images) are arranged on a plane, for example. Among them, when one card is detected to be similar to the reference image, a Homography matrix H for converting from the reference image to the query image is calculated.

  FIG. 5 is an explanatory diagram in which the other card shown in the query image is associated with the reference image based on FIG.

  According to FIG. 5, a reference image similar to the other card of the query image can be detected only by separately calculating the rotation parallel progression T using the Homography matrix H calculated in FIG. 4. That is, according to FIG. 5, as the Homography matrix H, a projection image projected from a reference image different from the reference image is used.

  FIG. 6 is an explanatory diagram showing a usage pattern using a smartphone incorporating the search device of the present invention.

  According to FIG. 6, the user is photographing the card | curd arrange | positioned on the table using the camera of a smart phone. At this time, the smartphone can detect a plurality of cards (reference images) as accurately as possible from a query image (photographed photograph) showing a plurality of cards. Thereby, in the smart phone which downloaded the predetermined application, the character can also be made to appear on the display of the smart phone by capturing query images obtained by photographing a plurality of cards. For example, the character A can appear only when the card A and the card B are photographed together.

  As described above in detail, according to the program, the apparatus, and the method of the present invention, the similarity between the query image and the reference image with respect to the query image that is captured in a state in which the marker based on the reference image is arranged on a plane. The degree can be calculated as accurately as possible.

  Various changes, modifications, and omissions of the above-described various embodiments of the present invention can be easily made by those skilled in the art. The above description is merely an example, and is not intended to be restrictive. The invention is limited only as defined in the following claims and the equivalents thereto.

DESCRIPTION OF SYMBOLS 1 Search apparatus 11 Local feature point extraction part 12 Reference feature point storage part 13 Image search part 14 Matching / reranking part 15 Homography estimation part 16 Similarity calculation part

Claims (11)

A program for causing a computer mounted on the apparatus to function in order to calculate the similarity between a query image represented by a feature point set and a reference image,
Input a point correspondence set P of the feature points p ′ of the query image and the feature points p of the reference image, and a Homography matrix H that projects from the reference image to the query image,
A first step of arbitrarily selecting two point correspondences (p1, p1 ′) (p2, p2 ′);
A second step of calculating a rotation parallel progression T from the reference image to a planar image with respect to the query image using the two point correspondence;
A third step of counting the number of neighbors in which an error between a point transformed using the Homography matrix H and the rotation parallel progression T and a feature point of the query image falls within a predetermined range; , Repeating the first to third steps for all point correspondences of the point correspondence set P,
A fourth step of deriving a highly aligned rotating parallel train T having the largest number of neighbors;
A computer as similarity calculation means for executing a fifth step in which the number of neighboring points corresponding to the high-alignment rotation parallel progression T is within a predetermined range is set as the similarity between the reference image and the query image. A program characterized by functioning. Local feature point extraction means for extracting a feature point set of local feature points from the query image and the reference image;
The feature point set of the query image and the feature point set of the reference image are matched, and the feature point p ′ of the query image and the feature point p of the reference image that are similar in a predetermined range are output to the similarity calculation unit. The program according to claim 1, further causing the computer to function as matching means. The query image includes a plurality of reference images,
The computer program according to claim 2, wherein the computer functions so that the Homography matrix H is projected from the reference image different from the reference image onto the query image. Reference feature point storage means for storing a set of local feature points extracted from each of the plurality of reference images;
Image search means for searching for top N reference images similar to the feature point set of the query image using the reference feature point storage means;
For the top N reference images, the Homography matrix H between the query images is calculated in order from the top of the similarity, and the Homography matrix H that is similar to an inliner at a predetermined threshold or more is output. The program according to claim 2 or 3, further causing a computer to function as the homography matrix estimation means. When calculating the Homography matrix H between the query image and the query image in order from the top of the similarity with respect to the top N reference images, the Homography matrix estimation means first matches the neighborhood point (inliner). The program according to claim 4, wherein the computer is caused to function so as to output a Homography matrix having a predetermined number or more. The said matching means makes a computer function so that the most similar reference feature point p is matched and output with respect to each query feature point p ', The any one of Claim 2 to 5 characterized by the above-mentioned. Program. The matching means associates each query feature point p ′ with the most similar reference feature point p, and when a plurality of query feature points p ′ are associated with the reference feature point p, the matching feature point p is designated as the reference feature point p. The program according to claim 6, wherein the computer is caused to function so as to output only the most similar query feature point p ′ in association with each other. The rotational translation sequence T has one rotational parameter θ and two translation parameters tx, ty,
The feature point p ′ of the query image, the feature point p of the reference image, the Homography matrix H, and the rotation parallel sequence T are in a relationship of p ′ = H · T · p.

The program according to any one of claims 1 to 7, wherein the computer is caused to function as described above.   9. The computer system as claimed in claim 8, wherein the rotational parallel progression T is estimated from two sets of point correspondences using a nonlinear least square method, a Levenberg-Marquett method, or a Newton method. The program described in. An apparatus for calculating the similarity between a query image represented by a feature point set and a reference image,
Input a point correspondence set P of the feature points p ′ of the query image and the feature points p of the reference image, and a Homography matrix H that projects from the reference image to the query image,
A first step of arbitrarily selecting two point correspondences (p1, p1 ′) (p2, p2 ′);
A second step of calculating a rotation parallel progression T from the reference image to a planar image with respect to the query image using the two point correspondence;
A third step of counting the number of neighbors in which an error between a point transformed using the Homography matrix H and the rotation parallel progression T and a feature point of the query image falls within a predetermined range; , Repeating the first to third steps for all point correspondences of the point correspondence set P,
A fourth step of deriving a highly aligned rotating parallel train T having the largest number of neighbors;
It has similarity calculation means for executing a fifth step in which the corresponding number of neighboring points that fall within a predetermined range with respect to the highly aligned rotational parallel progression T is set as the similarity between the reference image and the query image. A device characterized by. A method of calculating a similarity between a query image represented by a feature point set and a reference image using an apparatus,
Input a point correspondence set P of the feature points p ′ of the query image and the feature points p of the reference image, and a Homography matrix H that projects from the reference image to the query image,
A first step of arbitrarily selecting two point correspondences (p1, p1 ′) (p2, p2 ′);
A second step of calculating a rotation parallel progression T from the reference image to a planar image with respect to the query image using the two point correspondence;
A third step of counting the number of neighbors in which an error between a point transformed using the Homography matrix H and the rotation parallel progression T and a feature point of the query image falls within a predetermined range; , Repeating the first to third steps for all point correspondences of the point correspondence set P,
A fourth step of deriving a highly aligned rotating parallel train T having the largest number of neighbors;
5. A method comprising: a fifth step in which a correspondence number between neighboring points that fall within a predetermined range with respect to the highly aligned rotation parallel progression T is set as a similarity between the reference image and the query image.

Images