JP2000348175A - Apparatus and method for detecting corresponding points - Google Patents

Abstract

(57) [Summary] [PROBLEMS] To detect a corresponding point between a search image and an image to be searched at a low calculation cost using a normalized correlation function. SOLUTION: A set of coordinates indicating a corresponding position between one image and another image by detecting a correlation between a plurality of images of the same object, searching for a correspondence between the images based on the correlation. Search means 11 for detecting a plurality of corresponding point candidates, and an evaluation value based on a correlation for each corresponding point candidate detected by the search means. Selecting means 12 for selecting a point candidate.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

[0001] 1. Field of the Invention [0002] The present invention relates to a corresponding point detecting apparatus and method for detecting corresponding points between a plurality of images by using images of the same object.

[0002]

2. Description of the Related Art With the development of three-dimensional image processing technology, particularly the development of display technology, applications using virtual reality (VR) have become widespread. And
For further development and spread in the future, it is very important to advance and simplify the technology for inputting shape data of real-world objects used in VR and the like.

The above-mentioned shape data is obtained by using a plurality of distance measuring devices arranged at different positions, obtaining distance data indicating the distance from each distance measuring device to the surface of the object, and obtaining an image taken by each distance measuring device. It can be obtained by pasting and editing.

When obtaining the distance data, a template composed of a plurality of pixels is extracted from a search image captured by each distance image measuring device, and the template is searched for on a search target image for which a correspondence relationship is to be obtained. Find the corresponding point on the searched image corresponding to
The distance between the distance measuring device and the object surface is obtained by using the corresponding point.

At this time, the correlation between the template included in the search image captured by each distance measuring device and the searched image is obtained by using a normalized correlation function, so that the correspondence between the template and the searched image is obtained. Ask for a relationship. According to this normalized correlation function, it is possible to determine the correspondence between each search image and the searched image, ignoring the linear change in the luminance value between the search image and the searched image. Then, the distance measuring device calculates the evaluation value according to the correlation obtained from the normalized correlation function. This evaluation value indicates the degree of similarity between the template and the corresponding point on the searched image. The evaluation value is high when the corresponding point has a high correlation, and is low when the corresponding point has a low correlation. As described above, the distance measuring apparatus generates a corresponding point candidate in which the coordinates of the point on the searched image having the highest evaluation value and the coordinates of the template are paired, and obtains the corresponding point candidate for the entire search image. Generate distance data.

[0006]

However, when the searched image from which the corresponding point is searched has a small change such as rotation or enlargement / reduction, the evaluation value of the corresponding point in the searched image sharply increases. Although it does not decrease, it gradually decreases as the searched image gradually changes. Therefore, unless the luminance value of the corresponding point in the image to be searched and the luminance value of the template of the search image are completely the same, the highest point of the evaluation value is not necessarily the corresponding point indicating the same point of the object. Not limited.

Further, even if another correlation function or the like different from the above-described normalized correlation function is used, it is difficult to regard a corresponding point having a high evaluation value as showing the same portion of the object as it is.

[0008] Furthermore, in the case of a correlation function that can reliably extract only the correspondence, the calculation cost becomes extremely large.

Accordingly, the present invention has been proposed in view of the above-described circumstances, and it is possible to detect a corresponding point between a search image and a search target image using a normalized correlation function with a small calculation cost. An object of the present invention is to provide an apparatus and a method for detecting corresponding points that can be performed.

[0010]

A corresponding point detection apparatus according to the present invention for solving the above-mentioned problems detects a correlation between a plurality of images of the same object and detects a correspondence between the images based on the correlation. Searching means for detecting a plurality of corresponding point candidates consisting of a set of coordinates indicating a corresponding position between one image and another image, and a method for detecting a plurality of corresponding point candidates based on a correlation of each corresponding point candidate detected by the searching means. Selecting means for selecting a corresponding point candidate having an evaluation value within a predetermined range from a plurality of corresponding point candidates using the evaluation value.

[0011] Such a corresponding point detecting device detects a highly accurate corresponding point candidate by selecting the corresponding point candidate obtained by searching by the searching means by the selecting means according to the evaluation value. The corresponding point detection device 1 detects, for example, a part of the search image as a template and detects a corresponding point candidate indicating the same place as a part of the target object indicated by the template.

Further, the corresponding point detecting method according to the present invention detects a correlation between a plurality of images of the same object, searches for a correspondence between the images based on the correlation, and detects one image and another image. A plurality of corresponding point candidates consisting of a set of coordinates indicating a corresponding position with the image are detected, and an evaluation value based on the correlation for each corresponding point candidate is used to calculate an evaluation value within a predetermined range from the plurality of corresponding point candidates. It is characterized in that corresponding point candidates are selected.

In such a corresponding point detection method, a corresponding point candidate having high accuracy is detected by selecting a corresponding point candidate obtained by searching according to an evaluation value. According to this corresponding point detection method, for example, a part of the search image is used as a template, and a corresponding point candidate indicating the same place as a part of the target object indicated by the template is detected.

[0014]

Embodiments of the present invention will be described below in detail with reference to the drawings.

A corresponding point detecting apparatus to which the present invention is applied is configured, for example, as shown in FIG.

The corresponding point detecting device 1 includes a corresponding point detecting unit 2 to which image data representing a plurality of images is input, and a storage unit used when the corresponding point detecting unit 2 searches for and selects a corresponding point candidate. 3 is provided.

The corresponding point detecting device 1 described below responds to the input of an image A and an image B indicated by image data obtained by imaging the same object to be imaged by imaging devices arranged at different positions. This is to detect point candidates. At this time, the corresponding point detection device 1 uses points included in the image A as a template and detects points included in the image B corresponding to the template.

In the following description, even when not only two images but also a plurality of images are input to the corresponding point detecting device 1, a corresponding point candidate is used by using one image and another image. Can be performed. For example, when the image A and the image B are input, a process of searching for and selecting a corresponding point candidate is performed.

The corresponding point detecting unit 2 is, for example, a JPEG (Join)
a search unit 11 for searching for a corresponding point candidate that is a candidate for a corresponding point indicating the same location of the target object between the image A and the image B indicated by the image data of the t Photographic coding Experts Group or bitmap format; And a selection unit 12 for deleting an inaccurate corresponding point candidate or the like from among the corresponding point candidates and selecting an accurate corresponding point candidate.

The search unit 11 receives images A and B from the outside.
In response to the above, a process of searching for a template that is a partial image in the image A to be searched and a point in the image B to be searched corresponding to the template to generate a corresponding point candidate is performed. The candidates are output to the selection unit 12 via the storage unit 3. Here, the search unit 11 performs a process of searching for corresponding point candidates as much as possible.

At this time, the search unit 11 performs a process of extracting a template composed of, for example, 32 × 32 pixels from the image A.

Next, the search section 11 performs a process of obtaining a correlation between the template and an arbitrary point on the image B by using a normalized correlation function at an arbitrary point on the image B.

The above-mentioned normalized correlation function means that when there are an image A and an image B having a horizontal pixel number w and a vertical pixel number h, A (x, y) Coordinates (x, y)
, And B (x, y) is represented by coordinates (x,
When the luminance value is set to y), the function is a function for calculating a correlation expressed by the following equation.

[0024]

(Equation 1)

Next, the search unit 11 obtains an evaluation value indicating the similarity between the template and an arbitrary point on the image B based on the correlation obtained by using the normalized correlation function, and determines that the evaluation value is a predetermined value. When the value is equal to or larger than the threshold value, a process of temporarily storing in the storage unit 3 a set of coordinates cut out from the image A as a template and coordinates at an arbitrary point on the image B as corresponding point candidates.

The search unit 11 performs a process of obtaining a correlation for the entire screen by repeating a process of obtaining a correlation using a normalized correlation function by sequentially shifting points on the image B for obtaining a correlation with the template. Then, a process of searching for corresponding point candidates for the entire screen is performed.

As a result, when the image as shown in FIG. 2A is input as the image A and the image as shown in FIG. By performing a search by setting templates a, b, c, and d for the image A as shown in FIG. 2A, as shown in FIG.
Areas a ′, b ′, c ′, and d ′ corresponding to c and d can be searched. Here, the black circles in FIG.
The coordinates of the templates a, b, c, and d shown in FIG.

When searching for a correlation, the search unit 11 considers not only the normalized correlation function represented by the above equation but also other calculation costs when executing the normalized correlation function. A correlation function may be used.

After performing the above-described search, the search unit 11 may further perform a search process after performing a selection process described later. At this time, the search unit 11 performs, for example, a process of searching only the vicinity of the corresponding point candidate selected by the selection unit 12 and performs a process of newly detecting the corresponding point candidate, thereby achieving a more accurate response. Many point candidates can be detected.

Further, the search unit 11 not only performs the search for the corresponding point candidate for the correlation on the entire screen but also reduces the processing amount and increases the processing efficiency, and performs the correlation for only a part of the image. And a process of searching for a corresponding point candidate may be performed. Further, a process of searching for a corresponding point candidate only for an image portion expressing an imaging target may be performed.

Further, the search unit 11 cannot calculate an accurate correlation when the average of the luminance values of the template to be cut out from the image A and the image B for which the correlation is calculated is low or when the variance of the pixel values is small. Alternatively, a threshold may be set for the average or variance of the luminance value to determine whether or not to calculate the correlation, thereby reducing the processing amount and increasing the processing efficiency.

Further, since the correlation does not greatly change at a position where the luminance value of an adjacent pixel is very small in the pixel pattern corresponding to the pattern of the image B, the search unit 11 determines the number of times to obtain the correlation according to the pattern. , The processing for obtaining the correlation may be thinned out, the processing amount may be reduced, and the processing efficiency may be improved.

The selecting unit 12 removes inaccurate corresponding point candidates from a plurality of corresponding point candidates using the evaluation value based on the correlation with respect to the corresponding point candidates stored in the storage unit 3 and searched by the search unit 11. I do. Here, the inaccurate corresponding point candidate is not an image indicating the same portion of the imaging target object, but is determined to have a high correlation by calculating a correlation only with the luminance value even though the correspondence is not established. This is a corresponding point candidate having an increased evaluation value.

The selection section 12 performs a first selection process using regression, a second selection process using stability, a third selection process using proximity, and a second selection process using uniformity of direction. By performing the selection process of No. 4, a process of sifting out inaccurate corresponding point candidates is performed.

The processing of the selection unit 12 when performing the first selection processing using the regression will be described.

The selection unit 12 uses the coordinates in the image B corresponding to the coordinates of the template in the image A among the coordinates of the corresponding point candidates obtained by the search unit 11 as a template, and
The image A is searched using the template in the image.

Then, as shown in FIG. 3, the selection unit 12 determines whether the area 22 in the image A searched using the template 21 in the image B is the template of the image A in the corresponding point candidate obtained by the search unit 11 described above. When the coordinates coincide with the coordinate area 23, processing for increasing the evaluation value of the corresponding point candidate is performed.

On the other hand, as shown in FIG. 4, the selection unit 12 determines whether the area 22 in the image A searched using the template 21 in the image B is the template of the image A in the corresponding point candidate obtained by the search unit 11 described above. When the coordinates are apart from the coordinates 23, a process of lowering the evaluation value of the corresponding point candidate is performed.

In the first selection process, the selecting section 12 uses the predetermined first threshold value, and when the evaluation value raised or lowered as described above is higher than the predetermined first threshold value, the corresponding point is determined. When the evaluation value is lower than the predetermined first threshold, the process is performed to delete the candidate from the corresponding point candidate.

Next, the processing of the selecting section 12 when performing the second selection processing using the above stability will be described.

The selection unit 12 cuts out the template from the image A performed by the search unit 11 described above, searches the image B for a point corresponding to the template, and changes the point searched as a corresponding point candidate to the image B. Is compared with a point on the image B obtained by the search, and the corresponding point candidate point searched by the search unit 11 matches the point when the changed image B is used. Is determined. At this time, the selection unit 1
2 performs a process of giving a change to the image B and correcting a search position when searching the image B using the template of the image A according to the change.

FIG. 5 is a diagram showing an example when searching in the image B using the template 31 in the image A. In this case, it is assumed that the area in the image B corresponding to the template 31 is the area 32.

FIG. 6 shows the template 3 in the image A.
1, using the image B slightly rotated clockwise, for example.
It is a figure showing about another example at the time of searching inside. In this case, the selection unit 12 determines that the area 32a in the image B corresponding to the template 31 is at the rotated position, and uses the corresponding point candidate point searched by the search unit 11 and the changed image B. When it is determined that the corresponding point matches, and the area 32b is searched as a point in the image B corresponding to the template 31, the corresponding point candidate point searched by the search unit 11 and the changed image B It is determined that the region 32b does not coincide with the region 32b.

When the selecting unit 12 determines that the point when the changed image B is searched using the template of the corresponding point candidate searched by the searching unit 11 matches, the corresponding point candidate is determined. A process of increasing the evaluation value and decreasing the evaluation value of the corresponding point candidate when it is determined that the point when the image B after the change is searched using the template of the corresponding point candidate searched by the search unit 11 does not match. I do.

In addition, in the above-described second selection processing, the selection unit 12 uses the predetermined second threshold value and determines whether the evaluation value raised or lowered as described above is higher than the predetermined second threshold value. When the evaluation value is lower than the second threshold value, the process is performed to delete the corresponding point candidate from the corresponding point candidate.

The selection unit 12 performs not only the case where the search is performed by rotating the image B as described above, but also the process of selecting the corresponding point candidate by performing the search by enlarging or reducing the image B. good.

Next, the processing of the selecting section 12 when performing the third selection processing using the proximity will be described.

The selecting unit 12 determines the positional relationship between the first area included in the image A and the second area included in the image A, and includes the positional relationship between the first area and the image B searched as a corresponding point candidate together with the first area. Then, the positional relationship between the point and the second area and the point included in the image B searched as the corresponding point candidate is compared, and it is determined whether or not both have the same positional relationship.

The selector 12 is, for example, as shown in FIG.
Region 41a and image B included in image A as corresponding point candidates
Are searched for, and the area 42a included in the image A and the area 4a included in the image B as corresponding point candidates are searched.
When 2b is searched, it is determined that the positional relationship between the region 41a and the region 42a and the positional relationship between the region 41b and the region 42b are the same.

On the other hand, as shown in FIG.
The positional relationship between the area 41a and the area 42a, and the area 42a corresponding to the area 41a and the area 42b corresponding to the area 42a
Is determined to be different from the positional relationship.

Then, the selection unit 12 determines the positional relationship between the template included in the image A and the region included in the image A, and the points included in the image B searched as the corresponding point candidates together with the template and the positions included in the image B. When it is determined that the positional relationships with the included points are the same, the evaluation value is increased, and when it is determined that the positional relationships are different, a process of decreasing the evaluation value is performed.

In addition, in the above-described third selection processing, the selecting unit 12 uses the predetermined third threshold value and determines whether the evaluation value raised or lowered as described above is higher than the predetermined third threshold value. If the evaluation value is lower than the third threshold, the process is performed to delete the corresponding point candidate.

Next, a description will be given of the processing of the selection unit 12 when performing the fourth selection processing using the uniformity in the direction.

The selection unit 12 outputs an image A input from the outside.
When the two imaging devices that have captured the image and the image B are arranged in a parallel relationship, there is a restriction that the correspondence relationship between the image A and the image B is substantially parallel, and the uniformity in the above direction is used. A fourth selection process is performed.

The selector 12 moves the image A and the image B in parallel between the area A included in the image A searched as the corresponding point candidate and the area B included in the image B corresponding to the area A. When they are superimposed only by one another, the direction when the translation is performed, and depending on the restriction, the length of the translation is further determined.

As shown in FIG. 9A, for example, as shown in FIG. 9A, the selecting unit 12 connects the points included in the searched image corresponding to the image A and the points included in the image B as vectors, and Determine the direction and length of the vector. And
The selection unit 12 performs a process of increasing the evaluation value of the corresponding point candidate that has formed a plurality of vectors having the same length depending on the direction and constraint of each vector, as indicated by the thick line in FIG. As shown by the thin line in FIG. 9B, a process of lowering the evaluation value of the corresponding point candidate forming a plurality of vectors having different directions and lengths of the respective vectors is performed.

Further, in the above-described fourth selection processing, the selecting unit 12 uses the predetermined fourth threshold value and determines whether the evaluation value raised or lowered as described above is higher than the predetermined fourth threshold value. If the evaluation value is lower than the predetermined fourth threshold value, a process of deleting the corresponding point candidate from the corresponding point candidate is performed.

The selection unit 12 that performs the above-described first to fourth selection processing is the corresponding point candidate remaining after performing all the first to fourth selection processing for all the corresponding point candidates from the search unit 11. Is output. At this time, the selection unit 12
Among the corresponding point candidates after performing the first to fourth selection processes, corresponding point candidates having an evaluation value equal to or more than a predetermined threshold are output to the outside.

Further, as described above, the selection unit 12 may delete the corresponding point candidate having the evaluation value equal to or smaller than the predetermined threshold value after each selection process, and after each of the first to fourth selection processes. A process of deleting a corresponding point candidate having an evaluation value equal to or less than a predetermined threshold value may be performed.

Further, the selection unit 12 may change the amount by which the evaluation value of the corresponding point candidate is changed in the first to fourth selection processes according to the type of the input image. Thus, the selection unit 12 can reduce the amount of processing and increase the processing efficiency, and can detect an accurate corresponding point candidate.

Furthermore, the selection unit 12 can select the order in which the first to fourth selection processes are performed and the range of the evaluation value to be varied in the first to fourth selection processes so as to greatly narrow the corresponding point candidates. It is preferable to perform the processing and the selection processing in which the evaluation value is not likely to be lowered by the incorrect corresponding point candidate due to the incorrect corresponding point candidate in the order described above. Thereby, the selecting unit 12 can efficiently detect an accurate corresponding point candidate.

Further, the selection unit 12 may further perform a selection process using the corresponding point candidates obtained by performing the above-described selection process. At this time, the selection unit 12 performs the selection process again by changing, for example, the threshold in each of the above-described selection processes, the amount by which the evaluation value is raised and lowered, and the like. Thereby, the selection unit 12
Can detect more accurate corresponding point candidates.

Further, when outputting the corresponding point candidate to the outside, the selecting section 12 may represent the set of coordinates on the image A and the image B by a character string. The relationship may be expressed by drawing. This allows
The selection unit 12 can output data indicating corresponding point candidates in a format according to, for example, a distance measuring device connected to the outside.

Next, an example of a procedure for detecting a corresponding point by the corresponding point detecting device 1 will be described with reference to a flowchart of FIG.

According to this flowchart, first, in step S10, the corresponding point detecting device 1 responds to the input of the image data indicating the image A and the image data indicating the image B to the corresponding point detecting section 2. Then, the process of detecting the corresponding point candidate at the subsequent stage is started.

In the next step S 20, the corresponding point detecting section 2 searches for a corresponding point candidate consisting of a set of points corresponding to the template included in the image A and the template included in the image B by the selecting section 12. I do. At this time,
The selection unit 12 obtains a correlation using the above-described normalized correlation function, and obtains a corresponding point candidate together with an evaluation value based on the correlation. At this time, the selection unit 12 obtains a point in the image B having the highest correlation with each template in the image A, and determines the coordinates of each template and the image B corresponding to each template.
A corresponding point candidate in which the coordinates of the points within are paired is determined. As a result, for all the templates in the image A, the selection unit 12 obtains corresponding point candidates in which the template and the coordinates of the point in the image B having the highest correlation are paired.

In the next step S30, the selecting section 12
Performs a process of deleting an inaccurate corresponding point candidate or the like from a plurality of corresponding point candidates and selecting an accurate corresponding point candidate.

That is, in step S31 of step S30, the selection unit 12 performs the first selection process using the regression described above. That is, the selection unit 12 uses the coordinates in the image B corresponding to the coordinates in the image A among the coordinates of the corresponding point candidates obtained by the search unit 11 as a template, and
The evaluation value for each corresponding point candidate is varied according to the result of searching the image A using the template in the above. If the evaluation value is higher than the predetermined first threshold, the evaluation value is left as a corresponding point candidate. When the value is lower than the predetermined first threshold value, a process of deleting from the corresponding point candidate is performed.

In the next step S32, the selecting section 12
Performs a second selection process using stability by using the corresponding point candidates remaining after performing the above-described step S31. In other words, the selection unit 12 changes the evaluation value of the corresponding point candidate according to the result of searching the image B after the change using the corresponding point candidate template, and the evaluation value becomes larger than the predetermined second threshold value. If the evaluation value is higher than the second threshold value, the process is deleted.

In the next step S33, the selection unit 12
Performs a third selection process using proximity using the corresponding point candidates remaining after performing the above-described step S32. That is, the selection unit 12 includes the positional relationship between the first area included in the image A and the second area included in the image A, and the positional relationship between the first area and the second area included in the image A. The evaluation value is varied according to the positional relationship between the first point and the second point included in the image B searched as the corresponding point candidate of the second area, and the evaluation value is changed to a predetermined third threshold value. If the evaluation value is higher than the third threshold value, the process is performed to delete the corresponding point candidate.

In the next step S34, the selection unit 12
Are the images A and B input in step S10 described above.
It is determined whether there is a restriction on the direction of, for example, parallel movement between the two. When the selecting unit 12 determines that there is a restriction on the direction, the selecting unit 12 proceeds to step S3.
The process proceeds to step S5, and when it is determined that there is no restriction on the direction, step S30 by the selection unit 12 is terminated, and the process proceeds to step S40.

In step S35, the selection unit 12
A fourth selection process using directionality is performed using the corresponding point candidates remaining after performing the above-described step S33. That is, when the selection unit 12 connects the points included in the image A and the points included in the image B, which are the corresponding point candidates indicating the same location of the imaging target, and expresses them as vectors, the direction and length of each vector The evaluation value is varied in accordance with the above. When the evaluation value is higher than a predetermined fourth threshold, the evaluation value is left as a corresponding point candidate,
When the evaluation value is lower than the predetermined fourth threshold, a process of deleting from the corresponding point candidate is performed.

In step S40, the selection unit 12
For example, based on the corresponding point candidates after performing the processing shown in step S35 or step S33,
It is determined whether to perform the fourth process. And the selection unit 1
When it is determined that the first to fourth processes are to be performed again, the process returns to step S30 so that the process of step S30 is performed again on the corresponding point candidate after performing the process shown in step S35 or step S33. When it is determined that the first to fourth processes are not performed, step S50 is performed.
Proceed to.

In step S50, the search unit 11
After performing the above-described selection processing in step S30, it is determined whether or not to perform further search processing. And the search unit 11
When it is determined that the search processing is performed again, step S20 is performed.
Then, the process from step S20 to step S50 is repeated again, and when it is determined that the search process is not performed again, the process ends.

The corresponding point detecting device 1 having the above-described configuration shown in FIG. 1 and performing the processing shown in the flowchart of FIG. , And the selection unit 12 performs the first to fourth selection processing, whereby the correlation between the image A and the image B is high, and it is possible to detect a suitable corresponding point candidate as a corresponding point. Therefore, the corresponding point detection device 1 includes the search unit 1
There is no need to search for corresponding point candidates by making the normalized correlation function used in 1 complex, and the corresponding point can be detected with a small calculation cost.

Therefore, according to the corresponding point detecting apparatus 1, by obtaining the corresponding point candidates by the search unit 11 and the selection unit 12, the positional relationship between the input image A and the image B in the space can be obtained. By outputting the corresponding point candidates to the outside, it is possible to generate three-dimensional object data of the imaging target.

Further, the corresponding point detecting apparatus 1 performs a process of selecting corresponding point candidates in the order of the first selection process, the second selection process, and the third selection process, thereby providing an inaccurate high rate. From the selection process that may include the corresponding point candidate, it is possible to perform the selection process in which it is not preferable that the incorrect corresponding point candidate is sequentially included at a high rate, and the more accurate corresponding point candidate can be efficiently generated. Can be detected.

Further, when the corresponding point detection device 1 performs the selection process again using the corresponding point candidates obtained by performing the above selection process, for example, the threshold value in each of the above selection processes, the amount by which the evaluation value is raised or lowered, etc. Can be changed, so that more accurate corresponding point candidates can be detected.

Further, the corresponding point detecting device 1 includes a selecting unit 1
After performing the selection process in step 2, for example, a process of searching only the vicinity of the corresponding point candidate selected by the selection unit 12 is performed, and a process of newly detecting the corresponding point candidate can be performed. Many corresponding point candidates can be detected.

Further, the corresponding point detecting device 1 not only performs the first to fourth selection processing as shown in FIG. 10 but also reduces the processing amount and performs the first to fourth selection processing. Any of the selection processes may be performed in combination. At this time, the corresponding point detection device 1 can more efficiently and accurately detect more corresponding point candidates by adjusting the threshold value in each selection process performed by the selector 12.

That is, in the description of the corresponding point detecting apparatus 1 described above, an example in which the processing is performed in the order of the first, second, third, and fourth selection processing has been described.
For the purpose of detecting more accurate corresponding point candidates, each selection process may be omitted, or a selection process based on directionality may be added.

Further, according to the corresponding point detecting device 1, for example, an image A shown in FIG. 11A and an image B shown in FIG. Then, by setting, for example, four templates for the image A and searching for corresponding point candidates, four regions corresponding to black points (template positions) in the image B are searched to generate corresponding point candidates. Thus, the correspondence between the image A and the image B can be set. This allows
The corresponding point detection device 1 can generate a panoramic image as shown in FIG. 12 by splicing the image A and the image B using the corresponding point candidates.

[0083]

As described above in detail, according to the apparatus and method for detecting corresponding points according to the present invention, the correlation between a plurality of images of the same object is detected, and the correlation between the images is detected based on the correlation. Search for a correspondence relationship, and detect a plurality of corresponding point candidates consisting of a set of coordinates indicating the corresponding position of one image and another image,
Using the evaluation value based on the correlation for each corresponding point candidate,
By selecting a corresponding point candidate having an evaluation value within a predetermined range from a plurality of corresponding point candidates, a highly accurate corresponding point candidate can be detected. Therefore, according to the corresponding point detection apparatus and method, it is not necessary to search for a corresponding point candidate using a complicated normalized correlation function, and the number of corresponding points between the search image and the searched image is reduced using the normalized correlation function. It can be detected at the calculation cost.

[Brief description of the drawings]

FIG. 1 is a block diagram illustrating a configuration of a corresponding point detection device to which the present invention has been applied.

FIG. 2A is a diagram for explaining a state in which a template is set for an image A input to a corresponding point detection device to which the present invention has been applied, and FIG. 2B is a diagram illustrating corresponding point detection to which the present invention has been applied; FIG. 5 shows corresponding points in image B searched by the device.

FIG. 3 is a diagram illustrating an example of a selection process using regression performed by a corresponding point detection device to which the present invention has been applied.

FIG. 4 is a diagram for explaining another example of selection processing using regression performed by the corresponding point detection device to which the present invention has been applied.

FIG. 5 is a diagram for explaining an example of selection processing using stability performed by the corresponding point detection device to which the present invention is applied.

FIG. 6 is a diagram for explaining another example of the selection processing using stability performed by the corresponding point detection device to which the present invention is applied.

FIG. 7 is a diagram for explaining an example of selection processing using proximity performed by the corresponding point detection device to which the present invention has been applied.

FIG. 8 is a diagram for explaining another example of the selection processing using proximity performed by the corresponding point detection device to which the present invention is applied.

FIG. 9 is a diagram illustrating an example of a selection process using directionality performed by the corresponding point detection device to which the present invention has been applied.

FIG. 10 is a flowchart for describing a processing procedure when detecting a corresponding point candidate by the corresponding point detection device to which the present invention is applied.

11A is a diagram for explaining a state in which a template is set for an image A input to a corresponding point detection device to which the present invention has been applied, and FIG. 11B is a diagram for explaining corresponding point detection to which the present invention has been applied; FIG. 5 shows corresponding points in image B searched by the device.

FIG. 12 is an image A shown in FIGS. 11 (a) and 11 (b).
FIG. 7 is a diagram showing an example of a panoramic image generated using the image B and the image B.

[Explanation of symbols]

1 Corresponding point detecting device, 2 Corresponding point detecting unit, 11 searching unit, 12 selecting unit

Claims (18)

[Claims] 1. A method for detecting a correlation between a plurality of images of the same object, searching for a correspondence between the images based on the correlation, and obtaining coordinates of coordinates indicating a corresponding position between one image and another image. Searching means for detecting a plurality of corresponding point candidates comprising a set, and using the evaluation value based on the correlation for each corresponding point candidate detected by the searching means, the correspondence of the evaluation values within a predetermined range from the plurality of corresponding point candidates. A corresponding point detecting device, comprising: selecting means for selecting a point candidate. 2. The method according to claim 1, wherein the selecting unit includes a pair of a first corresponding point included in the first image and a second corresponding point corresponding to the first corresponding point and included in the second image. Based on a first search result from the search means that has searched for a point candidate, a first corresponding point corresponding to the second corresponding point using the second corresponding point is used.
A second search result obtained by searching for the first corresponding point included in the image is obtained, and based on the first search result and the second search result from the search means, the first corresponding point and the second search result are obtained. The corresponding point detecting apparatus according to claim 1, wherein the corresponding point candidate is selected by changing an evaluation value of the corresponding point candidate formed of a pair with the corresponding point. 3. The method according to claim 1, wherein the selecting unit includes a pair of a first corresponding point included in the first image and a second corresponding point corresponding to the first corresponding point and included in the second image. A first search result obtained by searching for a point candidate and a second search performed using the first corresponding point to search for a second corresponding point included in the changed second image.
Is obtained, and based on the first search result and the second search result, the evaluation value of the corresponding point candidate composed of the pair of the first corresponding point and the second corresponding point is changed, The corresponding point detection device according to claim 1, wherein a point candidate is selected. 4. The image processing apparatus according to claim 1, wherein the selecting unit is configured to determine a first positional relationship between the first corresponding point and the second corresponding point included in the first image and a second image corresponding to the first corresponding point. The second positional relationship between the third corresponding point of the second image and the fourth corresponding point of the second image corresponding to the second corresponding point is obtained, and the first positional relationship and the second
Based on the positional relationship, the evaluation values of the corresponding point candidate composed of the first corresponding point and the third corresponding point and the corresponding point candidate composed of the second corresponding point and the fourth corresponding point are calculated as follows. The corresponding point detecting device according to claim 1, wherein the corresponding point candidate is selected by changing. 5. The method according to claim 1, wherein the selecting unit obtains a plurality of correspondences between a plurality of corresponding points included in the first image and a plurality of corresponding points of the second image corresponding to the plurality of corresponding points, The corresponding point detecting apparatus according to claim 1, wherein the evaluation values of the plurality of corresponding point candidates are changed based on the uniformity of the plurality of correspondences, and the corresponding point candidates are selected. 6. The correspondence means comprising a set of a first corresponding point included in a first image and a second corresponding point corresponding to the first corresponding point and included in a second image. A first corresponding point included in a first image corresponding to the second corresponding point is determined by using the second corresponding point based on a first search result from the searching unit that has searched for a point candidate. The searched second search result is obtained, and based on the first search result from the search means and the second search result, a corresponding point candidate consisting of a pair of a first corresponding point and a second corresponding point is determined. A first selection process of changing the evaluation value to select a corresponding point candidate, a first corresponding point included in the first image and a second corresponding point included in the second image corresponding to the first corresponding point; A first search result of searching for a corresponding point candidate composed of a set of corresponding points and a second correspondence included in the changed second image A second search result in which a point is searched using the first corresponding point is obtained, and a set of the first corresponding point and the second corresponding point is determined based on the first search result and the second search result. Selecting the corresponding point candidate by changing the evaluation value of the corresponding point candidate consisting of: the first positional relationship between the first corresponding point and the second corresponding point included in the first image And a second positional relationship between a third corresponding point of the second image corresponding to the first corresponding point and a fourth corresponding point of the second image corresponding to the second corresponding point. Based on the first positional relationship and the second positional relationship, a corresponding point candidate including a pair of a first corresponding point and a third corresponding point, and a second corresponding point and a fourth corresponding point. The method is characterized in that at least one of the third selection processes for selecting the corresponding point candidate by changing the evaluation value of the corresponding point candidate composed of a set is used. Corresponding point detecting device according to Motomeko 1. 7. The correspondence means comprises a set of a first corresponding point included in a first image and a second corresponding point corresponding to the first corresponding point and included in a second image. On the basis of the first search result from the search means that has searched for a point candidate, the second corresponding point is used by using the second corresponding point.
A second search result obtained by searching for a first corresponding point included in the first image corresponding to the corresponding point is obtained, and a second search result is obtained based on the first search result from the search means and the second search result. 1
The evaluation value of a corresponding point candidate consisting of a pair of the corresponding point and the second corresponding point is changed, and the first image included in the first image and the second image corresponding to the first corresponding point are changed. Using the first corresponding point, a first search result of searching for a corresponding point candidate consisting of a pair with a second corresponding point included in the second image and a second corresponding point included in the changed second image are obtained. A second search result obtained by the search is obtained, and an evaluation value of a corresponding point candidate composed of a pair of a first corresponding point and a second corresponding point is changed based on the first search result and the second search result. A first positional relationship between the first corresponding point and the second corresponding point included in the first image, and a third corresponding point and a second corresponding point of the second image corresponding to the first corresponding point. A second positional relationship with a fourth corresponding point of the second image corresponding to the second corresponding point is obtained, and based on the first positional relationship and the second positional relationship, Corresponding point candidate and the second consisting of 1 of the corresponding points and the set of the third corresponding point
2. The corresponding point detection device according to claim 1, wherein the corresponding point candidate is selected by changing an evaluation value of a corresponding point candidate composed of a pair of a corresponding point and a fourth corresponding point. 3. 8. Using a point included in one image located close to the corresponding point candidate selected by the selection means, detecting a correlation with another image, and detecting an image based on the correlation. The corresponding point detecting apparatus according to claim 1, further comprising a re-search means for searching for a corresponding point candidate indicating a corresponding relationship between the two. 9. The correspondence according to claim 1, further comprising a reselection unit that changes the range of the evaluation value using the corresponding point candidate selected by the selection unit and selects the corresponding point candidate again. Point detector. 10. A method for detecting a correlation between a plurality of images obtained by capturing the same object, searching for a correspondence between the images based on the correlation, and obtaining coordinates of coordinates indicating a corresponding position between one image and another image. Detecting a plurality of corresponding point candidates comprising a set, and using the evaluation value based on the correlation for each corresponding point candidate, selecting a corresponding point candidate having an evaluation value within a predetermined range from the plurality of corresponding point candidates. Corresponding point detection method. 11. A corresponding point candidate, which is a set of a first corresponding point included in a first image and a second corresponding point corresponding to the first corresponding point and included in a second image, is searched for. Based on the first search result, a second search result of searching for a first corresponding point included in the first image corresponding to the second corresponding point using the second corresponding point is obtained, Selecting a corresponding point candidate by changing an evaluation value of a corresponding point candidate composed of a pair of a first corresponding point and a second corresponding point based on the first search result and the second search result; The corresponding point detection method according to claim 10, wherein: 12. A corresponding point candidate composed of a set of a first corresponding point included in a first image and a second corresponding point corresponding to the first corresponding point and included in a second image is searched for. A first search result and a second search result obtained by searching for a second corresponding point included in the changed second image using the first corresponding point are obtained, and the first search result and the second search result are obtained. 11. A corresponding point candidate is selected by changing an evaluation value of a corresponding point candidate composed of a pair of a first corresponding point and a second corresponding point based on the search result of (i). Corresponding point detection method. 13. A first positional relationship between a first corresponding point and a second corresponding point included in a first image, and a third correspondence of a second image corresponding to the first corresponding point. A second positional relationship between the point and a fourth corresponding point of the second image corresponding to the second corresponding point is determined, and a first positional relationship is determined based on the first positional relationship and the second positional relationship. The corresponding point candidate is selected by changing the evaluation value of the corresponding point candidate composed of the pair of the corresponding point and the third corresponding point and the corresponding point candidate composed of the pair of the second corresponding point and the fourth corresponding point. The corresponding point detection method according to claim 10, wherein: 14. A plurality of correspondences between a plurality of corresponding points included in a first image and a plurality of corresponding point candidates of a second image corresponding to the plurality of corresponding points are obtained. The corresponding point detecting method according to claim 10, wherein the evaluation values of the plurality of corresponding point candidates are changed based on the uniformity of the corresponding points, and the corresponding point candidates are selected. 15. A corresponding point candidate composed of a set of a first corresponding point included in a first image and a second corresponding point corresponding to the first corresponding point and included in a second image is searched for. Based on the first search result, a second search result of searching for a first corresponding point included in the first image corresponding to the second corresponding point using the second corresponding point is obtained, The evaluation value of a corresponding point candidate composed of a pair of a first corresponding point and a second corresponding point is changed based on the first search result and the second search result to select a corresponding point candidate. 1, a corresponding point candidate composed of a set of a first corresponding point included in the first image and a second corresponding point corresponding to the first corresponding point and included in the second image is searched. A second search result in which the first search result and the second corresponding point included in the changed second image are searched using the first corresponding point. Is calculated based on the first search result and the second search result, and the evaluation value of the corresponding point candidate composed of the pair of the first corresponding point and the second corresponding point is changed. A second selection process to be selected, a first positional relationship between a first corresponding point and a second corresponding point included in the first image, and a second position of a second image corresponding to the first corresponding point. A second positional relationship between the third corresponding point and the fourth corresponding point of the second image corresponding to the second corresponding point is determined, and based on the first and second positional relationships, A corresponding point candidate consisting of a pair of a first corresponding point and a third corresponding point, and a second corresponding point
2. The method according to claim 1, wherein at least one of a third selection process of selecting a corresponding point candidate by changing an evaluation value of a corresponding point candidate formed of a set of corresponding points is used.
0. A corresponding point detection method according to claim 1. 16. A corresponding point candidate, which is a set of a first corresponding point included in a first image and a second corresponding point corresponding to the first corresponding point and included in a second image, is searched for. Based on the first search result, a second search result of searching for a first corresponding point included in the first image corresponding to the second corresponding point using the second corresponding point is obtained, An evaluation value of a corresponding point candidate composed of a pair of a first corresponding point and a second corresponding point is changed based on the first search result and the second search result, and the evaluation value of the corresponding candidate included in the first image is changed. A first search result of searching for a corresponding point candidate consisting of a set of the first corresponding point and the second corresponding point corresponding to the first corresponding point and included in the second image, and a second search result given the change. A second search result obtained by searching for a second corresponding point included in the image using the first corresponding point is obtained, and the first search result and the second search result are obtained. Based on the search result, the evaluation value of the corresponding point candidate composed of a pair of the first corresponding point and the second corresponding point is changed, and the first corresponding point and the second corresponding point included in the first image are changed. And a third positional relationship between the third corresponding point of the second image corresponding to the first corresponding point and the fourth corresponding point of the second image corresponding to the second corresponding point. And a corresponding point candidate consisting of a pair of a first corresponding point and a third corresponding point, and a second corresponding point based on the first positional relation and the second positional relation.
The corresponding point detecting method according to claim 10, wherein the corresponding point is selected by changing an evaluation value of a corresponding point candidate composed of a pair of the corresponding point and the fourth corresponding point. 17. A method for detecting a correlation with another image by using a point included in one image located close to a periphery of a selected corresponding point candidate, and determining a correspondence between the images based on the correlation. 11. The corresponding point detecting method according to claim 10, wherein the corresponding point candidate is searched for. 18. The corresponding point detecting method according to claim 10, wherein the corresponding point candidate is selected again by changing the range of the evaluation value using the selected corresponding point candidate.