JP2001338279A - 3D shape measuring device - Google Patents

Abstract

(57) [Summary] [PROBLEMS] A conventional three-dimensional shape measurement apparatus that reconstructs a three-dimensional shape from a plurality of images of a target object is susceptible to noise, is prone to errors, and is a simple and complicated method. However, there is a problem that a method having a problem with the restoration accuracy and a method of increasing the restoration accuracy require a reference shape in advance, and cannot be applied when the reference shape is unknown. SOLUTION: An image photographing means photographs a target object from a plurality of different viewpoints, acquires respective images, and performs photographing according to a photographing system model defining an optical system of the image photographing means and photographing parameters in the photographing system model. Each pixel value on each image is projected to the virtual space in the computer by the image projection means,
In the area where the projected projection planes overlap, calculate the degree of correlation of the pixel projection straight line that projected the pixel information of each pixel point, generate correlation degree data with the correlation degree calculation means, and construct a three-dimensional shape from the correlation degree data To generate the data to be processed.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

The present invention is used in industries such as product production (design, manufacture, inspection, etc.), information management (security systems, etc.), medical care (examination, treatment support, etc.), and services (clothing, etc.). 3D shape reconstruction.

[0002]

2. Description of the Related Art Conventional Technology 1 R. Dhond and JK Aggarwal: "Structure from stereo"
-A review ", IEEE Trans. On SMC, Vol.19, No.6, pp.
1489-1510 (1989). And RC Bolles, HH Baker, an
d MJ Hannah: "The JICST stereo evaluation", Proc.
of Image Understanding Workshop, pp. 23-274 (199
3). Ya, T. Kanade and M. Okutomi: "Astereo matchin
g algorithm with an adaptive window: Theory and exp
eriment ", IEEE Trans. on PAMI, Vol.16, No.4, pp.92
0-932 (1994). And MJHannah: "A System for Digita
l Stereo Image Maching ", Photorammetric Engineerin
gand Remote Sensing, Vol.55, No.12, pp.1765-1770
(1989). Describes a method of performing three-dimensional shape restoration by stereo image matching (hereinafter referred to as a correlation method).

Prior art 2 SB Marapane and MM Trivedi: "Multi-premitive hi
erarchical (MPH) stereo analysis ", IEEE Trans. on P
AMI, Vol.16, No.3, pp.227-240 (1994).
d and T. Skordas: "Structural matching for stereo v
ision ", Proc.of 9th Int'l Conf. on Pattern Recogni
tion, pp.439-445 (1994). and Y. Ohta and T. Kanad
e: "Stereo by intra- and inter-scanline search usi
ng dynamic programming ", IEEE Trans. on PAMI, Vol.
7, No.2, pp.139-154 (1985), describes a method for 3D shape reconstruction based on feature points.

Prior art 3 M. Okutomi and T. Kanade: "Multiple-Baseline Stere
o ", IEEE Trans. on PAMI, Vol.15, No.4, pp.353-363
(1993)., Y. Nakamura, T. Matsuura, and Y. Oht
a: "Occlusion detectable streo-Occlusion patterns
in camera matrix ", Proc. of CVPR, pp.371-378 (199
6). Describes a multi-baseline stereo method, which is a three-dimensional shape restoration method that reduces the ambiguity and improves the accuracy by integrating information of a plurality of stereo image pairs.

Prior art 4 Toshikazu Onda, Makoto Niwakawa, Nobuyuki Fujiwara: “Robot Vision Combining Feature Matching and Stereo Measurement”, Proc. Of the 3rd Image Sensing Symposium, pp. 177-180 (1997), A method for correcting the result of the prior art 1 to a more reliable three-dimensional shape restoration result is described. This is a method of calculating a deviation of a point group constituting a shape from a reference shape and removing a point having a large deviation to restore a highly reliable three-dimensional shape.

FIG. 7 is an explanatory diagram of a specific processing procedure of the prior art 4. Hereinafter, Prior Art 4 will be described with reference to FIG. 1. Change the shooting position and posture of two images of the same object
Each image is photographed by image photographing means comprising a digital camera of a CD element, and the photographed images and photographing parameters are used as input data. In the prior art 4, the image planes are arranged side by side in the x-axis direction in the same plane, and the coordinate system is defined based on the image plane. Therefore, the shooting parameters are: a. The distance between two images (relative shooting position) and b. The focal length (assuming that both images have the same focal length) is used.

Generally, the photographing parameters are as follows: (1) When considering a parallel projection photographing system, a. Shooting position of two images b. The shooting direction of each of those images c. The aspect ratio of each of these images (in the case of a digital camera having a CCD element, the aspect ratio changes when the arrangement density of the CCD elements differs vertically and horizontally, or when the CCD element arrangement plane is inclined with respect to the optical axis of the lens. In the case of parallel projection, the optical axis of the lens indicates a spatial axis in a direction in which a light beam enters the imaging surface.)

(2) In the case of an ideal (linear) projection projection imaging system, a. Shooting position of two images b. The shooting direction of each of those images c. Aspect ratio of each of these images (in the case of a digital camera having a CCD element, the aspect ratio changes when the arrangement density of the CCD elements differs vertically or horizontally, or when the CCD element arrangement plane is inclined with respect to the optical axis of the lens). Here, the optical axis of the lens represents a straight line passing through the point where light rays are condensed when entering the lens, and passing through the center of the bundle where light rays are condensed and diffused. Although the optical axis of the lens cannot be exactly represented by one line, it is generally defined as the straight line described above, assuming that the error is sufficiently small. Strictly speaking, the point at which light rays incident on the lens converge in the projection system is
It is not a point, but usually one point. d. Center coordinates of each of the images (coordinates of the intersection (image center point) of the perpendicular line drawn from the focal position of the digital camera to the image and the image plane) e. There are five focal lengths (distances from the image center point to the focal point) for each of those images.

[0009] 2. Corresponding to the arbitrary pixel of one of the two photographed images A and B projected into the virtual space in the computer based on the photographing parameters by the corresponding point searching means, by the geometric calculation based on the photographing system model and the correspondence probability calculation. Search for points and determine corresponding point candidates. Here, the photographing system model describes a photographed system and is modeled into a photographing system that considers an actual photographing system by measurement. That is,
The actual imaging system is not always exactly the same as the actual imaging system (it is natural to think that they do not match). The imaging system considered in the measurement is a model of the actual imaging system It is considered as.

As a model of the photographing system, for example, the photographing system of parallel projection and the photographing system of projective projection can be used.

In the geometric calculation, in the case of a parallel projection imaging system, (1) the position at which the projection is started, and (2) the space in the space from the direction of the projection. Calculation of where the projected light beam is located and in what direction is performed in each image, and a straight line projecting each pixel is defined in which position in the space. Is calculated, and how the pixel value of the intersection point (which may be any of luminance, hue, and saturation) is similar. In the correspondence probability calculation, a spatial point where a most similar point pair exists in each of two pixel points is set as a corresponding point candidate.

FIG. 7 shows a diagram of corresponding point candidates on one cross section for explanation, but actual corresponding point candidates exist in a three-dimensional space.

3. Step 2 is performed for all pixels, and corresponding point candidates for all pixels are determined. 4. To remove the estimated noise, select the corresponding point candidates from the corresponding point candidates and the reference shape obtained in step 3 using the corresponding point selection means.

In this conventional apparatus, the reference shape includes a plane including the average coordinate position of the estimated point on the surface of the three-dimensional shape and two spatial axes in which the estimated point has a large spatial spread (that is, the estimated point has the largest spread = The most scattered direction is defined as a first axis, the direction orthogonal to that axis and the direction in which the estimation points are spread next is defined as a second axis, and a plane composed of these two axes). The three-dimensional shape is restored.

The three-dimensional shape diagram shown in FIG. 7 describes corresponding point candidates that have been selected to form the three-dimensional shape. In FIG. 7, corresponding point candidates are described in a three-dimensional shape diagram for the sake of explanation, but actually, no corresponding point candidates are included in the three-dimensional shape. Further, the reference shape shown in FIG. 7 is for explanation of the concept, and a general three-dimensional shape is shown as an example of the reference shape instead of the reference shape shown in the paper of the related art 4. .

5. A surface is formed from the selected corresponding point candidate group, and a three-dimensional shape is formed. The surface can be represented, for example, by a triangular patch connecting the corresponding point candidates. FIG. 7 shows a diagram of a three-dimensional shape in one cross section for explanation, but an actual three-dimensional shape extends in a three-dimensional space.

[0017]

In the prior art 1, since the position and orientation of photographing are different for each image, even if the pixels photographed at the same physical point are inconsistent in shade and color. Also, depending on the shape and texture of the object, arbitrary matching may occur in a uniform area. In the matching method based on the region, the point having the highest correlation in the corresponding point search is determined as a candidate point. Therefore, the entire data is pulled to a corresponding point candidate that is greatly deviated, and thus is vulnerable to noise. Response is likely to occur.

In the method based on the features of the prior art 2, since parallax can be obtained only at feature points, it is necessary to interpolate the parallax of non-feature points. In addition, there remains the question of what are the feature points and how to calculate (given) them. At the present time, only a method using parallel projection has been proposed as an imaging system model due to the complicated calculation. Therefore,
There is a problem with the restoration accuracy.

In the multi-baseline stereo method of the prior art 3, ambiguity is reduced by integrating information of a plurality of stereo image pairs, but in order to improve reliability, it is necessary to capture images from many directions. However, there is still a problem such that erroneous correspondence cannot be fundamentally prevented when there are similar pixels in an image because it is basically the conventional technology 1.

In the method of selecting shape candidate points by deviation calculation according to the prior art 4, the shape candidate points are selected by deviation calculation in order to prevent the entire data from being pulled by the corresponding point candidates that are greatly deviated. For that purpose, it is necessary to give a reference shape serving as a reference at the time of calculation of the degree of deviation as preliminary knowledge in advance, and this cannot be applied when it is impossible to make the reference shape known. In addition, it is necessary to interpolate a portion from which noise has been removed.

[0021]

The three-dimensional shape measuring apparatus according to the present invention defines an image photographing means for photographing a target object from a plurality of different viewpoints and acquiring respective images, and an optical system of the image photographing means. Image projection means for projecting each pixel value on each image photographed by the image photographing means into the virtual space in the computer in accordance with the photographing system model and photographing parameters in the photographing system model, and a projection surface projected from each image. In the overlapping area, a correlation degree calculating means for calculating a correlation degree of a straight line (hereinafter, referred to as a pixel projection straight line) on which pixel information of each pixel point is projected to generate correlation degree data, and a three-dimensional shape from the correlation degree data. Shape configuration means for generating three-dimensional shape data to be configured.

Further, in the three-dimensional shape measuring apparatus according to the present invention, the data generated by the shape structuring means is surface data for restoring a three-dimensional shape, and the shape structuring means determines that each of the surface data is an object surface. Surface probability calculation means for calculating the configured surface probabilities from the similarity data, and surface configuration means for generating data forming the object surface from the surface probabilities.

Further, in the three-dimensional shape measuring apparatus according to the present invention, the data generated by the shape configuration means is volume data including transparency for restoring a three-dimensional shape in information, and the shape configuration means includes a similarity measure. Surface probability calculation means for generating surface probability data from the data, and volume data generation means for generating volume data from the surface probability data.

Further, in the three-dimensional shape measuring apparatus according to the present invention, the surface probability calculating means calculates eigenvalues / eigenvectors in a local region from the Hessian matrix of the correlation degree data, and compares the eigenvalues of each basis with the primitive of the shape. (Hereinafter, referred to as a local shape primitive), and the direction of the local shape primitive is calculated from the direction of the eigenvector.

Further, in the three-dimensional shape measuring apparatus according to the present invention, the surface probability calculating means may include:
It was configured to output one of four local shape primitives: uniform primitives, point primitives, straight line primitives, and planar primitives.

Further, in the three-dimensional shape measuring apparatus according to the present invention, the surface probability calculating means further uses the constraint condition that only one ray from one three-dimensional point is projected on one pixel, and It is configured to remove those with low surface probabilities from the original shape.

[0027]

Embodiment 1 Hereinafter, Embodiment 1 of the present invention will be described with reference to the drawings.
1 to 3 are explanatory diagrams of the first embodiment of the present invention. FIG.
FIG. 2 is an explanatory diagram of a configuration of the apparatus, FIG. 2 is an explanatory diagram of a processing procedure, and FIG. 3 is an explanatory diagram showing an embodiment of classification of local shapes in a surface probability calculating means.

In FIG. 1, reference numeral 201 denotes an image photographing means for photographing a target object, for example, a digital camera having a CCD element. 202 is an image projection unit for projecting each pixel value on the captured image captured by the image capturing unit 201 into the virtual space in the computer, and 203 is an area where the projection surfaces of the images projected by the image projection unit 202 overlap. Correlation degree calculating means for calculating correlation degree (similarity) of a straight line (hereinafter, referred to as pixel projection straight line) on which pixel information of each pixel point is projected to generate correlation degree data. This is a shape forming unit that generates three-dimensional shape data 207 for forming an original shape. The shape forming unit 204 includes a surface probability calculating unit 205 and a surface forming unit 206.

The processing procedure of the three-dimensional shape measuring apparatus configured as described above will be described with reference to FIG. 1. A target object is photographed by changing the position and orientation by the image photographing means 201, two images, an image A303 and an image B304 are generated, and the photographed images 303 and 304 and the photographing parameters 30 are generated.
Get 2

Here, the photographing parameters 302 are the same as those in the prior art. (1) When considering a parallel projection photographing system, a. Shooting position of two images b. The shooting direction of each of those images c. The aspect ratio of each of these images (in the case of a digital camera having a CCD element, the aspect ratio changes when the arrangement density of the CCD elements differs vertically and horizontally, or when the CCD element arrangement plane is inclined with respect to the optical axis of the lens. In the case of parallel projection, the optical axis of the lens indicates a spatial axis in a direction in which light rays enter the imaging surface.)

(2) In the case of an ideal (linear) projection projection imaging system, a. Shooting position of two images b. The shooting direction of each of those images c. Aspect ratio of each of these images (in the case of a digital camera having a CCD element, the aspect ratio changes when the arrangement density of the CCD elements differs vertically or horizontally, or when the CCD element arrangement plane is inclined with respect to the optical axis of the lens). Here, the optical axis of the lens represents a straight line passing through the point where light rays are condensed when entering the lens, and passing through the center of the bundle where light rays are condensed and diffused. Although the optical axis of the lens cannot be exactly represented by one line, it is generally defined as the straight line described above, assuming that the error is sufficiently small. Further, in the projection imaging system, the point at which light rays incident on the lens converge is not strictly one point, but is generally considered to be one point. d. The center coordinates of each of these images (the coordinates of the intersection (center point of the image) of the image plane with the perpendicular drawn down from the focal position of the digital camera to the image. Here, the focal point is the concentration of light rays in an ideal projection projection imaging system). Do) e. The focal length of each of these images (the distance from the image center point to the focal point) is five.

Here, the term "system" refers to the mechanism of the space that defines the distance. In the case of a non-linear model in a projection projection imaging system, coefficients representing second- and third-order distortions are further introduced. There is a need.

At this time, the shapes of the image A303 and the image B304 are not limited to flat surfaces but may be curved surfaces. However, image A303
When the shape of the image B304 is a curved surface, it is necessary to use a photographing system model corresponding to the curved surface. Also, the concept of the multi-baseline method may be introduced and extended to the present invention by taking three or more images.

2. Next, using the image projection means 202, the photographed images A303 and B304 are projected onto the virtual space in the computer based on the photographing parameters 302. The pixel information to be projected at this time may be anything as long as it can be used in the corresponding point search, such as luminance and color information (hue, saturation) of the pixels in the image. The projection of the pixel information is performed based on an imaging system model such as an imaging system for parallel projection or an imaging system for projection projection.

In the case of a parallel projection photographing system, the photographing parameters 302 (1) starting position, (2) projecting direction, and (3) aspect ratio of the image pass through the spatial point where each pixel is located. Further, a straight line orthogonal to the image plane is calculated, and pixel information is allocated on the straight line.

In the case of a projection projection imaging system, it is an imaging parameter 302 (1). Shooting position of two images (2). The shooting direction of each of those images (3). Aspect ratio of each of these images (4). The center coordinates of each of these images (5). From the focal length of each of these images, a straight line is calculated for the spatial point where each pixel is located and the focal point, and pixel information is allocated on the straight line. An example of a straight line group to which the above-described pixel information is assigned is shown as a pixel projection straight line 305. Also, as described in step 3 below,
Projection and processing may be performed on a plurality of pieces of pixel information, and the information may be integrated.

2. The continuity of points on the object surface is taken into account during the calculation as follows. 3. The correlation degree calculation means 203 is used to generate correlation degree data 306 indicating the probability that a pixel point corresponds to each spatial point in the virtual space in the computer. Specifically, the degree of correlation of two or more pixel projection straight lines 305 projected from two or more image planes is calculated, and the correlation is assigned to the data at the intersection of the two straight lines in the correlation degree data 306. And the correlation degree data 30
Generate 6.

For example, the correlation degree is obtained by integrating the correlation degrees such as luminance and hue of each pixel projection straight line 305 (original pixel) with weight. Instead of the correlation degree of the pixel information itself, a correlation degree of a frequency space characteristic amount such as a normalized second derivative amount of the pixel information, a texture characteristic amount, or a Gabor filter processing result may be used.

4. Using the surface probability calculation means 205, the surface probability which is a surface or a straight line is locally calculated from the correlation degree data 306.
The surface probability can be calculated, for example, as follows. The eigenvalue / eigenvector of the Hessian matrix is calculated from the local data at each spatial point of the correlation degree data 306, and the surface probability can be calculated by comparing the magnitude of each eigenvalue. Here, the local data is data of a local area (a part of the data, for example, a 3 × 3 × 3 cubic area) in the target data. Details of the operation of the surface probability calculation means 205 will be described later.

5. The surface probability calculated in step 4 is calculated at all the spatial points in the virtual space in the computer, and the surface probability data 30 is calculated.
Calculate 7. At this time, in order to stabilize the calculation, for example, processing with low-resolution data (global processing) to processing with high-resolution data (local processing) and integrating them are performed. Calculation may be performed.

6. Using the surface configuration means 206, a plane thinning process (equivalent to a thinning process on a straight line) is performed to extract only a portion having a high surface probability from the surface probability data 307 obtained in step 5. , Three-dimensional shape data 207 is generated. A restored three-dimensional shape 308 is obtained from the three-dimensional shape data 207.

In the thinning process, for example, He
Eigenvalue / eigenvector of ssian matrix can be used. In this eigenvalue / eigenvector, the eigenvector represents the normal vector of the surface to be estimated. Therefore, if the maximum value of the surface probability is searched for in the direction axis of the eigenvector in the surface probability data 307, the area where the portion with the high surface probability is collected Can be estimated.

Here, the three-dimensional shape data 207 is restored as a curved surface as shown in a restored shape 308. Further, the restored shape 308 may have information necessary for data display and processing, such as information on colors and surface inclinations, as the data.

FIG. 2 shows a diagram of the correlation degree data, the surface probability data, and the three-dimensional shape in one cross section for explanation, but the actual correlation degree data 306, the surface probability data 307, and the three-dimensional shape 207 are Exists in three-dimensional space.

Next, a method of calculating a surface probability that is a local surface or a straight line from the correlation degree data 306 by the surface probability calculation means 205 will be described in further detail. The surface probability calculation means 205 classifies a local shape by calculation from the correlation degree data 306, and determines locally a surface or a straight line from the classification. FIG. 3 shows an embodiment of the classification of the local shape.

The local shape is classified as follows. The surface probability calculating means 205 first calculates an eigenvalue / eigenvector in a local region from the correlation degree data. Eigenvalues / eigenvectors generate, for example, a Hessian matrix of data values,
It is obtained by calculating the unique information.

The Hessian matrix is a second-order partial differential matrix of pixel brightness.

(Equation 1) I can write

(Equation 2)

[0048]

(Equation 3)

Next, as shown below, classification processing of the local shape primitive is performed by comparing the eigenvalues of the respective bases, and the direction of the local shape primitive is calculated from the direction of the eigenvector. ● When the magnitudes of the three eigenvalues are equal and small: uniform primitive 601. ● If the magnitudes of the three eigenvalues are equal and large: point primitive 602. ● When three eigenvalues are larger than the other one: linear primitive 603. ● If one of the three eigenvalues is larger than the other two: the planar primitive 604. Although FIG. 3 shows four local shape primitives, only a part of them may be used as needed in an actual system.

In the first embodiment, in addition to the pixel information, the spatial continuity of points constituting the object surface is introduced as a constraint condition, and corresponding points are obtained. Specifically, by having the correlation degree calculating means and the shape composing means, first, the correlation degree data is calculated, and then the characteristic amount of the correlation degree data, for example, the surface probability data representing the line-likeness or the surface-likeness. Is generated, and a three-dimensional shape is constructed therefrom. Thus, not only “correlation of pixel information is high” but also “spatially continuous” data can be extracted as a three-dimensional shape.

As a specific effect, even if the correlation is high, a point that is isolated in position in the space can be treated as noise. It is not the point with the highest correlation but has a certain level of correlation. , And points that seem to form a surface or a line from the surrounding state in space (points where high correlation appears continuously in space) can be treated as a surface or a line, respectively. Strong and reliable.

If this is compared with two-dimensional image processing, the conventional method is a method of extracting a shape by binarization and then correcting the shape. In the first embodiment, the shape is extracted after applying an emphasis filter to the grayscale image. It can be compared to the technique of doing. In the method of extracting a shape by binarization and then correcting the shape, the reliability of the obtained result is low because the data whose information has been lost is corrected. In the first embodiment, since the three-dimensional shape is restored from the data holding the information, a great improvement in reliability and accuracy can be expected.

Further, in order to use the general constraint that the object surface is locally continuous,
There is no need for complicated processing such as designating or extracting a feature point corresponding between pixels in advance. Therefore, it is not necessary to adjust the parameters of the system for each purpose, and the versatility is excellent. In addition, the algorithm is practical because it does not include manual or unstable algorithms at the moment. Furthermore, using the general constraint that “the object surface is locally continuous” means that the shape is locally defined and the shape is restored by joining the shapes globally. It can be configured freely when viewed from the viewpoint, and does not require the prior knowledge about the three-dimensional shape unlike the prior art 4.

By adding not only information on pixel values but also spatial information on the surface of the object “a local object constitutes a plane” to the constraint condition, incorrect correspondence (not correct correspondence, Correspondence calculated correctly only from the conditions) can be reduced, and the calculation can be greatly stabilized and high reliability can be achieved.

Since the shape of the image plane is not limited to a plane, it can be applied to various photographing modalities.

By applying pixel information not only to luminance but also to color information and the like, the range of application is expanded.

By using the normalized second derivative as pixel information, it is possible to make the pixel brightness robust to the offset change of the pixel brightness (the influence of noise is small).

By using the texture feature amount as pixel information, it is possible to perform more reliable shape restoration for an object having a large texture information amount.

By using the feature amount of the frequency space as information,
More reliable shape restoration can be performed on an object having a large amount of texture information.

By calculating the surface probability from the correlation data using the eigenvalue / eigenvector of the Hessian matrix in the surface probability calculation means, the surface shape can be effectively extracted.

A surface probability calculation procedure using eigenvalues / eigenvectors is defined by providing, as a criterion, a discriminant method using four local shape primitives, that is, a uniform primitive, a point primitive, a straight line primitive, and a plane primitive. it can.

Introducing the concept of multi-resolution into the present invention enables more stable shape determination.

Applying the multi-baseline method of integrating images captured from many directions to the present invention enables highly reliable shape measurement.

The surface probability calculating means may have a configuration in which the three-dimensional shape has additional information such as texture and surface inclination.
As described above, by providing the three-dimensional shape with additional information such as texture and surface inclination, data can be effectively used in display or subsequent processing.

Embodiment 2 Embodiment 2 of the present invention will be described. FIG. 4 is a configuration diagram of the apparatus, and FIG. 5 is an explanatory diagram of a processing procedure. The second embodiment has the same configuration as that of the first embodiment except for the configuration unit 401, and the configuration unit 401 includes a volume data generation unit 402 instead of the surface configuration unit 206 in the first embodiment. . That is, the shape configuration means 401 is the surface probability calculation means 2
05 and volume data generating means 402.

The processing procedure will be described below with reference to FIG. 1. By the image capturing means 201, two images, an image A303,
The image B 304 is photographed while changing the photographing position and orientation, and the photographing parameters 302 are obtained. At this time, the shapes of the image A303 and the image B304 are not limited to flat surfaces but may be curved surfaces.
Alternatively, three or more images may be captured, and the concept of the multi-baseline method may be introduced and extended to the present embodiment.

2. Using the image projection means 202, the photographed images A303 and B304 are projected onto the virtual space in the computer based on the photographing parameters 302. The pixel information to be projected at this time may be anything as long as it can be used in the corresponding point search, such as the luminance and color information of the pixels in the image. Also, as described in Procedure 3 below, projection and processing may be performed on a plurality of pieces of pixel information, and these may be integrated.

3. Using the correlation degree calculation means 203, correlation degree data 306 indicating the probability that a pixel point corresponds to each space point in the virtual space in the computer is generated. Specifically, two pixel projection straight lines 305 projected from two image planes
Is calculated, and correlation degree data 306 is generated. The correlation degree is, for example, a value obtained by integrating the correlation degrees such as luminance and hue of each pixel projection straight line 305 (original pixel) with weight. Also, instead of the degree of correlation of the pixel information itself, the normalized second derivative of the pixel information, the feature amount of the texture, the Ga
A correlation degree of a feature amount in a frequency space such as a processing result of a bor filter may be used.

4. The surface probability which is a surface or a straight line is locally calculated from the correlation degree data using the surface probability calculation means 205. The surface probability can be calculated, for example, as follows. The eigenvalue / eigenvector of the Hessian matrix is calculated from the local data at each spatial point, and the surface probability can be calculated by comparing the magnitude of each eigenvalue. The surface probability is calculated by the method described in the first embodiment.

5. The surface probabilities obtained in step 4 are calculated at all spatial points in the virtual space in the computer, and the surface probability data is calculated. At that time, to stabilize the calculation, for example,
Surface probability calculation using multiple resolutions may be performed, for example, by processing from low-resolution data (global processing) to processing with high-resolution data (local processing) and integrating them.

After that, with respect to the surface probability data generated by the surface probability calculation means 205, "one pixel (if the image is sufficiently focused) is obtained from only one three-dimensional point. A method of removing noise using a constraint condition that “light rays are detected” may be applied.

This method is similar in technique to the corresponding point searching means of the prior art 4. Prior Art 4 extracts the point having the highest probability at each pixel point with respect to “correlation data of pixel data itself” when associating pixels.
On the other hand, the method of the present invention is based on “correlation data of pixel data
Furthermore, the point having the maximum probability at each pixel point is extracted from the “surface probability data taking into account the spatial continuity of the object surface”. Since the processing is performed, the effect of noise removal is significantly improved as compared with the related art 4.

6. Using the volume data generation means 402, volume data with the surface probability set to be transparent (to make the portion with low surface probability transparent and to make the high portion opaque) is generated from the surface probability data 307 obtained in step 5, The original shape data 207 is generated. Alternatively, the surface probability data 307 itself may be described as volume data.

A method for classifying the shape of a local region in volume data is described in, for example, SatoY., Nakajima S., Shi
raga N., Atsumi H., Yoshida S., Koller T., Gerig G., an
d Kikinis R: “Three-Dimensional Multi-Scale Line F
ilter for Segmentation and Visualization of Curvil
inear Structures in Medical Imeges ”, in MedicalIme
ge Analysis, 2 (2): 143-168, 1998. In this research, local classification is performed to extract blood vessels and the like from volume medical images. The second embodiment is an application of this technique to shape measurement.

Further, as shown in FIG. 6, a process for emphasizing the change in the surface probability may be added to the volume data generating means 402. In this method, for example, a process such as a thinning process is performed on the surface probability data, so that the object surface of the volume data can be made clearer.

FIG. 5 shows the correlation data, the surface probability data, and the three-dimensional shape in one section for the sake of explanation, but the actual correlation data 306, the surface probability data 307, and the three-dimensional shape 403 are the same. Exists in three-dimensional space.

The presence of the surface shape emphasizing means enables more effective display of volume data.

In the surface probability calculation means, noise (miscorresponding points) can be effectively removed by introducing a constraint that only one three-dimensional light ray is projected on one pixel.

[0079]

According to the present invention, there is provided a correlation degree calculating means and a shape forming means. First, the correlation degree data is calculated, and then the characteristic amount of the correlation degree data, for example, the line-likeness or the surface-likeness is represented therefrom. Surface probability data is generated, and a three-dimensional shape is formed from the data. That is, in addition to the pixel information, the spatial continuity of points constituting the object surface is introduced as a constraint condition, and a corresponding point is obtained. Thus, not only “correlation of pixel information is high” but also “spatially continuous” data can be extracted as a three-dimensional shape.

In addition, since a general constraint that “the object surface is locally continuous” is used, it is not necessary to have any prior knowledge about three-dimensional shapes such as feature points. Furthermore, since the shape is locally defined and the shape is restored by connecting the shapes globally, a free shape can be configured when viewed globally. Does not require knowledge.

The surface probability data is not surface data, and some processing is required to construct the surface shape data of the object. By adding shape forming means such as a thinning process to the surface probability data, it is possible to “generate surface data by uniform processing”, which is difficult to perform with the conventional method. This method enables more reliable shape measurement.

For display purposes, more robust display (with less influence of noise) can be expected with less processing time by using volume data for the three-dimensional shape than when using surface data. .

The surface probability can be effectively extracted by calculating the surface probability from the correlation data using the eigenvalue / eigenvector of the Hessian matrix.

By defining as a criterion a discriminant method using four local shape primitives, that is, a uniform primitive, a point primitive, a straight line primitive, and a plane primitive, or a part thereof, a surface probability calculation procedure using an eigenvalue / eigenvector is defined. it can.

[Brief description of the drawings]

FIG. 1 is a configuration diagram of an apparatus according to Embodiment 1 of the present invention.

FIG. 2 is an explanatory diagram of a processing procedure according to the first embodiment;

FIG. 3 is an explanatory diagram showing an example of local shape classification by a surface probability calculation unit.

FIG. 4 is a configuration diagram of an apparatus according to Embodiment 2.

FIG. 5 is an explanatory diagram of a processing procedure according to the second embodiment.

FIG. 6 is an operation explanatory diagram of another example in Embodiment 2.

FIG. 7 is an explanatory diagram of a processing procedure of a conventional device.

[Explanation of symbols]

201: image photographing means, 202: image projecting means, 203: correlation degree calculating means, 204: shape composing means, 205: surface probability calculating means,
206: surface configuration means, 207: three-dimensional shape data, 401: shape configuration means, 402: volume data generation means.

Claims (6)

[Claims] An image photographing means for photographing a target object from a plurality of different viewpoints and acquiring respective images, a photographing system model defining an optical system of the image photographing means, and photographing parameters according to photographing parameters in the photographing system model. Image projection means for projecting each pixel value on each image taken by the means into the virtual space in the computer, and a straight line projecting pixel information of each pixel point in an area where the projection plane projected from each image overlaps ( (Hereinafter referred to as pixel projection straight line) correlation degree calculation means for calculating correlation degree and generating correlation degree data, and shape construction means for generating three-dimensional shape data for forming a three-dimensional shape from the correlation degree data. A three-dimensional shape measuring apparatus comprising: 2. The method according to claim 1, wherein the data to be generated is surface data for restoring a three-dimensional shape, and each of the surface data is used to calculate a surface probability forming an object surface from the similarity data. A three-dimensional shape measuring apparatus comprising: probability calculation means; and surface configuration means for generating data constituting an object surface from surface probabilities. 3. A surface probability calculating means for generating surface probability data from similarity data, wherein the data to be generated is volume data including transparency for restoring a three-dimensional shape. A three-dimensional shape measuring apparatus comprising volume data generating means for generating volume data from probability data. 4. The method according to claim 1, wherein the surface probability calculation means calculates the H of the correlation data.
Calculate eigenvalues / eigenvectors in the local region from the essian matrix, calculate shape primitives (hereinafter referred to as local shape primitives) by comparing eigenvalues of each basis, and calculate the orientation of local shape primitives from the orientation of eigenvectors. The three-dimensional shape measuring apparatus according to claim 2 or 3, wherein 5. The surface probability calculating means is configured to output any one of four local shape primitives of a uniform primitive, a point primitive, a straight line primitive, and a plane primitive as a local shape primitive. Item 3. The three-dimensional shape measuring apparatus according to Item 4. 6. The surface probability calculating means further uses a constraint condition that a ray from only one three-dimensional point is projected on one pixel, and removes a low surface probability from the obtained three-dimensional shape. The three-dimensional shape measuring apparatus according to claim 2 or 3, wherein the three-dimensional shape measuring apparatus is configured.