EP2568253B1

EP2568253B1 - Structured-light measuring method and system

Info

Publication number: EP2568253B1
Application number: EP10850965.4A
Authority: EP
Inventors: Danwei Shi; Di Wu; Wenchuang Zhao; Qi XIE
Original assignee: Shenzhen Taishan Online Tech Co Ltd
Current assignee: Shenzhen Taishan Online Tech Co Ltd
Priority date: 2010-05-07
Filing date: 2010-05-07
Publication date: 2021-03-10
Anticipated expiration: 2030-05-07
Also published as: EP2568253A4; WO2011137596A1; CN102884397B; JP5567738B2; EP2568253A1; AU2010352828A1; JP2013525821A; CN102884397A; AU2010352828B2; US20130050476A1; US9360307B2

Description

TECHNICAL FIELD

The present application relates to a structured-light based measuring method and a structured-light based measuring system employing such a method, and particularly to a structured-light based measuring method and a structured-light based measuring system employing such a method with a high measurement precision.

TECHNICAL BACKGROUND

For a structured-light based measuring system, if a light beam spot and its corresponding coordinate Z are known, the coordinate of a cross point of the light beam on the surface of an object may be obtained. For this purpose, during the demarcation of the structured-light based measuring system, light beams of the structured-light are designed in such a way that the image distribution of a light beam of the structured-light does not overlap that of a neighboring beam of the structured-light within a camera, and the respective image positions are recorded for various depths (i.e. the coordinates Z). During the practical measurement, it is possible to quickly determine a light beam corresponding to an already known image point in a manner of determining a light beam corresponding to the demarcated image point that is closest to the known image point. An interpolating method may be used for determining 3D coordinates of the known image point. In an existing structured-light based measuring system, for the purpose of ensuring that the image distributions of any two light beams at different depths do not overlap with each other, it is necessary to concentrate the image distribution of each light beam, that is, a small image displacement corresponds to a large physical depth difference, which results in a low measuring precision.
WO 2004/011876 discloses a stereoscopic optical scanner which includes a marker generator. A plurality of marker generators may be disposed along a circumference of an object in order to project the optical markers on the surface of the entire object. The pattern projector projects predetermined patterns so that 3D scan data of the object can be obtained. Namely, space-encoded beams are projected on the surface of the object. That is, each marker is identified by relative position information of markers computed by processors based on 3D scans.
WO 2008/120457 discloses a 3D image measurement apparatus of non-static estimating movement state of object with a captured full-illumination reflected image and a reflected pattern image.
US 5,852,672 describes a 3D measuring system with six cameras placed around the measured object and that uses laser projection, where each of the sets of stereo pair of images is processed by the computer to generate three dimensional coordinates of the object surface at grid points, where the processing software establishes a correspondence of grid points seen from two perspectives i.e. cameras. In said document, a long-coherence length laser is divided into a pair of beams by a splitter. A pair of spatial filters directs the resulting pair of beam fans to cross. Then maximum intensity and minimum intensity at an intersect are used to determine information of an object.
US 2005/018209 A1 discloses an optical 3D digitizing method with an enlarged non-ambiguity zone, comprising: controllably projecting a fringe pattern having a shiftable position over a target area; capturing images obtained by high depth resolution sensing and low depth resolution sensing from respective measurement fields at least partially overlapping each other over the target area; determining absolute pixel 3D positions in the images obtained by low depth resolution sensing and high depth resolution sensing as a function of relations depending on the fringe pattern in the captured images and correspondence between the absolute pixel 3D positions in the images; extracting chromatic texture from the captured images; and building a complete textured 3D model from the absolute pixel 3D positions and the chromatic texture.
US 5,061,062 discloses a focus spot size controller for a variable depth triangulation ranging system. The ranging system includes an apparatus for emitting light beam to be focused onto an object, a light sensitive apparatus, a lens apparatus for imaging reflected light onto said light sensitive apparatus, and an apparatus for calculating system geometry and range from signals received from the light sensitive apparatus.

SUMMARY OF THE INVENTION

In view of the defect of a low measurement precision in the above structured-light based measuring system in the prior art, the technical problem to be addressed by the present invention is to provide a structured-light based measuring method and a structured-light based measuring system using such a method as defined by the independent claims 1 and 6.
To address the technical problem, the present invention provide a technical solution that includes a structured-light based measuring method including:

a matching process, which includes obtaining a sequence number and a low-precision depth of a laser light spot based on an image position of the laser light spot within a first camera according to a first mapping relationship in a demarcation database, searching for image positions of the laser light spot within a second camera according to the sequence number and the low-precision depth of the laser light spot to obtain candidates of matching point, and conducting matching according to the image position of the laser light spot within the first camera and the respective candidates of matching point within the second camera, to obtain a result of the matching; and
a calculating process, which includes obtaining an image position within the second camera that matches with the image position within the first camera according to the result of the matching, and determining a precise position of the laser light spot according to a second mapping relationship in the demarcation database.

In the inventive structured-light based measuring method, the demarcation database is obtained by a demarcating process including:
demarcating the first mapping relationship between an image position of each laser light spot within the first camera and the sequence number as well as the low-precision depth of the laser light spot, demarcating the second mapping relationship between an image position of each laser light spot within the second camera and the sequence number as well as the high-precision depth of the laser light spot, and storing the demarcated first and second mapping relationships in a memory to form the demarcation database for the use by the matching process and the calculating process.
In the inventive structured-light based measuring method, during the demarcating process, a position of a laser output port relative the first camera is adjusted to prevent image positions of any two laser light spots within the first camera from overlapping with each other.
In the inventive structured-light based measuring method, the distance between the second camera and the laser output port is larger than the distance between the first camera and the laser output port.
In the inventive structured-light based measuring method, the demarcating process and the matching process are performed in a condition that image positions at different depths of the same laser light spot are surrounded by a geometric region.
In the inventive structured-light based measuring method, the precise position of the laser light spot is obtained by an interpolating method applied on the image position in the second camera and the high-precision depth during the calculating process.
In the inventive structured-light based measuring method, conducting matching according to the image position of the laser light spot within the first camera and the respective candidates of matching point within the second camera during the matching process includes: searching for a reference matching pair according to a luminance difference of images of the laser light spot; and determining the optimal matching point using the reference matching pair.
In the inventive structured-light based measuring method, during the conducting matching according to the image position of the laser light spot within the first camera and the respective candidates of matching point within the second camera in the matching process, before searching for a reference matching pair according to a luminance difference of images of the laser light spot, the method further includes: conducting a 3D reconstruction of the candidates of matching point, to obtain a depth of each of the candidate of matching point; and conducting initial selection among the candidates of matching point according to the depths of the candidates of matching point.
There is provided a structured-light based measuring system, including a processing system, an imaging system and a projecting system, where the imaging system includes a first camera and a second camera, the projecting system includes a laser generator for generating laser light, and the processing system includes a matching module and a calculating module,
the matching module is adapted for obtaining a sequence number and a low-precision depth of a laser light spot based on an image position of the laser light spot within a first camera according to a first mapping relationship in a demarcation database, searching for image positions of the laser light spot within a second camera according to the sequence number and the low-precision depth of the laser light spot to obtain candidates of matching point, and conducting matching according to the image position of the laser light spot within the first camera and the respective candidates of matching point within the second camera, to obtain a result of the matching; and
the calculating module is adapted for obtaining an image position within the second camera that matches with the image position within the first camera according to the result of the matching, and determining a precise position of the laser light spot according to a second mapping relationship in the demarcation database.
In the inventive structured-light based measuring system, the demarcation database is obtained by a demarcating module through a demarcating process including: demarcating the first mapping relationship between an image position of each laser light spot within the first camera and the sequence number as well as the low-precision depth of the laser light spot, demarcating the second mapping relationship between an image position of each laser light spot within the second camera and the sequence number as well as the high-precision depth of the laser light spot, and storing the demarcated first and second mapping relationships in a memory to form the demarcation database for the use by the matching process and the calculating process.
In the inventive structured-light based measuring system, during the demarcating process by the demarcating module, a position of a laser output port relative the first camera is adjusted to prevent image positions of any two laser light spots within the first camera from overlapping with each other.
In the inventive structured-light based measuring system, the distance between the second camera and the laser output port is larger than the distance between the first camera and the laser output port.
In the inventive structured-light based measuring system, functions of the demarcating module and the matching module are implemented in a condition that image positions at different depths of the same laser light spot are surrounded by a geometric region.
In the inventive structured-light based measuring system, the precise position of the laser light spot is obtained by an interpolating method applied on the image position in the second camera and the high-precision depth during the calculating process.
In the inventive structured-light based measuring system, conducting matching according to the image position of the laser light spot within the first camera and the respective candidates of matching point within the second camera by the matching module includes: searching for a reference matching pair according to a luminance difference of images of the laser light spot; and determining the optimal matching point using the reference matching pair.
In the inventive structured-light based measuring system, during the conducting matching according to the image position of the laser light spot within the first camera and the respective candidates of matching point within the second camera by the matching module, before searching for a reference matching pair according to a luminance difference of images of the laser light spot, the matching module is further adapted for conducting a 3D reconstruction of the candidates of matching point, to obtain a depth of each of the candidate of matching point; and conducting initial selection among the candidates of matching point according to the depths of the candidates of matching point.
The structured-light based measuring method and a structured-light based measuring system using such a method of the present invention are advantageous in that the measurement precision of the system is greatly improved by adding the second camera for fine measurement to the existing structured-light measuring system.
In the invention, a demarcation database is established through the demarcating process, so that the structured-light measuring process is simplified. The image positions of any two laser light spots within the first camera do not overlap with each other, so that the accuracy of the mapping relationship between the image position and the depth of each laser light spot (that is formed on the measured object) within the first camera is ensured during the demarcating process. The distance between the second camera and the laser output port is larger than the distance between the first camera and the laser output port, so that a mapping relationship between the image position and the depth that is provided by the second camera is more precise than that provided by the first camera. A geometric region is used for surrounding image positions at various depths of the same laser light spot in order to conduct the demarcating process and the matching process, and the matching speed may be accelerated. During the calculating process, the precise position of the laser light spot is obtained by an interpolating method applied on the sequence of the image position in the second camera and the depth, so that multiple precise depths of the measured objected may be obtained. The matching is conducted according to the image position of the laser light spot within the first camera and the respective candidates of matching point within the second camera during the matching process, so that the matching result may be obtained more easily, simply and quickly.

BRIEF DESCRIPTION OF THE DRAWINGS

The present invention is now further described below with reference to the drawings and embodiments, in which,

Figure 1 is a schematic structural diagram of a structured-light based measuring system according to a preferred embodiment of the invention.
Figure 2 is a schematic diagram showing the images of laser light spots that are formed within a first camera of a structured-light based measuring system according to a preferred embodiment of the invention.
Figure 3 is a schematic diagram showing the division of image regions of laser light spots that are formed within the first camera of the structured-light based measuring system according to the preferred embodiment of the invention.
Figure 4 is a schematic diagram showing the images of neighboring laser light spots that are formed within a second camera of a structured-light based measuring system according to a preferred embodiment of the invention.
Figure 5 is a schematic diagram showing the division of image regions of laser light spots that are formed within a first camera of a structured-light based measuring system according to a preferred embodiment of the invention.
Figure 6 is a schematic structural diagram showing a structured-light based measuring system according to a preferred embodiment of the invention.
Figure 7 is a schematic structural diagram showing a structured-light based measuring system according to a preferred embodiment of the invention.

DETAILED DESCRIPTION OF PREFERRED EMBODIMENTS

Preferred embodiments of the present invention are described in detail below in connection with the attached drawings.
As shown in Figure 1, a structured-light based measuring method of the invention includes a matching process and a calculating process, in which the used demarcation database is obtained through a demarcating process.
During the demarcating process, a first mapping relationship between an image position of each laser light spot within a first camera 21 and the sequence number as well as a low-precision depth (i.e. scene depth) of the laser light spot is determined. Particularly, in the case of the first camera 21, it is assumed that an image position of a laser light spot i at a depth Zⁱ _j within the first camera 21 is denoted by (uⁱ _j, vⁱ _j ), and the image distributions of any two laser light spots are prevented from overlapping with each other by adjusting the position of the laser output port 31 (as shown in Figure 6) with respect to the first camera 21, as shown in the schematic image in Figure 2. Each of the point sets that are not overlapped and are separated in Figure 2 represents an image distribution of one laser light spot at various depths, and each point in the point set represents an image position of a laser light spot corresponding to the point set at a different depth. In this case, data recorded for the laser light spot i may be denoted by $\{\{(u_{1}^{i}, v_{1}^{i}), z_{1}^{i}\} \dots \{(u_{j}^{i}, v_{j}^{i}), z_{j}^{i}\} \dots \{(u_{N_{i}}^{i}, v_{N_{i}}^{i}), z_{N_{i}}^{i}\}\},$
here, N_i denotes the number of the demarcated images of the laser light spot i at various depths. In practice, to accelerate the matching, the point set may be surrounded by a regular geometric region in a shape such as a rectangle and an ellipse. Thus, the data recorded for the laser light spot i is denoted by $\{{Param}_{i} \{(u_{1}^{i}, v_{1}^{i}), z_{1}^{i}\} \dots \{(u_{j}^{i}, v_{j}^{i}), z_{j}^{i}\} \dots \{(u_{N_{i}}^{i}, v_{N_{i}}^{i}), z_{N_{i}}^{i}\}\},$
here N_i denotes the number of the demarcated images of the laser light spot i at various depths, and Param_i denotes a parameter for the region surrounding the point set, which may be the maximum and minimum horizontal and vertical coordinates of rectangle corners for a rectangular region surrounding the point set, or a center point and major and minor axes of an ellipse for an elliptic region surrounding the point set. Figure 3 shows schematic division by rectangular surrounding regions.
Further, during the demarcating process, a second mapping relationship between an image position of each laser light spot within a second camera 22 and the sequence number as well as a high-precision depth of the laser light spot is determined. Particularly, in the case of the second camera 22, it is assumed that an image position of a laser light spot i at a depth Zⁱ _j within the second camera 22 is denoted by (uⁱ _j, vⁱ _j ), in this case, data recorded for the laser light spot i may be denoted by $\{\{(u_{1}^{i}, v_{1}^{i}), z_{1}^{i}\} \dots \{(u_{j}^{i}, v_{j}^{i}), z_{j}^{i}\} \dots \{(u_{N_{i}}^{i}, v_{N_{i}}^{i}), z_{N_{i}}^{i}\}\},$
here, N_i denotes the number of experimental data of the laser light spot i. It shall be noted that the image positions of two laser light spots might be overlapped, as schematically shown in Figure 4. In practice, to accelerate the matching, the point set may be surrounded by a regular geometric region in a regular shape such as a rectangle and an ellipse. Thus, the data recorded for the laser light spot i is denoted by $\{{Param}_{i} \{(u_{1}^{i}, v_{1}^{i}), z_{1}^{i}\} \dots \{(u_{j}^{i}, v_{j}^{i}), z_{j}^{i}\} \dots \{(u_{N_{i}}^{i}, v_{N_{i}}^{i}), z_{N_{i}}^{i}\}\},$
here N_i denotes the number of the experimental data of the laser light spot i, and Param_i denotes a parameter for the region surrounding the point set, which may be the maximum and minimum horizontal and vertical coordinates of rectangle corners for a rectangular region surrounding the point set, or a center point and major and minor axes of an ellipse for an elliptic region surrounding the point set. Figure 5 shows schematic rectangular surrounding regions.
During the matching process, for example, the image position sequence within the first camera 21 is denoted by $(u_{1}^{A}, v_{1}^{A}), \dots, (u_{i}^{A}, v_{i}^{A}), \dots, (u_{M}^{A}, v_{M}^{A}),$
and the image position sequence within the second camera 22 is denoted by $(u_{1}^{B}, v_{1}^{B}), \dots, (u_{j}^{B}, v_{j}^{B}), \dots, (u_{N}^{B}, v_{N}^{B}) .$
Any possible point among the image position sequence within the second camera 22 that matches the image position $(u_{i}^{A}, v_{i}^{A})$
within the first camera 21 may be determined by the following steps (1), (2) and (3).
At step (1), the sequence number and a low-precision depth of the laser light spot are determined according to an image position within the first camera 21 and the recorded table (e.g. the demarcation database) in the first camera 21.
Because the image point sets of the various laser light spots do not overlap with each other within the first camera 21, according to the image position $(u_{i}^{A}, v_{i}^{A})$
within the first camera 21, the sequence number and a low-precision depth of a laser light spot corresponding to such image position may be determined directly (i.e. based on the first mapping relationship).
The determining of the sequence number and the low-precision depth may be as follows depending on different data in the recorded table obtained during the demarcation.

(A) If the point sets are not surrounded by geometric regions, in the case of an image position $(u_{i}^{A}, v_{i}^{A})$
within the first camera 21, the recorded table is searched and an item thereof containing an image position (uⁱ _j , vⁱ _j ) having the most similarity to the image position $(u_{i}^{A}, v_{i}^{A}),$
i.e. $\{\{(u_{1}^{i}, v_{1}^{i}), z_{1}^{i}\} \dots \{(u_{j}^{i}, v_{j}^{i}), z_{j}^{i}\} \dots \{(u_{N_{i}}^{i}, v_{N_{i}}^{i}), z_{N_{i}}^{i}\}\}$
is searched out. The calculating of the similarity may be based on a distance similarity of $R = - \{{(u_{j}^{i} - u_{i}^{A})}^{2} + {(v_{j}^{i} - v_{i}^{A})}^{2}\} .$
(B) If the point sets are surrounded by geometric regions, the recorded table is searched to find an item thereof containing an image position $(u_{i}^{A}, v_{i}^{A})$
falling within the geometric region, i.e. $\{{Param}_{i} \{(u_{1}^{i}, v_{1}^{i}), z_{1}^{i}\} \dots \{(u_{j}^{i}, v_{j}^{i}), z_{j}^{i}\} \dots \{(u_{N_{i}}^{i}, v_{N_{i}}^{i}), z_{N_{i}}^{i}\}\} .$
For example, in the case of a rectangular region, Param_i is {min_uⁱ ,max_uⁱ ,min_vⁱ ,max_vⁱ }, thus $\min_u^{i} \leq u_{i}^{A} \leq \max_u^{i}$
and $\min_v^{i} \leq v_{i}^{A} \leq \max_v^{i}$
shall be satisfied.

At step (2), according to the obtained sequence number of the laser light spot and the sequence number of the image position of the laser light spot within the first camera 21, the image distribution of the laser light spot within the second camera 22 is searched out and candidates of matching point are obtained based on the image distribution.
If the obtained sequence number of the laser light spot and the sequence number of the image position of the laser light spot within the first camera 21 are denoted by Index and T, respectively, the demarcated point data obtained from the demarcation data of the first camera 21 may be denoted by $\{(u_{T}^{Index}, v_{T}^{Index}), z_{T}^{Index}\} .$
The obtained sequence of the demarcated image distribution of the laser light spot within the second camera 22 may be denoted by $\{{Param}_{index} \{(u)\} \{(_{1}^{index}, v, _{1}^{index}), z, _{1}^{index}\} \dots \{(u_{j}^{index}, v_{j}^{index}), z_{j}^{index}\} \dots \{(u_{N_{index}}^{index}, v_{N_{index}}^{index}), z_{N_{index}}^{index}\}\} .$
Candidates of demarcated point are searched out from the sequence of the demarcated image distribution of a laser light spot Index within the second camera 22 by a manner of obtaining a demarcated point within a range having a center $z_{T}^{Index},$
i.e. $z_{T}^{Index} - d \leq z_{j}^{Index} \leq z_{T}^{Index} + d .$
Here, d denotes a matching search range that is defined manually.
If the number sequence of the satisfying demarcated points within the second camera 22 is denoted by {index ₁,···,index_P ,···,index_C }, for each element of the number sequence, any image point having satisfying similarity is determined from the image points $(u_{1}^{B}, v_{1}^{B}), \dots, (u_{j}^{B}, v_{j}^{B}), \dots, (u_{N}^{B}, v_{N}^{B})$
in the second camera 22 by a method (A) or (B) below depending on the data of the recorded table obtained during the demarcation.
(A) In the case that the point sets are not surrounded by geometric regions, for each demarcated image point $\{(u_{{index}_{p}}^{Index}, v_{{index}_{p}}^{Index}), z_{{index}_{p}}^{Index}\}$
from the sequence of satisfying demarcated image points within the second camera 22, any of the image points $(u_{1}^{B}, v_{1}^{B}), \dots, (u_{j}^{B}, v_{j}^{B}), \dots, (u_{N}^{B}, v_{N}^{B})$
that has satisfying similarity is determined as a candidate.
The calculation of the similarity may be based on a distance similarity $R = - \{{(u_{{index}_{p}}^{Index} - u_{j B}^{})}^{2} + {(v_{{index}_{p}}^{Index} - v_{j B}^{})}^{2}\},$
here R≥ Threshold and Threshold is a predefined value.
(B) In the case that the point sets are surrounded by geometric regions, any of the image points $(u_{1}^{B}, v_{1}^{B}), \dots, (u_{j}^{B}, v_{j}^{B}), \dots, (u_{N}^{B}, v_{N}^{B})$
that falls within the surrounding region Param_Index is determined.
For example, in the case of a rectangular surrounding region, the surrounding region Param_Index defined by {min_u^Index , max_u^Index ,min_v^Index ,max_v^Index } shall meet conditions of $\min_u^{Index} \leq u_{j}^{B} \leq \max_u^{Index}$
and $\min_v^{Index} \leq v_{j}^{B} \leq \max_v^{Index} .$
At step (3), the matching process is completed based on the known sequence of image positions $(u_{1}^{A}, v_{1}^{A}), \dots, (u_{i}^{A}, v_{i}^{A}), \dots, (u_{M}^{A}, v_{M}^{A})$
within the first camera 21 and the candidates of matching point therefor within the second camera 22.
As described above, the initial measurement of the depths of the laser light spot array forming image distribution within the first camera 21 has been completed, and the candidates of matching point from the point array within the second camera 22 are found. In the case of a certain point $(u_{i}^{A}, v_{i}^{A})$
within the first camera 21, if the depth of such point is determined as Z^A _i in the first camera 21 and candidates of matching point in the second camera 22 for such point are (u^B _j , v^B _j ), (u^B _k, v^B _k ) and (u^B _l, v^B _l ) (the description is made here by an example of three candidates, but the present invention is not limited to this), an object at the present stage is to find the best matching point for the point $(u_{i}^{A}, v_{i}^{A})$
among the candidates of matching point, e.g. by below steps (A), (B), (C) and (D).

(A) 3D reconstruction for the candidates of matching point is conducted. Here, Binocular Stereo Vision technologies may be used. The 3D reconstruction with a high precision may be completed once the binocular demarcating of the first and second cameras 21 and 22 is conducted. If the candidates of matching point (u^B _j, v^B _j ), (u^B _k, v^B _k ) and (u^B _l, v^B _l ) are respectively paired with the point $(u_{i}^{A}, v_{i}^{A})$
for the purpose of the 3D reconstruction, 3 depths $z_{j}^{AB},$
$z_{k}^{AB},$
and $z_{I}^{AB}$
may be obtained.
(B) Selection is made based on the depths. Particularly, an initial selection among the 3 depths may be made using $z_{i}^{A} .$
If a measurement precision of $z_{i}^{A}$
measured by the first camera 21 is d, any candidate of matching point that corresponds to a depth exceeding a range of $(z_{i}^{A} - \frac{d}{2}, z_{i}^{A} + \frac{d}{2})$
is discarded. Further, depending on various applications, if the shot scene depth range is limited, an allowable variation range such as a range from 1 to 5 meters may be provided for the reconstructed depth, for the purpose of the selection. For the ease of description, the candidate of matching point (u^B _l, v^B _l ) is discarded at this step, for example.
(C) The searching for a reference matching pair is conducted. Particularly, the optimal matching point for the point $(u_{i}^{A}, v_{i}^{A})$
shall be searched out from the remaining candidates of matching point. For example, if the candidate of matching point (u^B _l, v^B _l ) is discarded as above, an optimal matching point shall be searched out from the candidates of matching point of (u^B _j , v^B _j ) and (u^B _k, v^B _k ), which is relatively difficult because both of the candidates (u^B _j , v^B _j ) and (u^B _k, v^B _k ), which have a precision better than that of $z_{i}^{A},$
satisfy the precision requirement. Therefore, the selection can be made merely based on the image positions of the laser light spot in the camera. In this case, it is necessary to find a reference matching pair from the first and second cameras 21 and 22.
If the laser output port 31 is arranged between the first and second cameras, the common visual field is relatively large, which is preferable. Typically, a regular laser light spot array is obtained by the effects of both interference and diffraction of a point light source, and is characterized in that the luminance of spots close to the center of the light source is higher due to more interference than spots at the periphery of the light source, as a result, the areas of images of the spots close to the center of the light source are larger than those of the spots at the periphery of the light source. With such a character and the large common imaging visual field of the first and second cameras, a statistical method is used to search for a reliable reference matching pair as described below.
In the two pictures within the first and second cameras 21 and 22, the images of the light spots are sorted according to their areas, N light spot images with the largest areas are selected, a geometrical center position of the selected light spot images is calculated, and light spot images closest to the calculated geometrical center position are searched out, to obtain two light spot images within the first and second cameras 21 and 22 as the reliable reference matching pair.
It shall be noted that the above method for selecting reference matching points cannot ensure the reliability in the direction Y of the coordinate system, because of the comparable luminance of light spots along the same vertical line close to the central region. However, the reliability in the direction X of the coordinate system is high due to the relatively large luminance difference in the horizontal direction. Therefore, the X coordinate of the center region with high luminance can be obtained through the statistics above. Fortunately, the epipolar constraint can be used for the direction Y. Further in practice, a relatively regular laser light spot array is provided, and the candidates of matching point selected by the demarcation are typically around the same horizontal level and thus have very close Y coordinates, hence it is not necessary to conduct selection in the direction Y.
(D) The optimal matching points are obtained through the reference matching pair. If the obtained reference matching pairs are $(x_{\max}^{A}, y_{\max}^{A}),$
$(x_{\max}^{B}, y_{\max}^{B}),$
merely the comparison in the direction X is required as described above. That is, merely $|x_{\max}^{B} - u_{j}^{B}|$
and $|x_{\max}^{B} - u_{k}^{B}|$
are required for comparing with $|x_{\max}^{A} - u_{i}^{A}| .$
One of $|x_{\max}^{B} - u_{j}^{B}|$
and $|x_{\max}^{B} - u_{k}^{B}|$
that is closer to $|x_{\max}^{A} - u_{i}^{A}|$
is at a distance to the reference matching point in the second camera 22 that is the closest to the distance from the point $(u_{i}^{A}, v_{i}^{A})$
to the same reference matching point in the first camera 21 in the direction X, and thus is selected as the optimal matching point, which is assumed as $(u_{j}^{B}, v_{j}^{B})$
here. As such, the matching between the image position in the first camera 21 and the image position in the second camera 22 is finished, and $z_{j}^{AB}$
represents the depth of the point $(u_{i}^{A}, v_{i}^{A})$
in the first camera 21 with the higher precision.

Steps (A) and (B) above relate to the coarse selection and may be selected for performing as desired.
In addition, the following general assignment algorithm may be used for implementing the matching between the sequence $(u_{1}^{A}, v_{1}^{A}), \dots, (u_{i}^{A}, v_{i}^{A}), \dots, (u_{M}^{A}, v_{M}^{A})$
of known image positions in the first camera 21 and the respective candidates of matching point in the second camera 22.
If {p ₁,···,p _i···, p_M } and {q₁,···,q_j ···,q_M } denote two 2D labeling sequences, a corresponding similarity matrix C may be calculated as C = {c_i,j }, here C_i,j represents the similarity of p_i and q_j. The total similarity F may be defined as $F = \sum_{i = 1}^{M} \sum_{j = 1}^{M} c_{ij} x_{ij} .$
Here, $\sum_{i = 1}^{M} x_{ij} = 1, j = 1,2 \dots, M,$
$\sum_{j = 1}^{N} x_{ij} = 1, i = 1,2 \dots, M$
and x_ij is equal to 0 or 1. x_ij is an MxN matrix.
The maximum of the total similarity F may be obtained by varying x_ij. If x _ij=1, it may be determined that p_i and q_j forms a matching pair. Otherwise, p_i and q_j do not form a matching pair.
The matching process above involves a relatively large amount of calculation due to the assumption that any of p_i and q_j might form a matching pair, which is a Non-deterministic Polynomial (N-P) problem. Many optimized algorithms were proposed for the N-P problem, but involve a calculation amount which is dramatically increased along with the increase of the size of the similarity matrix C.
If the similarity matrix C is characterized by $C = [\begin{matrix} A & 0 \\ 0 & B \end{matrix}],$
the N-P problem may be simplified to apply the above calculation on the matrix A and the matrix B, and thus the calculation amount involved is decreased greatly. Therefore, $C = [\begin{matrix} A & 0 \\ 0 & B \end{matrix}]$
is preferably achieved through various constraint conditions such as the epipolar constraint of a stereo vision system and image similarity in the practice.
The similarity matrix C is calculated in the term of similarity of properties such as an image area, an image major axis, and an image minor axis, and information such as the epipolar constraint of a stereo vision system which is known to those skilled in the art may be taken into consideration.
After obtaining the matrix C, if $C = [\begin{matrix} A & 0 \\ 0 & B \end{matrix}],$
the matrixes A and B are calculated through known methods such as the Hungary algorithm and the Branch-and-Bound Method.
During the calculation, the precise position of the laser light spot is determined according to the result of the matching.
Based on the obtained sequence number and low-precision depth of the laser light spot, the image position of the light spot in the second camera 22 is obtained, and a precise position of the light spot may be obtained through an interpolating method according to the second mapping relationship between the image position of a light spot within the second camera 22 and the sequence number as well as the high-precision depth of the light spot.
As shown in Figures 6 and 7, the distance between the second camera 22 and the laser output port 31 is larger than that between the first camera 21 and the laser output port 31, and the laser output port 31 is at a side of the first camera 21 that is opposite to the second camera 22, or arranged between the first and second camera 21 and 22. As shown in Figure 6, the laser output port is located at a side of the first camera 21 that is opposite to the second camera 22, and the distance between the second camera 22 and the laser output port 31 is larger than the distance between the first camera 21 and the laser output port 31. As shown in Figure 7, the laser output port 31 is arranged between the first and second camera 21 and 22, and the distance between the second camera 22 and the laser output port 31 is larger than the distance between the first camera 21 and the laser output port 31. In this way, a mapping relationship between the image position and the depth provided by the second camera 22 is more precise than that provided by the first camera 21, and the precision of the second camera 22 may be adjusted as desired actually. The position of the laser output port 31 may be adjusted as desired as long as the distance between the second camera 22 and the laser output port 31 is larger than the distance between the first camera 21 and the laser output port 31.
The embodiments of the present invention are described as above in conjunction with the drawings. However, the present invention is not limited to above detailed embodiments, since the full scope of protection of the present invention is only defined by the appended claims.

Claims

A structured-light based measuring method, comprising:
a demarcating process which includes, demarcating a first mapping relationship between an image position of each laser light spot numbered with sequence numbers within a first camera (21) and the sequence number as well as a low-precision depth of the laser light spot, demarcating a second mapping relationship between an image position of each laser light spot within a second camera (22) and the sequence number as well as a high-precision depth of the laser light spot, and storing the demarcated first and second mapping relationships in a memory to form a demarcation database for the use by a matching process and a calculating process;

the matching process, which includes obtaining the sequence number and the low-precision depth of the laser light spot based on an image position of the laser light spot within the first camera (21) according to the first mapping relationship in the demarcation database, searching for image positions of the laser light spot within the second camera (22) according to the sequence number and the low-precision depth of the laser light spot to obtain candidates of matching point, and conducting matching according to the image position of the laser light spot within the first camera (21) and the respective candidates of matching point within the second camera (22), to obtain a result of the matching; and

the calculating process, which includes obtaining an image position within the second camera (22) that matches with the image position within the first camera (21) according to the result of the matching, and determining the precise position of the laser light spot according to the second mapping relationship in the demarcation database;

wherein conducting matching according to the image position of the laser light spot within the first camera (21) and the respective candidates of matching point within the second camera (22) during the matching process comprises:
searching for a reference matching pair according to a luminance difference of images of the laser light spot; and

determining the optimal matching point using the reference matching pair;

wherein during the conducting matching according to the image position of the laser light spot within the first camera (21) and the respective candidates of matching point within the second camera (22) in the matching process, before searching for a reference matching pair according to a luminance difference of images of the laser light spot, the method further comprises:
conducting a 3D reconstruction of the candidates of matching point, to obtain a depth of each of the candidates of matching point; and

conducting initial selection among the candidates of matching point according to the depths of the candidates of matching point.
The method of claim 1, wherein during the demarcating process, a position of a laser output port (31) relative the first camera (21) is adjusted to prevent image positions of any two laser light spots within the first camera (21) from overlapping with each other.
The method of claim 2, wherein a distance between the second camera (22) and the laser output port (31) is larger than a distance between the first camera (21) and the laser output port (31).
The method of one of the preceding claims, wherein the demarcating process and the matching process are performed in a condition that image positions at different depths of the same laser light spot are surrounded by a geometric region.
The method of one of the preceding claims, wherein the precise position of the laser light spot is obtained by an interpolating method applied on the image position in the second camera (22) and the high-precision depth during the calculating process.
A structured-light based measuring system, comprising a processing system (1), an imaging system (2) and a projecting system (3), wherein the imaging system (2) comprises a first camera (21) and a second camera (22), the projecting system (3) comprises a laser generator for generating laser light, and the processing system (1) comprises a demarcating module, a matching module and a calculating module,
the demarcating module is adapted for demarcating a first mapping relationship between an image position of each laser light spot numbered with sequence numbers within the first camera (21) and the sequence number as well as a low-precision depth of the laser light spot, demarcating a second mapping relationship between an image position of each laser light spot within the second camera (22) and the sequence number as well as a high-precision depth of the laser light spot, and storing the demarcated first and second mapping relationships in a memory to form the demarcation database for the use by the matching module and the calculating module;
the matching module is adapted for obtaining the sequence number and the low-precision depth of the laser light spot based on an image position of the laser light spot within the first camera (21) according to the first mapping relationship in the demarcation database, searching for image positions of the laser light spot within the second camera (22) according to the sequence number and the low-precision depth of the laser light spot to obtain candidates of matching point, and conducting matching according to the image position of the laser light spot within the first camera (21) and the respective candidates of matching point within the second camera (22), to obtain a result of the matching, wherein conducting matching according to the image position of the laser light spot within the first camera (21) and the respective candidates of matching point within the second camera (22) by the matching module comprises:
searching for a reference matching pair according to a luminance difference of images of the laser light spot and determining the optimal matching point using the reference matching pair; and

the calculating module is adapted for obtaining an image position within the second camera (22) that matches with the image position within the first camera (21) according to the result of the matching, and determining a precise position of the laser light spot according to the second mapping relationship in the demarcation database;

whereby the matching module is further adapted for conducting a 3D reconstruction of the candidates of matching point, to obtain a depth of each of the candidates of matching point conducting initial selection among the candidates of matching point according to the depths of the candidates of matching point during the conducting matching according to the image position of the laser light spot within the first camera (21) and the respective candidates of matching point within the second camera (22) by the matching module, before searching for a reference matching pair according to a luminance difference of images of the laser light spot.
The system of claim 6, wherein the calculation module is adapted for obtaining the precise position of the laser light spot by an interpolating method applied on the image position in the second camera (22) and the high-precision depth during a calculating process.
The system of claim 6, wherein the system is adapted for adjusting during the demarcating process by the demarcating module, a position of a laser output port (31) relative the first camera (21) to prevent image positions of any two laser light spots within the first camera (21) from overlapping with each other.
The system of claim 8, wherein a distance between the second camera (22) and the laser output port (31) is larger than a distance between the first camera (21) and the laser output port (31).
The system of claim 8 or 9, wherein functions of the demarcating module and the matching module are implemented in a condition that image positions at different depths of the same laser light spot are surrounded by a geometric region.