WO2018201677A1 - Bundle adjustment-based calibration method and device for telecentric lens-containing three-dimensional imaging system - Google Patents

Abstract

Disclosed in the present invention are a bundle adjustment-based calibration method and device for a telecentric lens-containing three-dimensional imaging system. In the method, a camera apparatus having a telecentric lens and a projector apparatus having a telecentric lens are employed in a three-dimensional imaging system, a bundle adjustment technique is employed to perform non-linear calibration on the camera apparatus and projector apparatus, and calibration parameters of the calibrated camera apparatus and projector apparatus are employed to calibrate the three-dimensional imaging system. By employing the bundle adjustment technique to effectively reduce influences of errors of a target on calibration accuracy of the three-dimensional imaging system, the present invention effectively improves the calibration accuracy, and accordingly enhances precision of the three-dimensional imaging system.

Description

Calibration method and device for telecentric lens three-dimensional imaging system based on beam adjustment Technical field

The invention relates to the field of three-dimensional imaging and optical three-dimensional reconstruction, in particular to a calibration method and device for a telecentric lens three-dimensional imaging system based on beam adjustment.

Background technique

Three-dimensional imaging based on phase fringe coding is simple, fast in calculation and high in measurement accuracy, and is widely used in three-dimensional imaging and measurement of objects. The image containing the sinusoidal fringes is projected onto the object to be tested by the projection device, and the shape of the surface of the object causes the projected structured light image to be differently modulated, and the structured light image with the modulated information is acquired by the camera, combined with the mediation method and stereoscopic vision. The technology can obtain three-dimensional digital topography data of an object.

In the prior art, due to the limitations of the optical structure itself, the three-dimensional imaging and three-dimensional scanning technology still have a great room for the development of special objects such as metal reflectors, small and odd-shaped objects, and in addition to the system calibration, the object phase Research on the completeness and accuracy of information acquisition requires further deepening of relevant core issues such as 3D reconstruction methods and technologies.

At present, how to improve the accuracy of the three-dimensional imaging system is an urgent problem to be solved.

Summary of the invention

The main object of the present invention is to provide a calibration method and apparatus for a telecentric lens three-dimensional imaging system based on beam adjustment, which aims to solve the technical problem of low precision of the three-dimensional imaging system in the prior art.

To achieve the above object, a first aspect of the present invention provides a calibration method for a telecentric lens three-dimensional imaging system based on beam adjustment, the method being applied to a three-dimensional imaging system, the three-dimensional imaging system comprising: a telecentric lens Projection device, camera device having a telecentric lens, and a movable platform; the optical axis of the projection device is perpendicular to the movable platform placed horizontally, and the optical axis of the imaging device is preset with the movable platform An angle on which the target is placed on the movable platform, the movable platform The calibration method includes:

Step 1. Move the movable platform such that the target is in a plurality of different target postures, and in each target posture, project the target with uniform light, and collect the image through the imaging device. Delineating a target image under the target attitude, and projecting the fringe pattern to the target by using the projection device, and acquiring a fringe pattern under the target posture by using the imaging device;

Step 2: Initializing an internal parameter and an external parameter of the imaging device to obtain an internal reference and an external parameter after initializing the parameter of the imaging device, and using a target image and a plurality of target postures under different target postures according to the beam adjustment method Performing joint nonlinear optimization on the three-dimensional coordinate system parameter of the target and the initialized internal and external parameters to obtain calibration parameters of the imaging device to complete calibration of the imaging device;

Step 3: De-phase and phase-expand the fringe pattern acquired by a plurality of different target poses by using the N-step phase shift method to obtain a phase unwrapping diagram of a plurality of different target poses;

Step 4: Initializing an internal parameter and an external parameter of the projection device to obtain an internal parameter and an external parameter after initialization of the projection device, and using a phase adjustment diagram of the plurality of different target postures according to a beam adjustment method, The three-dimensional coordinate system parameter of the target and the initialized internal and external parameters are jointly nonlinearly optimized to obtain calibration parameters of the projection device to complete calibration of the projection device;

Step 5: Calibrate the three-dimensional imaging system by using calibration parameters of the imaging device and calibration parameters of the projection device, and obtain calibration parameters of the three-dimensional imaging system to complete calibration of the three-dimensional imaging system.

To achieve the above object, the present invention also provides a calibration device for a telecentric lens three-dimensional imaging system based on beam adjustment, the device being applied to a three-dimensional imaging system, the three-dimensional imaging system comprising: a projection device having a telecentric lens, An imaging device having a telecentric lens and a movable platform; the optical axis of the projection device is perpendicular to the movable platform placed horizontally, and an optical axis of the imaging device is at a preset angle with the movable platform. A target is placed on the movable platform, and the movable platform is always within a common depth of field of the imaging device and the projection device, and the calibration device includes:

An acquisition module for moving the movable platform such that the target is in a plurality of different target positions And in each target posture, using uniform light to project to the target, collecting the target image in the target posture by the imaging device, and projecting the fringe pattern to the projection device by using the projection device Using the imaging device to acquire a fringe pattern under the target posture;

An imaging device calibration module is configured to initialize an internal parameter and an external parameter of the imaging device to obtain an internal reference and an external parameter after the initialization of the parameter of the imaging device, and use a plurality of different target postures based on a beam adjustment method The target image, the three-dimensional coordinate system parameter of the target, and the initialized internal reference and the external parameter are jointly nonlinearly optimized to obtain calibration parameters of the imaging device to complete calibration of the imaging device;

The obtaining module is configured to dephase and phase unfold the fringe pattern acquired by the plurality of different target poses by using the N-step phase shift method, and obtain a phase unwrapping diagram of the plurality of different target poses;

a projection device calibration module, configured to initialize an internal parameter and an external parameter of the projection device, to obtain an internal reference and an external parameter after the initialization of the projection device, and use the plurality of different target postures according to a beam adjustment method a phase unwrapping map, a three-dimensional coordinate system parameter of the target, and a joint non-linear optimization of the initialized internal and external parameters to obtain calibration parameters of the projection device to complete calibration of the projection device;

a system calibration module, configured to calibrate the three-dimensional imaging system by using calibration parameters of the imaging device and calibration parameters of the projection device, to obtain calibration parameters of the three-dimensional imaging system, to complete calibration of the three-dimensional imaging system .

The invention provides a calibration method for a telecentric lens three-dimensional imaging system based on beam adjustment, in which a camera device with a telecentric lens and a projection device with a telecentric lens are used in a three-dimensional imaging system, and the camera is imaged by a beam adjustment method. The device and the projection device are non-linearly calibrated, and the three-dimensional imaging system is calibrated by using the calibration parameters of the calibrated imaging device and the projection device. By using the beam adjustment method, the influence of the target error on the calibration accuracy of the three-dimensional imaging system can be effectively reduced, the calibration accuracy is effectively improved, and the accuracy of the three-dimensional imaging system is improved.

DRAWINGS

In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings used in the embodiments or the description of the prior art will be briefly described below. Obviously, the drawings in the following description are only It is a certain embodiment of the present invention, and those skilled in the art can obtain other drawings according to these drawings without any creative work.

1 is a schematic structural view of a telecentric lens three-dimensional imaging system based on beam adjustment according to an embodiment of the present invention;

Figure 1.1 is a schematic diagram of a double telecentric lens in an embodiment of the present invention;

Figure 1.2 is a schematic diagram of a coordinate system established in an embodiment of the present invention;

2 is a schematic flow chart of a calibration method of a three-dimensional imaging system of a telecentric lens based on beam adjustment according to an embodiment of the present invention;

3 is a schematic diagram of functional modules of a calibration device of a three-dimensional imaging system of a telecentric lens based on beam adjustment in an embodiment of the present invention.

detailed description

The technical solutions in the embodiments of the present invention will be clearly and completely described in conjunction with the drawings in the embodiments of the present invention. The embodiments are merely a part of the embodiments of the invention, and not all of the embodiments. All other embodiments obtained by a person skilled in the art based on the embodiments of the present invention without creative efforts are within the scope of the present invention.

For a better understanding of the technical solution in the embodiments of the present invention, please refer to FIG. 1 , which is a schematic structural diagram of a three-dimensional imaging system for a telecentric lens based on beam adjustment according to an embodiment of the present invention, the three-dimensional imaging system includes: a telecentric lens Projection equipment, camera equipment with telecentric lenses, and a movable platform.

The optical axis of the projection device is perpendicular to the horizontally movable platform, and the optical axis of the imaging device is at a predetermined angle with the movable platform, and the preset angle may be in a range of 70 to 85 degrees. among them, The movable platform can be placed on a horizontal test bench, and the target can be placed on the movable platform.

Wherein, the movable platform can move up and down and move left and right, and can also adjust the angle of the movable platform to the horizontal plane, wherein by moving the movable platform, the target can be placed in different target postures.

Among them, the telecentric lens can use a double telecentric lens, the projection device can use a DMD projector, and the camera device can use a CMOS device or a digital camera.

It should be noted that, in order to achieve calibration, the movable platform is always within the common depth of field of the imaging device and the projection device, so that the image projected by the projection device onto the target is clear, and the image captured by the imaging device is also clear.

In order to better understand the technical solution in the embodiment of the present invention, the related content of the telecentric lens will be described below.

Compared with the existing pinhole model lens, the telecentric lens has high measurement accuracy and is suitable for high-precision measurement. The telecentric lens can be divided into an object telecentric lens, an image telecentric lens, and a double telecentric lens. Taking a double telecentric lens as an example, please refer to FIG. 1.1, which is the principle of the double telecentric lens in the embodiment of the present invention. Figure.

The schematic diagram can also be called a telecentric lens model. From the perspective of the model, the telecentric lens model is a parallel imaging model. In the depth of field of the telecentric lens, the distance between the object and the lens does not change. Affecting the magnification of the image by the camera device, in order to better describe the telecentric lens model, a coordinate system as shown in Fig. 1.2 is established, wherein O _w (X _w , Y _w , Z _w ) is the world coordinate system as a unified coordinate system. Describe the relative position of all objects in the space. The camera coordinate system is O _c (X _c Y _c Z _c ). Usually, the optical axis of the camera is selected as the Z _{c of the} camera coordinate system. The coordinate conversion relationship is similar to the pinhole model. The camera coordinate system is a coordinate system based on the imaging plane. Generally, the optical axis is taken as the coordinate system Z axis, and the points on the world coordinate system correspond to the points on the camera coordinate system. . The image coordinates may also be referred to as pixel coordinates, belonging to the coordinates in the pixel coordinate system, and the pixel coordinate system refers to the arrangement coordinate system of the pixel points on the pixel surface of the camera. Usually, the pixel point in the upper left corner of the image is taken as the coordinate origin, and the abscissa is represented. The number of rows of pixels, the ordinate indicates the number of columns.

Since telecentric lens imaging is parallel projection imaging, the camera coordinate system loses the Z-axis during imaging. The above information, therefore, telecentric lens imaging can be understood as two-dimensional coordinate information projected onto the imaging surface of the imaging device at a certain magnification.

Assuming that the lens magnification is m here, the camera coordinates and image coordinates are as follows:

即

which is

Due to the above imaging relationship, considering the internal and external parameters of the telecentric lens, since the external parameter is a parameter describing the relationship between the positional relationship between the world coordinates and the image coordinates, and because the external parameters of the telecentric lens lack the parameter variation on the Z axis, Information, therefore, the initial conversion relationship between the telecentric lens world coordinates and the image coordinates can be established based on the magnification m of the telecentric lens. Therefore, the telecentric lens imaging model is as follows:

Where m denotes the magnification of the telecentric lens, s denotes the distortion factor of the telecentric lens, (u ₀ , v ₀ ) denotes the origin of the pixel coordinate system, r, t denotes the rotation matrix and the translation matrix in the outer parameter, respectively, d _u , d _v are the physical dimensions of each pixel in the X-axis and Y-axis directions, respectively.

It should be noted that the external parameter is obtained by combining the known initialization internal parameter and the normalization matrix. Here, it is assumed that h _ij represents the number of the i-th row and the j-th column in the normalized matrix H, according to the above formula ( 3.1) The normalized matrix formula (3.2) can be obtained:

Similarly, this normalization function H can be directly solved by the DLT method. According to R, the unit is orthogonal to the matrix, so you can get:

The formula (3.4) can be obtained by the simultaneous formula:

Assuming that m/d _u =m/d _v =a from equation (3.1), combining Equation 3.1 with Equation 3.4 above, the following formula can be obtained:

这里的i＝1,2且j＝1,2.，这样就可以得到a的解，即求得R矩阵中左上的一个2×2的旋转矩阵。There are two non-negative solutions in the above equation (a ² is the above solution), so there is a need to have a condition to solve. Basic conditions can be obtained from equations (3.1) and (3.2).

Here i = 1, 2 and j = 1, 2. Thus, the solution of a can be obtained, that is, a 2 × 2 rotation matrix in the upper left of the R matrix is obtained.

In the initialization process, the values of r ₁₃ and r ₂₃ are required to be obtained by the point multiplication relationship of the formula (3.1) and the formula (3.2). The displacement parameter t _s and the image center point (u ₀ , v ₀ ) can be satisfied. The following formula:

In the above formula (u ₀ , v ₀ ) and t _s are unknown variables, so they cannot be solved directly by (3.6). This step is not the same as the solution of the pinhole model. The reason for this phenomenon is the normalization function. The change in the coordinate of H in the Z-axis direction is meaningless because the telecentric lens has no focus (or is understood to be the focus at infinity), so h ₁₃ , h ₂₃ are numerically zero. Therefore, the position of the image plane can be arbitrarily set within the depth of field of the telecentric lens, and it is assumed here that the pixel point at the center of the image is regarded as the center of the image, so that (u ₀ , v ₀ ) can be determined by known CCD information, thereby The translation matrix t _s in equation (3.6) can be determined.

In addition, the above description of the 2x2 rotation matrix in the upper left of the rotation matrix R is already available. Then the remaining matrix parameters can be obtained by the combination of equations (3.3) and (3.7) using R as the unit orthogonal array.

r ₃ =r ₁ ×r ₂ (3.7)

Where r ₁ , r ₂ , and r ₃ are the coordinate vectors of the R matrix, because the unit orthogonal matrix feature of R can only provide a constraint using a circular marker point target, taking any point in the marker point as a control point. And to ensure that the target has a slight displacement along the Z-axis through the displacement platform, and r ₁₃ and r ₂₃ can also be obtained by using equation (3.8).

Therefore, in the above manner, the telecentric lens imaging model of the telecentric lens can be obtained, and the initialization of the external parameters and the internal parameters of the imaging device and the projection device can be realized based on the telecentric lens imaging model.

Further, considering the lens distortion, the following formula can be obtained:

(3.9)

在公式中k₁～k₅均是畸变系数，其中(k₁,k₂,k₅)表示三个方向上的不同径向参数，(k₃,k₄)表示切向上的畸变参数。利用式(3.9)能够确定畸变系数。Where u and v represent points without distortion in the ideal coordinate system, while u' and v' represent image points with distortion and satisfy

In the formula, k ₁ to k ₅ are distortion coefficients, where (k ₁ , k ₂ , k ₅ ) represent different radial parameters in three directions, and (k ₃ , k ₄ ) represents a distortion parameter in the tangential direction. The distortion coefficient can be determined using equation (3.9).

Based on the above three-dimensional imaging system, please refer to FIG. 2, which is a schematic flowchart of a calibration method of a telecentric lens three-dimensional imaging system based on beam adjustment according to a first embodiment of the present invention, including:

Step 201: Move the movable platform such that the target is in a plurality of different target postures, and in each target posture, use uniform light to project to the target, and collect the image through the imaging device. Delineating a target image under the target attitude, and projecting the fringe pattern to the target by using the projection device, and acquiring a fringe pattern under the target posture by using the imaging device;

In the embodiment of the present invention, it is required to collect target images and fringe patterns in a plurality of different target postures. And after the movable platform moves to a certain target attitude, the uniform light is emitted by the uniform light source, and the uniform light is projected onto the target, and the target image under the target posture is acquired by the imaging device. After the target image is acquired in the target posture, the target posture is kept unchanged, the fringe pattern is projected onto the target by using the projection device, and the fringe pattern under the target posture is acquired by using the imaging device. . Pass In the above manner, the target image and the fringe pattern in a certain target posture can be collected, and the target image in a plurality of different target postures can be obtained by moving the movable platform to different target postures. And striped pattern.

It should be noted that a plurality of fringe patterns have been saved in the projection device, and the plurality of fringe patterns can use four sinusoidal fringe patterns with a fringe period of 20 and nine fringe patterns in two directions. And implanting the projection device such that the projection device can project the sine fringe pattern and the Gray code fringe pattern to the target. Specifically, in each target pose, the projection device will project the sinusoidal fringe pattern and the Gray code fringe pattern onto the target in sequence, and the imaging device will also perform a collection after each fringe pattern is projected. And collecting the plurality of fringe patterns as the fringe pattern acquired under the target posture.

It should be noted that, in the embodiment of the present invention, the movable platform is always within the depth of field of the imaging device and the projection device regardless of the change of the target posture.

It can be understood that in the embodiment of the present invention, the target image and the fringe pattern need to be collected in at least three different target poses to complete the collection work.

It can be understood that the angle between the uniform light source and the movable platform in the embodiment of the present invention does not limit its size as long as there is uniform light projected onto the target on the movable platform, and the image of the projected light is made. Clear, and the target image captured by the camera device is clear.

Step 202: Initialize an internal parameter and an external parameter of the imaging device to obtain an internal parameter and an external parameter after initializing the parameter of the imaging device, and use a target image and a plurality of target positions under different target postures according to the beam adjustment method. Performing joint nonlinear optimization on the three-dimensional coordinate system parameter of the target and the initialized internal and external parameters to obtain calibration parameters of the imaging device to complete calibration of the imaging device;

In the embodiment of the present invention, the imaging device is calibrated based on the beam adjustment method, and the calibration parameters of the imaging device may be obtained according to the following formula:

表示在第i个标靶姿态下的标靶图像中的第j个标志点的成像点；O^ij(R,t_s,m,u₀,v₀,k,X)表示第i个标靶姿态下的标靶图像中的第j个标志点在像素坐标系中的理想点；X表示所述标靶的三维坐标系参数，R为所述摄像设备外参的旋转矩阵，及t_s为所述摄像设备外参的平移矩阵，m为所述摄像设备内参中远心镜头的放大率，(u₀,v₀)表示所述像素坐标系的原点，k为畸变系数。Where x _BA ^* represents the re-projection error residual value, N represents the total number of target images in a plurality of different target poses, and M represents the number of identified points in the target,

An imaging point representing the jth landmark in the target image at the i-th target pose; O ^ij (R, t _s , m, u ₀ , v ₀ , k, X) represents the ith target The ideal point of the jth marker point in the target image in the pixel coordinate system; X represents the three-dimensional coordinate system parameter of the target, R is the rotation matrix of the external parameter of the imaging device, and t _s is a translation matrix of the external reference of the imaging device, m is a magnification of the telecentric lens in the internal reference of the imaging device, (u ₀ , v ₀ ) represents an origin of the pixel coordinate system, and k is a distortion coefficient.

在非线性优化中尽可能的逼近标志点的O^ij(R,t_s,m,u₀,v₀,k,X)。Where the re-projection error residual value is as small as possible, so that

In the nonlinear optimization, the O ^ij (R, t _s , m, u ₀ , v ₀ , k, X) of the landmarks are approximated as much as possible.

Among them, the beam adjustment method is a joint nonlinear optimization of the three-dimensional image parameters of the target and the external and internal parameters of the imaging device.

Among them, in the initialization, the initial value of m is obtained after the camera device is built; the external parameter R, t _s is initialized by the parameter, and R usually obtains the partial derivative of R and then substitutes the optimization by Rodrigues`s law; k is Distortion coefficient, where the initial value of the distortion parameter is set to 0 for the first optimization, and then the second optimization is used to obtain the optimal solution; the initial value of (u ₀ , v ₀ ) is determined by the camera resolution. For this nonlinear optimization problem, it can be solved by the Levbery-Marquardt method.

It can be understood that the beam adjustment method is a nonlinear optimization method. The problem of solving the above beam adjustment method becomes the problem of the optimal solution of the Jacobian matrix nonlinear optimization. In the calibration of the camera equipment, The target contains M marker points, and the target image and the fringe pattern of the N different target poses are collected. The dimensions of the Jacobian matrix are:

4MN×(26+6N+3M)

Due to the particularity of the telecentric lens, the partial derivative of the function to the unrelated variable is 0. As can be seen from the above equation, the Jacobian matrix is large. In order to improve the computational efficiency, reduce the calibration time and introduce the properties of the sparse matrix, and then partition the Jacobian matrix. The operation is performed so that the beam adjustment method can be solved and the corresponding calibration parameters are obtained.

In the embodiment of the present invention, after the calibration of the imaging device is completed, the calibration of the imaging device can be obtained. The parameter, and the calibration parameter includes at least pixel coordinates of the marker point in the target of the imaging device, the magnification of the optimized imaging device, and the optimized rotation matrix and translation matrix, and the internal parameters of the imaging device.

Step 203: Dephase and phase unfold the fringe pattern acquired by the plurality of different target poses by using the N-step phase shift method, and obtain a phase unwrapping diagram of the plurality of different target poses;

In the embodiment of the present invention, for the stripe maps of the plurality of different target poses collected, the N-step phase method is used to dephase and phase unroll the acquired fringe patterns in the plurality of different target poses. Obtaining a phase unwrapped graph of the plurality of different target poses. Among them, for the fringe pattern, the phase folding map can be obtained by the phase unwrapping technique, and the phase unwrapping graph can be obtained by the phase unwrapping technique.

It can be understood that the N-step phase shift method can be a four-step phase shift method or other multi-step phase shift method.

Step 204: Initialize an internal parameter and an external parameter of the projection device to obtain an internal parameter and an external parameter after the initialization of the projection device, and use a phase unwrapping diagram of the plurality of different target postures according to a beam adjustment method. The three-dimensional coordinate system parameter of the target and the initialized internal and external parameters are jointly nonlinearly optimized to obtain calibration parameters of the projection device to complete calibration of the projection device;

In the embodiment of the present invention, the calibration parameters of the projection device obtained by using the beam adjustment method are as follows:

表示在第i个标靶姿态下的相位展开图中的第j个标志点的成像点；O^ij(R,t_s,m,u₀,v₀,k,X)表示第i个标靶姿态下的相位展开图中的第j个标志点在像素坐标系中的理想点；X表示所述标靶的三维坐标系参数，R为所述投影设备外参的旋转矩阵，及t_s为所述投影设备外参的平移矩阵，m为所述投影设备内参中远心镜头的放大率，(u₀,v₀)表示所述像素坐标系的原点，k为畸变系数。Where x _BA ^* represents the reprojection error residual value, N represents the total number of phase unwrapped graphs in a plurality of different target poses, and M represents the number of identified points in the target,

An imaging point representing the jth landmark point in the phase unwrapped map at the i-th target pose; O ^ij (R, t _s , m, u ₀ , v ₀ , k, X) represents the ith target The ideal point of the jth marker point in the phase unwrapped graph in the pixel coordinate system; X represents the three-dimensional coordinate system parameter of the target, R is the rotation matrix of the external reference of the projection device, and t _s is a translation matrix of the external reference of the projection device, m is a magnification of the telecentric lens in the internal reference of the projection device, (u ₀ , v ₀ ) represents an origin of the pixel coordinate system, and k is a distortion coefficient.

In the embodiment of the present invention, the marker point of the target can be obtained by the phase unwrapping map on the projection device. The pixel coordinates, ie the inverse "imaging image" of the projection device, results in a virtual "captured image" of the projection device.

In the embodiment of the present invention, the pixel coordinates of the target marker point recorded by the projection device can be obtained by the beam adjustment method described above, and the pixel coordinates and the phase expansion map of the target marker point recorded by the projection device are obtained. Corresponding relationship between pixel coordinates of each marking point of the device to complete calibration of the projection device, wherein the correspondence relationship belongs to one of calibration parameters of the projection device, and further, the calibration parameter of the projection device further includes an optimized amplification of the projection device Rate, optimized rotation matrix and translation matrix.

Step 205: Calibrate the three-dimensional imaging system by using calibration parameters of the imaging device and calibration parameters of the projection device to obtain calibration parameters of the three-dimensional imaging system to complete calibration of the three-dimensional imaging system.

In the embodiment of the present invention, the calibration of the three-dimensional imaging system by using the calibration parameters of the imaging camera device and the calibration parameters of the projection device includes:

Step A: Obtain calibration parameters of the three-dimensional imaging system according to the following formula:

R ^p ·p _w +t ^p =R ^s ·(R ^c ·p _w +t ^c )+t ^s

Wherein, R ^p represents a rotation matrix in the external reference of the projection device, t ^p represents a translation matrix in the external reference of the projection device, R ^c represents a rotation matrix in the external reference of the imaging device, and t ^c represents the imaging The translation matrix in the external reference of the device, R ^s represents the rotation matrix of the three-dimensional imaging system, t ^s represents the translation matrix of the three-dimensional imaging system, and p _w represents the world coordinate vector of any point P in the space;

Where P _w = [x _w , y _w , z _w ] ^T ;

Step B: performing least squares optimization on the rotation matrix and the translation matrix of the three-dimensional imaging system, and determining the conversion relationship between the pixel coordinates and the world coordinates of the three-dimensional imaging system by using the optimized rotation matrix and the translation matrix.

Wherein, the pixel coordinate is a coordinate in the image coordinate system, and the image coordinate system may also be referred to as a camera coordinate system. According to the coordinate system defined by the imaging plane, the optical axis is generally taken as the coordinate system Z axis, and the point on the world coordinate system may be One-to-one correspondence with points on the image coordinate system.

In an embodiment of the present invention, an imaging apparatus having a telecentric lens and a tool are used in a three-dimensional imaging system A projection device having a telecentric lens, and nonlinearly calibrating the imaging device and the projection device by the beam adjustment method, and calibrating the three-dimensional imaging system by using the calibration parameters of the calibrated imaging device and the projection device. By using the beam adjustment method, the influence of the target error on the calibration accuracy of the three-dimensional imaging system can be effectively reduced, the calibration accuracy is effectively improved, and the accuracy of the three-dimensional imaging system is improved. Further, by using an imaging device having a telecentric lens and an imaging device having a telecentric lens, since the telecentric lens can perform precise measurement, it is particularly suitable for measurement of an object having a certain depth and thickness and having a radius of a stepped hole. Therefore, the calibration accuracy and measurement accuracy can be further improved.

3 is a schematic diagram of functional modules of a calibration apparatus for a telecentric lens three-dimensional imaging system based on beam adjustment according to a second embodiment of the present invention, wherein the apparatus is applied to a three-dimensional imaging system, and the three-dimensional imaging system includes: a projection apparatus having a telecentric lens, an imaging apparatus having a telecentric lens, and a movable platform; an optical axis of the projection apparatus being perpendicular to the movable platform placed horizontally, an optical axis of the imaging apparatus and the movable The platform is at a preset angle, and the target is placed on the movable platform, and the movable platform is always in a common depth of field of the imaging device and the projection device, and the calibration device includes:

An acquisition module 301, configured to move the movable platform such that the target is in a plurality of different target postures, and in each target posture, the uniform light is projected onto the target through the imaging The device collects a target image in the target posture, and projects a fringe pattern to the target by using the projection device, and uses the imaging device to collect a fringe pattern under the target posture;

The imaging device calibration module 302 is configured to initialize an internal parameter and an external parameter of the imaging device to obtain an internal reference and an external parameter after the initialization of the parameter of the imaging device, and use a plurality of different target postures according to a beam adjustment method. The target image, the three-dimensional coordinate system parameter of the target, and the internal parameter and the external parameter after the initialization are jointly nonlinearly optimized, and the calibration parameters of the imaging device are obtained to complete calibration of the imaging device;

The obtaining module 303 is configured to perform phase cancellation and phase unwrapping of the fringe patterns collected by the plurality of different target poses by using the N-step phase shift method, and obtain phase unwrapping diagrams of the plurality of different target poses;

a projection device calibration module 304, configured to initialize an internal parameter and an external parameter of the projection device, to obtain an internal reference and an external parameter after the initialization of the projection device, and use the plurality of different target postures according to a beam adjustment method a phase unwrapping diagram, a three-dimensional coordinate system parameter of the target, and a joint nonlinear optimization of the initialized internal and external parameters to obtain calibration parameters of the projection device to complete calibration of the projection device;

a system calibration module 305, configured to calibrate the three-dimensional imaging system by using calibration parameters of the imaging device and calibration parameters of the projection device, to obtain calibration parameters of the three-dimensional imaging system, to complete the three-dimensional imaging system Calibration.

Further, the imaging device calibration module 302 is specifically configured to initialize an internal parameter and an external parameter of the imaging device, obtain an internal parameter and an external parameter after the initialization of the parameter of the imaging device, and obtain the imaging device according to the following formula. Calibration parameters:

表示在第i个标靶姿态下的标靶图像中的第j个标志点的成像点；O^ij(R,t_s,m,u₀,v₀,k,X)表示第i个标靶姿态下的标靶图像中的第j个标志点在像素坐标系中的理想点；X表示所述标靶的三维坐标系参数，R为所述摄像设备外参的旋转矩阵，及t_s为所述摄像设备外参的平移矩阵，m为所述摄像设备内参中远心镜头的放大率，(u₀,v₀)表示所述像素坐标系的原点，k为畸变系数。Where x _BA ^* represents the re-projection error residual value, N represents the total number of target images in a plurality of different target poses, and M represents the number of identified points in the target,

An imaging point representing the jth landmark in the target image at the i-th target pose; O ^ij (R, t _s , m, u ₀ , v ₀ , k, X) represents the ith target The ideal point of the jth marker point in the target image in the pixel coordinate system; X represents the three-dimensional coordinate system parameter of the target, R is the rotation matrix of the external parameter of the imaging device, and t _s is a translation matrix of the external reference of the imaging device, m is a magnification of the telecentric lens in the internal reference of the imaging device, (u ₀ , v ₀ ) represents an origin of the pixel coordinate system, and k is a distortion coefficient.

Further, the projection device calibration module 304 is specifically configured to initialize an internal parameter and an external parameter of the projection device, obtain an internal parameter and an external parameter after the initialization of the projection device, and obtain a calibration of the projection device according to the following formula. parameter:

表示在第i个标靶姿态下的相 位展开图中的第j个标志点的成像点；O^ij(R,t_s,m,u₀,v₀,k,X)表示第i个标靶姿态下的相位展开图中的第j个标志点在像素坐标系中的理想点；X表示所述标靶的三维坐标系参数，R为所述投影设备外参的旋转矩阵，及t_s为所述投影设备外参的平移矩阵，m为所述投影设备内参中远心镜头的放大率，(u₀,v₀)表示所述像素坐标系的原点，k为畸变系数。Where x _BA ^* represents the reprojection error residual value, N represents the total number of phase unwrapped graphs in a plurality of different target poses, and M represents the number of identified points in the target,

An imaging point representing the jth marker point in the phase expansion map at the i-th target pose; O ^ij (R, t _s , m, u ₀ , v ₀ , k, X) represents the ith target The ideal point of the jth marker point in the phase unwrapped graph in the pixel coordinate system; X represents the three-dimensional coordinate system parameter of the target, R is the rotation matrix of the external reference of the projection device, and t _s is a translation matrix of the external reference of the projection device, m is a magnification of the telecentric lens in the internal reference of the projection device, (u ₀ , v ₀ ) represents an origin of the pixel coordinate system, and k is a distortion coefficient.

Further, the system calibration module 305 is specifically configured to:

The calibration parameters of the three-dimensional imaging system are obtained according to the following formula:

R ^p ·p _w +t ^p =R ^s ·(R ^c ·p _w +t ^c )+t ^s

Wherein, R ^p represents a rotation matrix in the external reference of the projection device, t ^p represents a translation matrix in the external reference of the projection device, R ^c represents a rotation matrix in the external reference of the imaging device, and t ^c represents the imaging The translation matrix in the external reference of the device, R ^s represents the rotation matrix of the three-dimensional imaging system, t ^s represents the translation matrix of the three-dimensional imaging system, and p _w represents the world coordinate vector of any point P in the space;

Where P _w = [x _w , y _w , z _w ] ^T ;

And performing least squares optimization on the rotation matrix and the translation matrix of the three-dimensional imaging system, and determining the conversion relationship between the pixel coordinates and the world coordinates of the three-dimensional imaging system by using the optimized rotation matrix and the translation matrix.

In the embodiment of the present invention, an imaging device having a telecentric lens and a projection device having a telecentric lens are used in the three-dimensional imaging system, and the imaging device and the projection device are nonlinearly calibrated by the beam adjustment method, and the calibration is completed by using the beam calibration method. The calibration parameters of the subsequent imaging device and projection device are calibrated to the three-dimensional imaging system. By using the beam adjustment method, the influence of the target error on the calibration accuracy of the three-dimensional imaging system can be effectively reduced, the calibration accuracy is effectively improved, and the accuracy of the three-dimensional imaging system is improved. Further, by using an imaging device having a telecentric lens and an imaging device having a telecentric lens, since the telecentric lens can perform precise measurement, it is particularly suitable for measurement of an object having a certain depth and thickness and having a radius of a stepped hole. Therefore, the calibration accuracy and measurement accuracy can be further improved.

In the several embodiments provided by the present application, it should be understood that the disclosed apparatus and method can be In other ways. For example, the device embodiments described above are merely illustrative. For example, the division of the modules is only a logical function division. In actual implementation, there may be another division manner, for example, multiple modules or components may be combined or Can be integrated into another system, or some features can be ignored or not executed. In addition, the mutual coupling or direct coupling or communication connection shown or discussed may be an indirect coupling or communication connection through some interface, device or module, and may be electrical, mechanical or otherwise.

The modules described as separate components may or may not be physically separated. The components displayed as modules may or may not be physical modules, that is, may be located in one place, or may be distributed to multiple network modules. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of the embodiment.

In addition, each functional module in each embodiment of the present invention may be integrated into one processing module, or each module may exist physically separately, or two or more modules may be integrated into one module. The above integrated modules can be implemented in the form of hardware or in the form of software functional modules.

The integrated modules, if implemented in the form of software functional modules and sold or used as separate products, may be stored in a computer readable storage medium. Based on such understanding, the technical solution of the present invention, which is essential or contributes to the prior art, or all or part of the technical solution, may be embodied in the form of a software product stored in a storage medium. A number of instructions are included to cause a computer device (which may be a personal computer, server, or network device, etc.) to perform all or part of the steps of the methods described in various embodiments of the present invention. The foregoing storage medium includes: a U disk, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disk, and the like. .

It should be noted that, for the foregoing method embodiments, for the sake of brevity, they are all described as a series of action combinations, but those skilled in the art should understand that the present invention is not described. The sequence of actions is limited because certain steps may be performed in other sequences or concurrently in accordance with the present invention. In the following, those skilled in the art should also understand that the embodiments described in the specification are all preferred embodiments, and the actions and modules involved are not necessarily required by the present invention.

In the above embodiments, the descriptions of the various embodiments are all focused, and the parts that are not detailed in a certain embodiment can be referred to the related descriptions of other embodiments.

The above is a description of a method and a device for calibrating a telecentric lens three-dimensional imaging system based on the beam adjustment provided by the present invention. For those skilled in the art, according to the idea of the embodiment of the present invention, the specific implementation manner and the application range There is a change in the above, and in summary, the contents of this specification should not be construed as limiting the invention.

Claims (8)

A calibration method for a telecentric lens three-dimensional imaging system based on beam adjustment, wherein the method is applied to a three-dimensional imaging system, the three-dimensional imaging system comprising: a projection device having a telecentric lens, and a telecentric lens An imaging device and a movable platform; the optical axis of the projection device is perpendicular to the movable platform placed horizontally, and an optical axis of the imaging device is at an initial angle with the movable platform, and the movable platform is on the movable platform Positioning the target, the movable platform is always within a common depth of field of the imaging device and the projection device, and the calibration method includes: Step 1. Move the movable platform such that the target is in a plurality of different target postures, and in each target posture, project the target with uniform light, and collect the image through the imaging device. Delineating a target image under the target attitude, and projecting the fringe pattern to the target by using the projection device, and acquiring a fringe pattern under the target posture by using the imaging device; Step 2: Initializing an internal parameter and an external parameter of the imaging device to obtain an internal reference and an external parameter after initializing the parameter of the imaging device, and using a target image and a plurality of target postures under different target postures according to the beam adjustment method Performing joint nonlinear optimization on the three-dimensional coordinate system parameter of the target and the initialized internal and external parameters to obtain calibration parameters of the imaging device to complete calibration of the imaging device; Step 3: De-phase and phase-expand the fringe pattern acquired by a plurality of different target poses by using the N-step phase shift method to obtain a phase unwrapping diagram of a plurality of different target poses; Step 4: Initializing an internal parameter and an external parameter of the projection device to obtain an internal parameter and an external parameter after initialization of the projection device, and using a phase adjustment diagram of the plurality of different target postures according to a beam adjustment method, The three-dimensional coordinate system parameter of the target and the initialized internal and external parameters are jointly nonlinearly optimized to obtain calibration parameters of the projection device to complete calibration of the projection device; Step 5: Calibrate the three-dimensional imaging system by using calibration parameters of the imaging device and calibration parameters of the projection device, and obtain calibration parameters of the three-dimensional imaging system to complete calibration of the three-dimensional imaging system. The method according to claim 1, wherein in the step 2, the light is based The beam adjustment method uses the target image in a plurality of different target poses, the three-dimensional coordinate system parameter of the target, and the internal reference and the external reference after the initialization to perform joint nonlinear optimization to obtain calibration of the imaging device. Parameters, including: The calibration parameters of the camera device are obtained according to the following formula:

Where x _BA ^* represents the re-projection error residual value, N represents the total number of target images in a plurality of different target poses, and M represents the number of identified points in the target,

An imaging point representing the jth landmark in the target image at the i-th target pose; O ^ij (R, t _s , m, u ₀ , v ₀ , k, X) represents the ith target The ideal point of the jth marker point in the target image in the pixel coordinate system; X represents the three-dimensional coordinate system parameter of the target, R is the rotation matrix of the external parameter of the imaging device, and t _s is a translation matrix of the external reference of the imaging device, m is a magnification of the telecentric lens in the internal reference of the imaging device, (u ₀ , v ₀ ) represents an origin of the pixel coordinate system, and k is a distortion coefficient. The method according to claim 1, wherein in step 4, based on a beam adjustment method, a phase unwrapping map of the plurality of different target poses, a three-dimensional coordinate system parameter of the target, and a The internal parameter and the external parameter after initialization are jointly nonlinearly optimized, and the calibration parameters of the projection device are obtained, including: The calibration parameters of the projection device are obtained according to the following formula:

Where x _BA ^* represents the reprojection error residual value, N represents the total number of phase unwrapped graphs in a plurality of different target poses, and M represents the number of identified points in the target,

An imaging point representing the jth landmark point in the phase unwrapped map at the i-th target pose; O ^ij (R, t _s , m, u ₀ , v ₀ , k, X) represents the ith target The ideal point of the jth marker point in the phase unwrapped graph in the pixel coordinate system; X represents the three-dimensional coordinate system parameter of the target, R is the rotation matrix of the external reference of the projection device, and t _s is a translation matrix of the external reference of the projection device, m is a magnification of the telecentric lens in the internal reference of the projection device, (u ₀ , v ₀ ) represents an origin of the pixel coordinate system, and k is a distortion coefficient. The method according to claim 1, wherein the step 5 specifically comprises: Step 51: Obtain calibration parameters of the three-dimensional imaging system according to the following formula: R ^p ·p _w +t ^p =R ^s ·(R ^c ·p _w +t ^c )+t ^s Wherein, R ^p represents a rotation matrix in the external reference of the projection device, t ^p represents a translation matrix in the external reference of the projection device, R ^c represents a rotation matrix in the external reference of the imaging device, and t ^c represents the imaging The translation matrix in the external reference of the device, R ^s represents the rotation matrix of the three-dimensional imaging system, t ^s represents the translation matrix of the three-dimensional imaging system, and p _w represents the world coordinate vector of any point P in the space; Where P _w = [x _w , y _w , z _w ] ^T ; Step 52: Perform least squares optimization on the rotation matrix and the translation matrix of the three-dimensional imaging system, and determine the conversion relationship between the pixel coordinates and the world coordinates of the three-dimensional imaging system by using the optimized rotation matrix and the translation matrix. A calibration device for a telecentric lens three-dimensional imaging system based on beam adjustment, wherein the device is applied to a three-dimensional imaging system, the three-dimensional imaging system comprising: a projection device having a telecentric lens, and a telecentric lens An imaging device and a movable platform; the optical axis of the projection device is perpendicular to the movable platform placed horizontally, and an optical axis of the imaging device is at an initial angle with the movable platform, and the movable platform is on the movable platform Positioning a target, the movable platform is always within a common depth of field of the imaging device and the projection device, and the calibration device comprises: An acquisition module for moving the movable platform such that the target is in a plurality of different target poses, and in each target pose, is projected onto the target with uniform light, through the imaging device Collecting a target image under the target attitude, and projecting a fringe pattern to the target by using the projection device, and acquiring a fringe pattern under the target posture by using the imaging device; An imaging device calibration module is configured to initialize an internal parameter and an external parameter of the imaging device to obtain an internal reference and an external parameter after the initialization of the parameter of the imaging device, and use a plurality of different target postures based on a beam adjustment method Performing joint nonlinear optimization on the target image, the three-dimensional coordinate system parameter of the target, and the initialized internal and external parameters to obtain calibration parameters of the imaging device to complete the imaging device Calibration The obtaining module is configured to dephase and phase unfold the fringe pattern acquired by the plurality of different target poses by using the N-step phase shift method, and obtain a phase unwrapping diagram of the plurality of different target poses; a projection device calibration module, configured to initialize an internal parameter and an external parameter of the projection device, to obtain an internal reference and an external parameter after the initialization of the projection device, and use the plurality of different target postures according to a beam adjustment method a phase unwrapping map, a three-dimensional coordinate system parameter of the target, and a joint non-linear optimization of the initialized internal and external parameters to obtain calibration parameters of the projection device to complete calibration of the projection device; a system calibration module, configured to calibrate the three-dimensional imaging system by using calibration parameters of the imaging device and calibration parameters of the projection device, to obtain calibration parameters of the three-dimensional imaging system, to complete calibration of the three-dimensional imaging system . The device according to claim 5, wherein the imaging device calibration module is specifically configured to initialize an internal parameter and an external parameter of the imaging device, and obtain an internal reference and an external parameter after the initialization of the parameter of the imaging device, and The calibration parameters of the camera device are obtained according to the following formula:

Where x _BA ^* represents the re-projection error residual value, N represents the total number of target images in a plurality of different target poses, and M represents the number of identified points in the target,

An imaging point representing the jth landmark in the target image at the i-th target pose; O ^ij (R, t _s , m, u ₀ , v ₀ , k, X) represents the ith target The ideal point of the jth marker point in the target image in the pixel coordinate system; X represents the three-dimensional coordinate system parameter of the target, R is the rotation matrix of the external parameter of the imaging device, and t _s is a translation matrix of the external reference of the imaging device, m is a magnification of the telecentric lens in the internal reference of the imaging device, (u ₀ , v ₀ ) represents an origin of the pixel coordinate system, and k is a distortion coefficient. The device according to claim 5, wherein the projection device calibration module is specifically configured to initialize an internal parameter and an external parameter of the projection device, obtain an internal reference and an external parameter after initialization of the projection device, and follow The calibration parameters of the projection device are obtained by the following formula:

Where x _BA ^* represents the reprojection error residual value, N represents the total number of phase unwrapped graphs in a plurality of different target poses, and M represents the number of identified points in the target,

An imaging point representing the jth landmark point in the phase unwrapped map at the i-th target pose; O ^ij (R, t _s , m, u ₀ , v ₀ , k, X) represents the ith target The ideal point of the jth marker point in the phase unwrapped graph in the pixel coordinate system; X represents the three-dimensional coordinate system parameter of the target, R is the rotation matrix of the external reference of the projection device, and t _s is a translation matrix of the external reference of the projection device, m is a magnification of the telecentric lens in the internal reference of the projection device, (u ₀ , v ₀ ) represents an origin of the pixel coordinate system, and k is a distortion coefficient. The device according to claim 5, wherein the system calibration module is specifically configured to: The calibration parameters of the three-dimensional imaging system are obtained according to the following formula: R ^p ·p _w +t ^p =R ^s ·(R ^c ·p _w +t ^c )+t ^s Wherein, R ^p represents a rotation matrix in the external reference of the projection device, t ^p represents a translation matrix in the external reference of the projection device, R ^c represents a rotation matrix in the external reference of the imaging device, and t ^c represents the imaging The translation matrix in the external reference of the device, R ^s represents the rotation matrix of the three-dimensional imaging system, t ^s represents the translation matrix of the three-dimensional imaging system, and p _w represents the world coordinate vector of any point P in the space; Where P _w = [x _w , y _w , z _w ] ^T ; And performing least squares optimization on the rotation matrix and the translation matrix of the three-dimensional imaging system, and determining the conversion relationship between the pixel coordinates and the world coordinates of the three-dimensional imaging system by using the optimized rotation matrix and the translation matrix.

Images