CN201707037U - Optical three-dimensional imager - Google Patents

Abstract

The utility model belongs to the field of manufacturing optical three-dimensional sensing instruments in the visible light region, in particular to an optical three-dimensional imager based on structured light illumination. It includes at least one optical structuring device equipped with a binary space-time grating to project structured light to the object; at least one camera shoots the deformed image reflected by the object at another position; analyzes the deformed image reflected by the object to obtain the same image plane as the camera The coordinates of the points on the projection plane corresponding to the points on the surface; the coordinates of the object measurement points are calculated according to the geometric relationship, and the three-dimensional reconstruction is performed to restore the three-dimensional shape of the object. This scheme can obtain high-density three-dimensional measurement results and obtain the texture of the object at the same time.

Description

An optical three-dimensional imager

Technical field:

The utility model belongs to the technical field of manufacturing optical three-dimensional sensing equipment in visible light and infrared regions, and in particular relates to an optical three-dimensional imager based on structured light illumination. the

Background technique:

Optical 3D sensing technology based on structured light illumination has a wide range of applications in the fields of industrial inspection, quality control, machine vision, film and television special effects, and biomedicine. One of the keys of this technique is the matching of the projection plane and the corresponding points of the camera image plane. Corresponding point matching is usually achieved by using color, grayscale, grating phase, and symbols or symbol sequences encoded with the above parameters. the

Figure 1 is a system schematic diagram of the three-dimensional imaging technology of structured light illumination. It consists of projectors and cameras. O _p , O _c are the optical centers of the projector and the camera, respectively, and (x _p0 , y _p0 ), (x _c0 , y _c0 ) are the image coordinates of the principal points of the projector and the camera, respectively. P _p and P _c are corresponding points of a pair of projection planes and camera image planes. P is a measurement point on the surface of the object, and its coordinates are (X _w , Y _w , Z _w ). A specific pattern is projected by the projector, and the camera captures the deformed pattern reflected by the surface of the object at another location. By analyzing the captured images, the coordinates of the points on the projection plane corresponding to the points on the image plane of the camera are obtained. Finally, in the triangle formed by PO _c O _p , the coordinates of point P are calculated according to the geometric relationship, and then the three-dimensional shape of the object is restored.

The existing structured light lighting 3D sensing technology can be divided into color-based and grayscale-based coding schemes according to whether it is related to color. According to different parameters used for encoding, it can be divided into sinusoidal grating phase encoding scheme and light intensity encoding scheme. the

Phase encoding schemes mainly include phase measurement profilometry PMP and Fourier transform profilometry FTP. the

Light intensity coding schemes can be divided into time coding schemes, spatial neighborhood coding schemes and direct coding schemes. High-resolution measurement results can be obtained by using the technique of sinusoidal grating projection. But using single-frequency sinusoidal grating projection, it is difficult to unwrap the phase of discontinuous objects. This problem can be solved by using time phase expansion method or combining with Gray code, but the time complexity of measurement will increase. It can also be solved by using composite grating or combining with color and binary coding, but there are certain restrictions on the measurement range or the surface texture of the object. the

Due to the need to project more patterns in the time coding scheme, the time complexity of measurement is usually high, and it is difficult to achieve high-density measurement, but the measurement accuracy is high. the

The spatial coding scheme has fast measurement speed, but the decoding process is more complicated, usually with low precision and low spatial resolution. the

Direct coding schemes are usually only suitable for the measurement of non-color or light-color objects, and it is not easy to achieve high measurement accuracy. the

At present, most optical 3D imaging sensors on the market are color-based spatial neighborhood encoding schemes, but this type of technology has an obvious disadvantage, that is, it has limitations on measuring the color of objects, and it cannot simultaneously obtain object texture from encoded images. the

Utility model content:

Aiming at the deficiencies of the above-mentioned prior art, the utility model provides an optimized optical three-dimensional imager. The optical three-dimensional imaging method adopted by the sensor has a measurement speed between the ordinary PMP and the time encoding scheme, the measurement accuracy and robustness are equivalent to the time encoding scheme, and the resolution is equivalent to the PMP. Not only high-density three-dimensional measurement results can be obtained, but also the texture of the object can be obtained at the same time. It can also be measured as long as enough coded regions are projected onto isolated objects. the

An optical three-dimensional imager, characterized in that:

At least one light structuring device equipped with a binary space-time grating projects structured light onto the object,

At least one camera at another location captures the deformed image reflected back by the object,

A signal processing device analyzes and three-dimensionally reconstructs the deformed image reflected by the object. the

The light structuring device can be a projector. the

The overall structure of the binary space-time grating equipped with the optical structuring device is determined by a pseudo-random sequence, which consists of the local spatial coordinates of several adjacent pixel points on the projection plane and their relative time coordinates when their state is 1 Combinations to encode a set of symbols. the

The binary space-time encoded grating is composed of a set of binary gratings displayed sequentially at the same spatial position. Its core technology is to encode a group of symbols by combining the local space coordinates of several adjacent pixel points called coding units on the projection plane and their relative time coordinates when their state is 1; A pseudo-random sequence of , to determine the overall structure of each binary raster. the

The pseudo-random sequence satisfies the following conditions:

In the sequence, the subsequence greater than or equal to the window length is unique; there is no repeated symbol in any subsequence whose length is equal to the window length. the

For example, a sequence with a window length of 4

"ABDECFADBEFDBECDABFECBDEFBDCEABCDAECFBDE". the

The state of each pixel on the projection plane meets the following requirements: it is 1 in one and only one grating, and it is 0 in other gratings. the

Adjacent coding units are distinguished by time coordinates. When the pixel state is 1, the relative time coordinates are classified as one coding unit if the relative time coordinate is less than the threshold value, and are classified as another coding unit if it is greater than or equal to the threshold value. the

Taking the space and time coordinate encoding of three adjacent pixels as an example, the principle of the binary space-time encoding grating is illustrated. Fig. 2 is a schematic diagram of the encoding principle. Vertical stripes are used here, and the state of the pixels on the projection plane only changes in the row direction, but remains unchanged in the column direction. Pick a line to describe. The figure arranges the rasters from top to bottom in the order shown. Each raster is divided into several small areas consisting of 3 pixels. As required, an area with a size of 3 pixels corresponds to three rasters in time. It can be seen from the figure: in space, in a raster, there is one and only one pixel in each unit is white, and the others are black; in time, in the same unit, each pixel is in one and only one It is white in one raster and black in the other rasters. In this way, a combination of a pixel's local spatial and temporal coordinates can be used to encode a region. As shown in the area marked "B" in Figure 2, the space-time coordinates of the first pixel in the unit are (1, 1), that is, the spatial position of the first pixel in the unit is 1, and the relative position when the state is 1 The time coordinate is 1, the space-time coordinate of the second pixel is (2, 3), and the space-time coordinate of the third pixel is (3, 2). Arrange these coordinates in order to obtain the code word "113223", which is represented by the letter "B" here. In the same way we can get other codewords. According to the permutation knowledge, you can get 3! = 6 codewords, represented by "A"-"F" respectively. In order to distinguish adjacent coding units, a threshold of 4 is set here. A total of 6 gratings are designed, and the time coordinates of pixel state 1 in the first coding unit are less than 4. The time coordinate at which the pixel state is 1 in the second coding unit is greater than or equal to 4. The time coordinate at which the pixel state is 1 in the third CU is less than 4. And so on, for coding. the

According to the above encoding principles, the result of superposition of all projection modes is an all-white image. Projecting an all-white image is equivalent to using a projector as a light source to illuminate an object, and what the camera captures is the texture of the object. Therefore, the texture of the object can be obtained as long as the captured image sequences are superimposed, and the texture does not need to be photographed separately. If a color camera is used, color textures can be obtained. the

A pseudo-random sequence of a specified length is constructed by continuously selecting a symbol randomly from a set of specified symbols and adding it to the constructed sequence to form a new sequence that meets the requirements. Specific steps are as follows:

First, randomly select a number of symbols equal to the window length from the specified symbols to form an initial sequence;

Second, randomly select a symbol from the specified symbols and add it to the constructed sequence;

Thirdly, check whether the subsequence in the window including the newly added symbol meets the requirement of the pseudo-random sequence described in the principle of binary space-time coding. If so, check whether the length of the entire sequence meets the requirements, if so, end, otherwise go to the second step above; if not, delete the symbol just added in the constructed sequence, and go to the second step above. the

The main steps of the three-dimensional reconstruction process in the utility model are as follows:

(1) For each point on the image plane of the camera, find the image with the highest light intensity of the corresponding pixel from the captured image sequence. Since projection and shooting are carried out in the same order, the acquisition sequence number of the image corresponding to the pixel with the largest light intensity is the time coordinate of the corresponding point on the projection plane whose state is 1, referred to as the time coordinate of the corresponding point. The corresponding point time coordinates of all points on the camera image plane form a corresponding point time coordinate map. the

(2) In the time coordinate diagram of the corresponding point, scan by row, identify the coding unit according to the coding rule and restore the corresponding projection symbol. The projection symbols corresponding to all coding units constitute a projection symbol map. the

(3) In the projected symbol graph, scan by row with the window length as the unit, match the subsequence in the window with the subsequence in the original projected sequence, and obtain the position of the symbol in the window in the original projected sequence. Then according to the encoding rules, the corresponding point maps of cameras and projectors are obtained. the

(4) After obtaining the corresponding points of the camera and projector, restore the three-dimensional shape of the object according to the principle of triangulation. the

(5) Add the intensities of the corresponding points of the captured image sequence to obtain the texture map of the object. the

Description of drawings:

Figure 1 is a schematic diagram of a three-dimensional imaging system with structured light illumination

Figure 2 is a schematic diagram of the principle of space-time binary encoding

Detailed ways:

The utility model will be further described below in conjunction with the accompanying drawings, but not limited to the scope shown in this embodiment. the

Example 1:

This embodiment is realized through the following technical solutions:

A projector equipped with a binary space-time grating casts structured light on the object,

A camera at another location captures the distorted image reflected by the object,

A signal processing device analyzes and three-dimensionally reconstructs the deformed image reflected by the object. the

in,

1. Use a sequence with a window length of 4:

"ABDECFADBEFDBECDABFECBDEFBDCEABCDAECFBDEABFCABDFAECBADEFCAEDCFEBDFECABFDEAFCEADBCADFBACDEACBDACEBDCABECAFECDBECDEBAFDEBFADECADEBCAEBDAEBCDEBADCBAFEBACEDFABDCFAEBFDCAEFDCBEACDBAECDC". the

The specific steps of a set of grating designed in this scheme are as follows:

(1) Calculate the number of symbols, coding unit size and pseudo-random sequence length according to the number of pixel columns of the projector, adopt the algorithm described in the utility model to construct the pseudo-random sequence;

(2) According to the binary space-time coding principle of the present invention, design the coding unit corresponding to each symbol;

(3) According to the code sequence constructed in (1), arrange the coding units in order, and finally obtain the binary space-time coding grating corresponding to the pseudo-random sequence;

(4) Project the designed grating onto the measured object successively according to its time coordinates, and simultaneously use a camera to capture the deformed grating modulated by the shape of the object;

(5) After all the gratings are projected, process the captured deformed gratings according to the steps described in 2.4 to obtain the three-dimensional shape and texture of the object. the

Firstly, the JAI_CVA50CCD industrial camera with a resolution of 740×572 is used to measure the typical scene with discontinuous surface height distribution and reflectance distribution, shadow and occlusion. the

The scenario adopted in this program is: a square cardboard box and a mouse are placed on a flat plate with a checkerboard pattern. This scheme does not adopt error correction algorithm, and discards coding units whose time coordinates of corresponding points are incompletely restored. Most of the effective areas are measured, and shadows have no effect on decoding. Due to the characteristics of the code sequence, an error recovery code affects the measurement results of 4 coding units at most. If a certain error detection and control algorithm is adopted, most errors are further limited to this unit and will not affect other units. Decoding errors are confined to a small area and are not propagated outward. The main reasons for the voids in the picture are: shadows; the surface reflectance of the object is too low to detect the change of light intensity; the camera line of sight is blocked; at the step texture of the object surface, due to the low-pass filtering characteristics of the camera, the coding unit The time coordinates of the inner corresponding points are restored incorrectly. the

Then Sony's DCR-SR100 civilian color CCD camera is used for experiments with a resolution of 2016×1512. The measurement object is a plaster model with a maximum depth of about 100mm and a part of the white screen that can be regarded as an isolated object. the

Due to the use of civilian cameras, the signal-to-noise ratio of the image is lower, and larger noise appears in the place where there is no stripe in the time coordinate map of the corresponding point. As can be seen from the projected symbol map represented by color, the noise generated in the previous stage has little effect on the projected symbol recovery. . the

The measured object represented by the colored grid is different from the projector transverse pixel coordinates of the reference plane, and the shape of the object is well restored. the

Claims (2)

1. An optical three-dimensional imager, characterized in that: at least one light structuring device equipped with a binary spatiotemporal grating projects structured light onto the object, at least one camera at another location captures the deformed image reflected back by the object, A signal processing device analyzes and three-dimensionally reconstructs the deformed image reflected by the object. 2. The optical three-dimensional imager according to claim 1, characterized in that said optical structuring device is a projector. the

Images