CN114488103A - Ranging system, ranging method, robot, equipment and storage medium - Google Patents

Abstract

The present invention provides a ranging system, a ranging method, a robot, a system and a storage medium. The ranging system includes: a stereo vision device, a projection device and a processing device; the stereo vision device includes at least two camera devices; The projection device includes a light-emitting device, a pattern mask and a lens group; wherein, the light-emitting device is used to emit light; the light passes through the pattern mask to form a patterned light beam; the lens group is used to The light beam of the pattern is projected onto the target object to form a texture on the target object; the camera device is used for capturing an image of the target object to obtain a texture image; the processing device is used for capturing images according to the at least two camera devices The texture image determines the depth of the target object. The invention reduces the manufacturing cost of the existing distance measuring device.

Description

Ranging system, ranging method, robot, equipment and storage medium

technical field

The present invention relates to the field of computer vision, and in particular, to a ranging system, a ranging method, a robot, a device and a storage medium.

Background technique

Binocular StereoVision is an important form of computer vision. It is based on the principle of parallax and uses imaging equipment to obtain two images of the measured object from different positions, and obtains by calculating the positional deviation between the corresponding points of the images. Methods for 3D geometric information of objects. A condition for the existing binocular stereo vision technology to successfully calculate the distance of the object is that the object to be measured must have a certain degree of texture, otherwise it cannot be detected by the algorithm at the imaging position of the two cameras, and it cannot be matched and subsequent. calculate.

For the problem that textureless objects cannot be matched by binocular vision algorithms, the prior art mainly uses VCSEL (Vertical-Cavity Surface-Emitting Laser) combined with DOE (Diffractive optical element) to solve the problem. The laser emitted by the VSCEL becomes parallel light after collimation, and then projects a dense light spot that conforms to a certain law through the DOE, covering the surface of the object with a dense texture that meets the requirements of the algorithm. However, in the scheme based on VCSEL+DOE projection speckle to create textures for objects, DOE is made by VLSI manufacturing process, and in order to make the texture clearer and eliminate the influence of ambient light, infrared light VCSELs are generally used, camera lenses The visible light will also be filtered out on the side, resulting in a relatively short distance observed by the stereo vision algorithm, and will also lose information such as the original color and texture of the object.

It can be seen from the above that the existing technology will lead to problems of high cost and large loss of visual information.

SUMMARY OF THE INVENTION

The main purpose of the present invention is to provide a ranging system, a ranging method, a robot, a device and a storage medium. It aims to solve the problems of high cost of existing ranging technology and large loss of visual information.

In order to achieve the above object, the present invention provides a ranging system, including: a stereo vision device, a projection device and a processing device; the stereo vision device includes at least two camera devices; the projection device includes a light emitter, a pattern mask, and a lens group; of which,

the light emitter is used to emit light; the light passes through the pattern mask to form a patterned light beam; the lens group is used to project the patterned light beam onto the target object to form a texture on the target object;

The camera device is used for image acquisition of the target object to obtain a texture image;

The processing device is configured to determine the depth of the target object according to the texture images collected by the at least two camera devices.

Optionally, the pattern mask includes a light-shielding area and a light-transmitting area, the light-transmitting area and the light-shielding area form a pattern, and the light passes through the light-transmitting portion to form a patterned light beam.

Optionally, the light-transmitting part is made of a transparent material; or, the light-transmitting part is provided with a plurality of light-transmitting holes; the light passes through the pattern mask to form a patterned light beam.

Optionally, the projection device further includes a light collimator, the light collimator is arranged between the light emitter and the pattern mask, and is used for collimating the light emitted by the light emitter into parallel light. .

In addition, in order to achieve the above object, the present invention also provides a ranging method, which is applied to a ranging system, and the ranging system includes a stereo vision device and a projection device; the stereo vision device includes at least two Camera equipment; the projection device includes a light-emitting device, a pattern mask and a lens group; the ranging method includes the steps:

Light is emitted through the light emitter; the light passes through the pattern mask to form a patterned light beam;

Projecting the patterned light beam onto the target object through the lens group to form texture on the target object;

Image acquisition of the target object by the camera device to obtain a texture image;

The depth of the target object is determined according to the texture images collected by the at least two camera devices.

Optionally, the step of determining the depth of the target object according to the texture images collected by the at least two camera devices includes:

acquiring calibration parameters of the at least two camera devices;

Respectively extract feature information of texture images at different scales collected by the at least two camera devices;

According to the feature information at the different scales, calculating the matching cost corresponding to the texture images collected by the at least two camera devices;

Perform cost aggregation calculation on the calculated matching cost, and obtain the target disparity value according to the calculated result;

According to the calibration parameter and the target parallax value, the depth of the target object is calculated.

Optionally, the step of calculating the matching cost corresponding to the texture images collected by the at least two camera devices according to the feature information at different scales includes:

According to the feature information at different scales, the texture images collected by the at least two camera devices are traversed pixel by pixel within a preset parallax search range, and the matching cost of each pixel under different parallaxes is calculated.

Optionally, the step of calculating the depth of the target object according to the calibration parameter and the target parallax value includes:

Calculate the object depth value corresponding to each pixel according to the relative distance between the at least two imaging devices, the focal length of the imaging devices, and the target parallax value;

Calculate the depth of the target object according to the object depth value corresponding to each pixel.

In addition, the present invention also provides a robot comprising the above distance measuring system.

In addition, the present invention provides a distance measuring device, the distance measuring system includes a robot, a memory, a processor, and a distance measuring program stored on the memory and executable on the processor, the distance measuring When the program is executed by the processor, the steps of the above-mentioned ranging method are implemented.

In addition, the present invention also provides a storage medium on which a ranging program is stored, and when the ranging program is executed by a processor, implements the steps of the above-mentioned ranging method.

The present invention provides a ranging system, a ranging method, a robot, a system and a storage medium. The ranging system includes: a stereo vision device, a projection device and a processing device; the stereo vision device includes at least two camera devices; The projection device includes a light emitter, a pattern mask and a lens group; wherein, the light emitter is used to emit light; the light passes through the pattern mask to form a patterned beam; the lens group is used to transmit the patterned light beam. The light beam is projected onto the target object to form a texture on the target object; the camera device is used to collect images of the target object to obtain a texture image; The texture image determines the depth of the target object. The distance measuring method includes the steps of: emitting light through the light emitter; the light passing through the pattern mask to form a patterned light beam; projecting the patterned light beam onto a target object through the lens group, on the target object forming a texture; capturing an image of a target object by the camera device to obtain a texture image; determining the depth of the target object according to the texture images collected by the at least two camera devices. Through the above structure and method, the present invention can realize the distance measurement of non-textured objects. By performing rough texture projection on the non-textured objects, the accurate distance between the robot and the object can be measured, thereby solving the problem of existing distance measurement methods. The disadvantages of having to use a precise projection device for projection, and the problem of large loss of visual information in the existing projection device, at the same time, the manufacturing cost of the robot can be reduced by the ranging system of the present application.

Description of drawings

1 is a schematic diagram of a device structure of a hardware operating environment involved in an embodiment of the present invention;

2 is a schematic structural diagram of a ranging system of the present invention;

3 is a schematic structural diagram of a projection device in the ranging system of the present invention;

FIG. 4 is a schematic flowchart of the first embodiment of the ranging method according to the present invention;

5 is a schematic diagram of a refinement flow of step S40 in the first embodiment of the ranging method of the present invention;

FIG. 6 is a schematic diagram of a refinement flow of step S45 in the first embodiment of the ranging method of the present invention.

The realization, functional characteristics and advantages of the present invention will be further described with reference to the accompanying drawings in conjunction with the embodiments.

Detailed ways

In order to make the objectives, technical solutions and advantages of the present invention clearer, the present invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention, but not to limit the present invention.

The robots involved in this application may include cleaning robots, housekeeping robots, service robots, logistics robots, etc. Specifically, the types of cleaning robots may include sweeping robots, mopping robots, sweeping and mopping integrated robots, etc., wherein the cleaning robots can be used for The ground is automatically cleaned, and the application scenarios can be household indoor cleaning, large-scale venue cleaning, etc. The following takes the cleaning robot as an example to describe:

On the cleaning robot, a cleaning member and a driving device are provided, and the driving device may include a motor and a driving wheel. Driven by the driving device, the cleaning robot moves itself according to the preset cleaning path, and cleans the ground through the cleaning piece. For the sweeping cleaning robot, the cleaning piece is a sweeping piece, and the sweeping cleaning robot is provided with a vacuuming device. During the cleaning process, the sweeping piece sweeps dust, garbage, etc. to the suction port of the vacuuming device, so that the vacuuming device will Absorb and temporarily store dust, garbage, etc. For the cleaning robot, the cleaning member is a mopping member (such as a mop), and the mopping member is in contact with the ground. During the movement of the cleaning robot, the mopping member rubs the ground to clean the ground. For the sweeping and mopping integrated cleaning robot, the cleaning parts include sweeping parts and mopping parts. The sweeping parts and the mopping parts can work at the same time for mopping and sweeping, or they can work separately, mopping and sweeping respectively. The sweeping member may further include a side brush and a roller brush (also called a middle brush). The side brush sweeps dust and other garbage from the outside to the middle area, and the roller brush continues to sweep the garbage to the vacuuming device.

In order to facilitate the use of users, the cleaning robot is often used in conjunction with the base station. The base station can be used to charge the cleaning robot. When the power of the cleaning robot is less than the threshold during the cleaning process, the cleaning robot automatically moves to the base station for charging. . For the cleaning robot, the base station can also clean the mopping member (such as a mop). After the mopping member of the cleaning robot wipes the ground, the mopping member often becomes dirty and needs to be cleaned. For this purpose, the base station can be used to clean the mopping parts of the cleaning robot. Specifically, the mopping and cleaning robot can be moved to the base station, so that the cleaning mechanism on the base station can automatically clean the mopping parts of the cleaning robot. In addition to the above functions, the base station can also manage the robot through the base station, so that the robot can control the robot more intelligently during the process of performing cleaning tasks, and improve the intelligence of the robot's work.

As shown in FIG. 1 , FIG. 1 is a schematic structural diagram of a robot of a hardware operating environment involved in an embodiment of the present invention.

The main solutions of the embodiments of the present invention are: emit light through the light emitter; the light passes through the pattern mask to form a patterned light beam; the patterned light beam is projected onto the target object through the lens group, and the patterned light beam is projected on the target object through the lens group. A texture is formed on the object; an image of the target object is acquired by the camera device to obtain a texture image; the depth of the target object is determined according to the texture images collected by the at least two camera devices.

Because in the prior art, in the process of moving the cleaning robot, it often encounters obstacles (such as trash cans, walls, etc.), and when it passes through the obstacles, the cleaning robot often fails to recognize the obstacles, and is different from the obstacles. A collision occurs, which affects the normal operation of the robot.

The present invention provides the above solution, aiming at obtaining information on obstacles, avoiding collision between the cleaning robot and obstacles, and ensuring the normal operation of the robot.

The present invention proposes a cleaning robot, which can be an automated device for environmental cleaning, such as a sweeping robot, a mopping robot, and the like. In addition, in other embodiments, the robot may also be other types of robots, such as service robots and the like.

As shown in FIG. 1 , the robot may include: a processor 1001 such as a CPU, a communication bus 1002 , a user interface 1003 , a network interface 1004 , a memory 1005 and a perception unit 1006 . Among them, the communication bus 1002 is used to realize the connection and communication between these components. The user interface 1003 may include a display screen (Display), an input unit such as a keyboard (Keyboard), and the optional user interface 1003 may also include a standard wired interface and a wireless interface. Optionally, the network interface 1004 may include a standard wired interface and a wireless interface (eg, a Wi-Fi interface).

The memory 1005 is provided on the main body of the robot, and a program is stored in the memory 1005, and the program is executed by the processor 1001 to realize corresponding operations. Memory 1005 is also used to store parameters for use by the robot. The memory 1005 may be high-speed RAM memory, or may be non-volatile memory, such as disk memory. Optionally, the memory 1005 may also be a storage device independent of the aforementioned processor 1001 .

The robot can communicate with the user terminal through the network interface 1004 . The robot can also communicate with the base station through short-range communication technology. Among them, the base station is a cleaning device used with the robot.

The sensing unit 1006 includes various types of sensors, such as lidar, collision sensors, distance sensors, drop sensors, counters, and gyroscopes, among others.

The lidar is set on the top of the robot body. When working, the lidar rotates and emits a laser signal through the transmitter on the lidar. The laser signal is reflected by the obstacle, so that the receiver of the lidar receives the laser signal reflected by the obstacle. . By analyzing the received laser signal, the circuit unit of the lidar can obtain information about the surrounding environment, such as the distance and angle of obstacles relative to the lidar. In addition, a camera can also be used to replace the lidar, and by analyzing the obstacles in the images captured by the camera, the distance and angle of the obstacles relative to the camera can also be obtained.

Crash sensors include crash housings and trigger sensors. The collision shell surrounds the head of the robot body. Specifically, the collision shell may be disposed at the head of the robot body and the front positions of the left and right sides of the robot body. The trigger sensor is located inside the robot body and behind the collision shell. An elastic buffer is provided between the collision shell and the robot body. When the robot collides with the obstacle through the collision shell, the collision shell moves into the robot and compresses the elastic buffer. After the collision shell moves a certain distance into the robot, the collision shell contacts the trigger sensor, and the trigger sensor is triggered to generate a signal, which can be sent to the robot controller in the robot body for processing. After hitting the obstacle, the robot moves away from the obstacle, and under the action of the elastic buffer, the collision shell moves back to its original position. It can be seen that the collision sensor can detect obstacles and play a buffering role after hitting the obstacle.

Specifically, the distance sensor may be an infrared detection sensor, which may be used to detect the distance from the obstacle to the distance sensor. The distance sensor can be arranged on the side of the robot body, so that the distance value from the obstacle located near the side of the robot to the distance sensor can be measured by the distance sensor. The distance sensor may also be an ultrasonic ranging sensor, a laser ranging sensor, or a depth sensor, etc., which is not limited here.

The drop sensors can be arranged on the bottom edge of the robot body, and the number can be one or more. When the robot moves to the edge of the ground, the drop sensor can detect the risk of the robot falling from a height, so as to execute the corresponding anti-drop response, such as the robot stops moving, or moves away from the falling position, etc.

There are also counters and gyroscopes inside the robot body. The counter is used to accumulate the total number of rotation angles of the driving wheel, so as to calculate the length of the distance that the driving wheel drives the robot to move. The gyroscope is used to detect the angle of rotation of the robot, so that the orientation of the robot can be determined.

Those skilled in the art can understand that the robot structure shown in FIG. 1 does not constitute a limitation to the robot, and may include more or less components than the one shown, or combine some components, or arrange different components.

As shown in FIG. 1 , the memory 1005 as a computer storage medium may include an operating system, a network communication module, a user interface module, and a ranging program of the robot.

In the robot shown in FIG. 1 , the network interface 1004 is mainly used to connect the base station, the charging stand, etc., which are used together with the robot, and perform data communication with the base station. Cleaning, etc.; the user interface 1003 is mainly used to connect the client and communicate data with the client; and the processor 1001 can be used to call the ranging program of the robot stored in the memory 1005, and perform the following operations:

The ranging method includes the steps:

Light is emitted through the light emitter; the light passes through the pattern mask to form a patterned light beam;

Projecting the patterned light beam onto the target object through the lens group to form texture on the target object;

Image acquisition of the target object by the camera device to obtain a texture image;

The depth of the target object is determined according to the texture images collected by the at least two camera devices.

Further, the processor 1001 can call the ranging program stored in the memory 1006, and also perform the following operations:

The step of determining the depth of the target object according to the texture images collected by the at least two camera devices includes:

acquiring calibration parameters of the at least two camera devices;

Respectively extract feature information of texture images at different scales collected by the at least two camera devices;

According to the feature information at the different scales, calculating the matching cost corresponding to the texture images collected by the at least two camera devices;

Perform cost aggregation calculation on the calculated matching cost, and obtain the target disparity value according to the calculated result;

According to the calibration parameter and the target parallax value, the depth of the target object is calculated.

Further, the processor 1001 can call the ranging program stored in the memory 1006, and also perform the following operations:

The step of calculating the corresponding matching cost between the texture images collected by the at least two camera devices according to the feature information at different scales includes:

According to the feature information at different scales, the texture images collected by the at least two camera devices are traversed pixel by pixel within a preset parallax search range, and the matching cost of each pixel under different parallaxes is calculated.

Further, the processor 1001 can call the ranging program stored in the memory 1006, and also perform the following operations:

The step of calculating the depth of the target object according to the calibration parameter and the target parallax value includes:

Calculate the object depth value corresponding to each pixel according to the relative distance between the at least two imaging devices, the focal length of the imaging devices, and the target parallax value;

Calculate the depth of the target object according to the object depth value corresponding to each pixel.

Based on the above hardware structure, various embodiments of the ranging system of the present invention are proposed.

Please refer to FIG. 2 and FIG. 3 , FIG. 2 is a schematic structural diagram of a ranging system of the present invention, and FIG. 3 is a structural schematic diagram of a projection device in the ranging system of the present invention. The ranging system includes: a stereo vision device 01, a projection device 02 and a processing device 03; the stereo vision device 01 includes at least two camera devices; the projection device 02 includes a light-emitting device 021, a pattern mask 022 and a lens group 023; wherein, the light emitter 021 is used to emit light; the light passes through the pattern mask 022 to form a patterned light beam; the lens group 023 is used to project the patterned light beam onto the target object, in A texture is formed on the target object; the camera equipment is used to collect images of the target object to obtain a texture image; the processing device 03 is used to determine the texture image of the target object according to the texture images collected by the at least two camera equipment. depth.

In this embodiment, the projection device 02 can be applied to a robot, can be installed on the robot, or can be installed in a scene that requires projection ranging, such as a white wall without texture, furniture without texture, etc., the installation method is flexible , which can be applied to various occasions where ranging is required. It should be noted that the relationship between the stereoscopic vision device 01 and the projection device 02 needs to satisfy that the projection area of the projection device 02 and the stereoscopic digital vision device, that is, the visual coverage area of the camera equipment, partially or completely overlap, and the specific arrangement can be as described above. The stereo vision device 01 and the projection device 02 can be assembled on the same plane. Specifically, the projection device 02 can be arranged in the middle, and the stereo vision device 01 can be arranged on both sides, that is, two camera devices are arranged on both sides of the projection device 02; The projection device 02 is on the side, and the stereo vision device 01 is in the middle; or two camera devices in the stereo vision device 01 and the projection device 02 are in a triangular positional relationship, or a plurality of camera devices in the stereo vision device 01 and the projection device 02 are in a five-dimensional relationship. The arrangement and position relationship of the cylinders, etc., are not limited in this application. In addition, the stereoscopic vision device 01 and the projection device 02 may not be arranged on the same plane, as long as the projection area of the projection device 02 and the stereoscopic digital vision device, that is, the visual coverage area of the imaging device, partially or completely overlap the condition. . The light emitter 021 can be selected from LED or VCSEL to generate light. The lens group 023 is used to project the patterned light beam onto the target object according to a preset angle. In this embodiment, the above-mentioned structure can replace the existing VCSEL+DOE solution with high cost in the prior art, avoiding the situation of large loss of visual information in the existing solution, and this solution has a long ranging distance, which is convenient for combined measurement Distance algorithms to develop more vision applications, such as semantic segmentation, object detection, etc.

In one embodiment, the pattern mask 022 includes a light-shielding region and a light-transmitting region, the light-transmitting region and the light-shielding region form a pattern, and the light passes through the light-transmitting portion to form a patterned light beam.

In one embodiment, the light-transmitting portion is made of a transparent material; or, the light-transmitting portion is provided with a plurality of light-transmitting holes; the light passes through the pattern mask 022 to form a patterned light beam.

In this embodiment, the pattern designed in advance can be attached to a transparent medium, such as glass, by means of laser printing or etching. The laser-printed or etched part forms a light-transmitting area, while the non-laser-printed or etched part forms a light-transmitting area. When the light passes through the pattern mask 022, the printed or etched part of the area will be blocked, so that the light-transmitting area will pass through. A patterned beam is formed. Alternatively, a hollowed-out method is used to form a patterned hollowed-out portion on the pattern mask 022, and when the light passes through the hollowed-out portion, a patterned light beam will be formed. In this embodiment, the patterned light beam is formed by the above method, the manufacturing cost is low, and the formed pattern is clear, which can meet the texture projection in different situations, and make the distance measurement more accurate.

In one embodiment, the projection device 02 further includes a light collimator, and the light collimator is disposed between the light emitter 021 and the pattern mask 022 for emitting the light emitter 021. The rays are collimated into parallel rays. In this embodiment, the light collimator can make the projection pattern of the projection device 02 more accurate and clear.

Please refer to FIG. 4. FIG. 4 is a schematic flowchart of a first embodiment of a ranging method according to the present invention. The ranging method is applied to a ranging system, and the ranging system includes a stereo vision device and a projection device; the stereo vision device It includes at least two imaging devices; the projection device includes a light emitter, a pattern mask and a lens group; the ranging method provided by this embodiment includes the following steps:

Step S10, emit light through the light emitter; the light passes through the pattern mask to form a patterned beam;

Step S20, projecting the patterned light beam onto the target object through the lens group to form a texture on the target object;

In this embodiment, the lens group projects a patterned light beam onto a target object through a preset angle, and the target object is an object that needs to be measured.

Step S30, performing image acquisition on the target object by the camera device to obtain a texture image;

In this embodiment, the number of the camera devices is at least two. Of course, multiple camera devices can also be set for image acquisition, which is not limited in the present invention. For the convenience of description, the present invention adopts two camera devices to perform image capture. collection.

Step S40, determining the depth of the target object according to the texture images collected by the at least two imaging devices.

In one embodiment, referring to FIG. 5 , the step S40 further includes:

Step S41, acquiring calibration parameters of the at least two camera devices;

Step S42, respectively extracting the feature information of the texture images collected by the at least two camera devices at different scales;

In this embodiment, the calibration parameters include the focal length of the imaging device, the optical center, the distortion coefficient, and the relative position between two or more imaging devices. The different scales include macroscopic large scales and microscopic small scales. Specifically, the minimum scale and the maximum scale can be set, and then the minimum scale and the maximum scale are equally divided, and then the features in the texture image of each equally divided scale are obtained. information, or use an image pyramid to obtain texture images of different resolutions, that is, texture images of different scales, by downsampling the resolution of the texture image. The higher the pyramid level, the smaller the image scale and the lower the resolution. The feature information can be obtained according to a convolutional neural network. In addition, after acquiring the two texture images, it is necessary to perform image stereo correction on the texture images, that is, the two images that are not on the same plane and have the same spatial point after actual shooting are corrected into coplanar alignment. , so that the matching pixel search is changed from a two-dimensional search to a one-dimensional search, thereby improving the efficiency of the matching search.

Step S43, according to the feature information under the different scales, calculate the matching cost corresponding to the texture images collected by the at least two camera devices;

In this embodiment, the cost calculation is a process of finding the corresponding positional relationship of the same texture on different texture images.

In one embodiment, the step S43 further includes:

Step A431: Perform pixel-by-pixel traversal on the texture images collected by the at least two camera devices with a preset parallax search range according to the feature information at different scales, and calculate the matching cost of each pixel under different parallaxes.

In this embodiment, the parallax is the distance between the pixel coordinates of two matching pixels in the two texture images, where the distance is in pixels, and the purpose of calculating the matching cost is to measure the distance between the pixel to be matched and the candidate pixel. Correlation. No matter whether two pixels are the same name point, the matching cost can be calculated by the matching cost function. The smaller the cost, the greater the correlation, and the greater the probability of being the same name point. The same name point is the same point.

Step S44, performing cost aggregation calculation on the calculated matching cost, and obtaining a target disparity value according to the calculated result;

In the embodiment of this city, after the cost of all pixels is calculated separately, the overall optimization of the cost distribution of the whole image is further performed, which can more effectively avoid the problem that the correct image correspondence cannot be found due to sparse texture. For example, there may be the i-th pixel in the first texture image, and the cost corresponding to the positional relationship between two or more different pixels in the second texture image is the same or similar. It is necessary to further determine which pixel is correct. Therefore, the cost of the surrounding pixels can be used to determine the cost. Specifically, the global or semi-global cost can be aggregated for the multi-scale cost to obtain the optimized cost.

Step S45: Calculate the depth of the target object according to the calibration parameter and the target parallax value.

In one embodiment, referring to FIG. 6 , the step S45 further includes:

Step S451: Calculate the object depth value corresponding to each pixel according to the relative distance between the at least two imaging devices, the focal length of the imaging devices, and the target parallax value;

The object depth value corresponding to each pixel refers to the distance between each pixel in the texture image and the target object. Specifically, the object depth value corresponding to each pixel is calculated according to the following formula:

z=Fb/d;

Among them, z is the depth distance, F is the focal length of the camera, b is the relative distance between the cameras corresponding to the texture image, and d is the target disparity value between the pixels of the same object on the texture images of different cameras.

Step S452, calculating the depth of the target object according to the object depth value corresponding to each pixel;

In this embodiment, according to the object depth value of each pixel, the overall depth value of the target object can be directly calculated, that is, the distance from the robot to the target object. The minimum depth value of all pixels can be taken as the depth of the target object, or the depth of the target object can be further selected according to the overall depth of the object.

The present invention provides a ranging method, which comprises the steps of: emitting light through the light emitter; passing the pattern mask to form a patterned light beam; projecting the patterned light beam through the lens group onto the target object, forming a texture on the target object; capturing images of the target object by the camera device to obtain a texture image; determining the depth of the target object according to the texture images collected by the at least two camera devices. Through the above structure and method, the present invention can realize the distance measurement of non-textured objects. By performing rough texture projection on the non-textured objects, the accurate distance between the robot and the object can be measured, thereby solving the problem of existing distance measurement methods. The disadvantages of having to use a precise projection device for projection, and the problem of large loss of visual information in the existing projection device, at the same time, the manufacturing cost of the robot can be reduced by the ranging system of the present application. And through the combination of the ranging system and the ranging method in the present invention, the requirement for the fineness of the object texture can be greatly reduced. Under this premise, it is feasible that the distance measuring device used in the present invention replaces VCSEL+DOE to add a certain quality of texture to the original textureless object, which reduces the manufacturing cost of the distance measuring device.

In addition, an embodiment of the present invention also provides a computer-readable storage medium, where a ranging program is stored on the computer-readable storage medium, and when the ranging program is executed by a processor, the following operations are implemented:

Light is emitted through the light emitter; the light passes through the pattern mask to form a patterned light beam;

Projecting the patterned light beam onto the target object through the lens group to form texture on the target object;

Image acquisition of the target object by the camera device to obtain a texture image;

The depth of the target object is determined according to the texture images collected by the at least two camera devices.

Further, when the ranging program is executed by the processor, the following operations are also implemented:

The step of determining the depth of the target object according to the texture images collected by the at least two camera devices includes:

acquiring calibration parameters of the at least two camera devices;

Respectively extract feature information of texture images at different scales collected by the at least two camera devices;

According to the feature information at the different scales, calculating the matching cost corresponding to the texture images collected by the at least two camera devices;

Perform cost aggregation calculation on the calculated matching cost, and obtain the target disparity value according to the calculated result;

According to the calibration parameter and the target parallax value, the depth of the target object is calculated.

Further, when the ranging program is executed by the processor, the following operations are also implemented:

The step of calculating the corresponding matching cost between the texture images collected by the at least two camera devices according to the feature information at the different scales includes:

According to the feature information at different scales, the texture images collected by the at least two camera devices are traversed pixel by pixel within a preset parallax search range, and the matching cost of each pixel under different parallaxes is calculated.

Further, when the ranging program is executed by the processor, the following operations are also implemented:

The step of calculating the depth of the target object according to the calibration parameter and the target parallax value includes:

Calculate the object depth value corresponding to each pixel according to the relative distance between the at least two imaging devices, the focal length of the imaging devices and the target parallax value;

Calculate the depth of the target object according to the object depth value corresponding to each pixel.

The specific embodiments of the computer-readable storage medium of the present invention are basically the same as the above-mentioned embodiments of the ranging method, and are not repeated here.

It should be noted that, herein, the terms "comprising", "comprising" or any other variation thereof are intended to encompass non-exclusive inclusion, such that a process, method, article or system comprising a series of elements includes not only those elements, It also includes other elements not expressly listed or inherent to such a process, method, article or system. Without further limitation, an element qualified by the phrase "comprising a..." does not preclude the presence of additional identical elements in the process, method, article or system that includes the element.

The above-mentioned serial numbers of the embodiments of the present invention are only for description, and do not represent the advantages or disadvantages of the embodiments.

From the description of the above embodiments, those skilled in the art can clearly understand that the methods of the above embodiments can be implemented by means of software plus a necessary general hardware platform, and of course hardware can also be used, but in many cases the former is better implementation. Based on this understanding, the technical solutions of the present invention can be embodied in the form of software products in essence or the parts that make contributions to the prior art. The computer software products are stored in a storage medium (such as ROM/RAM) as described above. , magnetic disk, optical disk), including several instructions to make a robotic device execute the method described in each embodiment of the present invention.

The above are only preferred embodiments of the present invention, and are not intended to limit the scope of the present invention. Any equivalent structure or equivalent process transformation made by using the contents of the description and drawings of the present invention, or directly or indirectly applied in other related technical fields , are similarly included in the scope of patent protection of the present invention.

Claims (11)

1. A ranging system, comprising: a stereoscopic vision device, a projection device and a processing device; the stereoscopic vision apparatus includes at least two image pickup devices; the projection device comprises a light emitter, a pattern mask and a lens group; wherein,

the light emitter is used for emitting light; the light passes through the pattern mask to form a light beam with a pattern; the lens group is used for projecting the light beam with the pattern onto a target object to form texture on the target object;

the camera shooting device is used for carrying out image acquisition on a target object to obtain a texture image;

and the processing device is used for determining the depth of the target object according to the texture images acquired by the at least two camera devices.

2. The range finding system of claim 1 wherein the pattern mask includes a light-blocking area and a light-transmitting area, the light-transmitting area and the light-blocking area forming a pattern, and light passing through the light-transmitting area forming a patterned beam.

3. The range finding system of claim 2 wherein the light transmissive portion is a transparent material; or the light transmission part is provided with a plurality of light transmission holes; the light passes through the pattern mask to form a patterned beam.

4. A ranging system according to any of claims 1-3, wherein the projection means further comprises a light collimator arranged between the light emitter and the pattern mask for collimating light emitted by the light emitter into parallel light rays.

5. A distance measurement method is characterized in that the distance measurement method is applied to a distance measurement system, and the distance measurement system comprises a stereoscopic vision device and a projection device; the stereoscopic vision apparatus includes at least two image pickup devices; the projection device comprises a light emitter, a pattern mask and a lens group; the distance measuring method comprises the following steps:

emitting light through the light emitter; the light passes through the pattern mask to form a light beam with a pattern;

projecting the patterned light beam onto a target object through the lens group to form a texture on the target object;

acquiring an image of a target object by the camera equipment to obtain a texture image;

and determining the depth of the target object according to the texture images acquired by the at least two camera devices.

6. The range finding method of claim 5, wherein the step of determining the depth of the target object from the texture images acquired by the at least two camera devices comprises:

acquiring calibration parameters of the at least two camera devices;

respectively extracting characteristic information of texture images acquired by the at least two camera devices under different scales;

calculating corresponding matching cost between texture images acquired by the at least two camera devices according to the characteristic information under different scales;

performing cost aggregation calculation on the calculated matching cost, and obtaining a target parallax value according to a calculated result;

and calculating the depth of the target object according to the calibration parameters and the target parallax value.

7. The distance measuring method according to claim 5, wherein the step of calculating the matching cost corresponding to the texture images acquired by the at least two image capturing devices according to the feature information at the different scales comprises:

and according to the feature information under different scales, traversing the texture images acquired by the at least two camera devices pixel by pixel in a preset parallax searching range, and calculating the matching cost of each pixel under different parallaxes.

8. The ranging method according to claim 6 or 7, wherein the step of calculating the depth of the target object based on the calibration parameters and the target disparity value comprises:

calculating an object depth value corresponding to each pixel according to the relative distance between the at least two camera devices, the focal length of the camera devices and the target parallax value;

and calculating the depth of the target object according to the object depth value corresponding to each pixel.

9. A robot, characterized in that it comprises a ranging system according to claims 1-4.

10. A ranging apparatus comprising a robot, a memory, a processor, and a ranging program stored on the memory and executable on the processor, the ranging program when executed by the processor implementing the steps of the ranging method of any of claims 5 to 8.

11. A storage medium having stored thereon a ranging program which, when executed by a processor, performs the steps of the ranging method according to any one of claims 5 to 8.

Images