TWI880360B - Laser sensing device and calibration method thereof - Google Patents

Abstract

A laser sensing device includes a laser source, an image sensor and a processor. The laser source is configured to emit a laser line to a target, and moves in a first direction. The image sensor is configured to face a space area at which the target locates and capture an object image. The processor is configured to: compute a reference object distance between the target and the image sensor from the object image; compute a field of view of a reference plane according to the reference object distance and a nearest field of view, a farthest field of view and a depth of field of the image sensor; and zoom in and out a length of a second direction which is vertical to the first direction in the object image according to a reference field of view, the field of view of the reference plane, a target image aspect ratio of a reference image corresponding to the target in the reference field of view and a physical aspect ratio of the target.

Description

Laser sensing device and calibration method thereof

The present invention relates to laser sensing, and in particular to a laser sensing device and a calibration method thereof.

Laser triangulation scanners are often used to scan targets with varying depths on their surfaces. Through line scanning, the three-dimensional structure of the target can be quickly captured. However, in the structure of the sensor lens in the laser triangulation scanner, when the distance between the target and the laser triangulation scanner changes, the resolution of the scanned image in the horizontal direction will be different, causing deformation of the two-dimensional plane size ratio.

Currently, to solve this problem, precise calibration calculations are required. First, the resolutions corresponding to different depths are calculated, and then the ratio of the original scanned image that needs to be converted at each depth is calculated. The calculation process is complex and time-consuming.

The present invention provides a laser sensing device and a calibration method thereof. When the laser sensing device scans the target, the reference object distance of the target relative to the laser sensing device is obtained. Then, according to the basic parameters of the laser sensing device such as the nearest field of view, the farthest field of view, the depth of field, and the target image aspect ratio of the image captured in the nearest field of view or the farthest field of view and the physical aspect ratio of the target, the horizontal length ratio of the image is scaled and adjusted. The image deformed in the horizontal direction of the reference object distance can be corrected, simplifying the current cumbersome calculation process of precise calibration by calculating the absolute distance value in space. The calibration method of the present invention is easier to introduce into the laser sensing device and the laser sensing device is easier to maintain.

One aspect of the present invention is to provide a laser sensing device, which includes a laser source, an image sensor and a processor. The laser source is configured to emit laser light toward a target object and move along a first direction. The image sensor is configured to capture an object image toward a spatial region where the target object is located. The processor is configured to perform the following operations: calculate the reference object distance of the target object relative to the image sensor from the object image; calculate the reference plane view according to the reference object distance and the nearest view, the farthest view and the depth of field of the image sensor; and scale and adjust the length of the object image in a second direction perpendicular to the first direction according to the reference view, the reference plane view, the target image aspect ratio of the reference image corresponding to the reference view, and the physical aspect ratio of the target object, wherein the target image aspect ratio is substantially equal to the ratio of the length of the reference image in the second direction to the length of the first direction, and the physical aspect ratio is substantially equal to the ratio of the length of the physical body of the target object in the second direction to the length of the first direction.

According to an embodiment of the present invention, when the reference field of view is the closest field of view, the processor scales and adjusts the length ratio of the object image in the second direction by R=(FOV(z ₁ )/FOV(d ₁ ))(D _o /D _d1 ); wherein FOV(z ₁ ), FOV(d ₁ ), D _o and D _d1 respectively represent the reference plane field of view, the closest field of view, the entity aspect ratio and the target object image aspect ratio of the reference image corresponding to the reference image when the reference field of view is the closest field of view.

According to an embodiment of the present invention, when the reference field of view is the farthest field of view, the processor scales and adjusts the length ratio of the object image in the second direction by R=(FOV(z ₁ )/FOV(d ₂ ))(D _o /D _d2 ); wherein FOV(z ₁ ), FOV(d ₂ ), D _o and D _d2 respectively represent the reference plane field of view, the farthest field of view, the entity aspect ratio and the target image aspect ratio of the reference image corresponding to the farthest field of view.

According to an embodiment of the present invention, the processor calculates the reference object distance by: capturing multiple depth values corresponding to the object image; and selecting the depth value corresponding to the largest number from these depth values as the reference object distance.

According to an embodiment of the present invention, the processor selects the depth value corresponding to the largest number from these depth values, including: substituting these depth values into a Gaussian model or a Gaussian mixture model to calculate and obtain a reference object distance.

；其中z₁、FOV(d₁)、FOV(d₂)以及r分別代表參考物距、最近視野、最遠視野以及景深。 According to one embodiment of the present invention, the reference plane field of view calculated by the processor

; wherein z ₁ , FOV(d ₁ ), FOV(d ₂ ) and r represent the reference object distance, the closest field of view, the farthest field of view and the depth of field respectively.

Another aspect of the present invention is to provide a calibration method for a laser sensing device, which comprises: using the laser sensing device to emit a laser beam toward a target object, and moving along a first direction, and capturing an object image of a spatial region where the target object is located; calculating a reference object distance of the target object relative to the laser sensing device from the object image; calculating a reference plane field of view based on the reference object distance and the nearest field of view, the farthest field of view, and the depth of field of the laser sensing device; and calculating a reference plane field of view based on the reference object distance and the nearest field of view, the farthest field of view, and the depth of field of the laser sensing device. The length of the object image in the second direction perpendicular to the first direction is scaled and adjusted according to the reference field of view, the reference plane field of view, the target image aspect ratio of the reference image corresponding to the reference field of view, and the physical aspect ratio of the target, wherein the target image aspect ratio is substantially equal to the ratio of the length of the reference image in the second direction to the length in the first direction, and the physical aspect ratio is substantially equal to the ratio of the length of the entity of the target in the second direction to the length in the first direction.

According to an embodiment of the present invention, when the reference field of view is the closest field of view, it further includes: scaling and adjusting the length ratio of the object image in the second direction R=(FOV(z ₁ )/FOV(d ₁ ))(D _o /D _d1 ); wherein FOV(z ₁ ), FOV(d ₁ ), D _o and D _d1 respectively represent the reference plane field of view, the closest field of view, the entity aspect ratio and the target image aspect ratio of the reference image corresponding to the target object in the closest field of view.

According to an embodiment of the present invention, when the reference field of view is the farthest field of view, it further includes: scaling and adjusting the length ratio of the object image in the second direction R=(FOV(z ₁ )/FOV(d ₂ ))(D _o /D _d2 ); wherein FOV(z ₁ ), FOV(d ₂ ), D _o and D _d2 respectively represent the reference plane field of view, the farthest field of view, the entity aspect ratio and the target image aspect ratio of the reference image corresponding to the farthest field of view.

；其中z₁、FOV(d₁)、FOV(d₂)以及r分別代表參考物距、最近視野、最遠視野以及景深。 According to one embodiment of the present invention, the calculated reference plane view

; wherein z ₁ , FOV(d ₁ ), FOV(d ₂ ) and r represent the reference object distance, the closest field of view, the farthest field of view and the depth of field respectively.

100: Laser sensor device

110: Laser source

120: Image sensor

130: Processor

200: Target

300: Calibration method of laser sensing device

FOV(d ₁ ): closest field of view

FOV(d ₂ ): Maximum field of view

l _cx : length

l _fx :length

l _ox : length

l _cy : length

l _fy : length

l _oy : length

r: Depth of field

S310, S320, S330: Steps

z ₁ : reference object distance

In order to more fully understand the embodiments and their advantages, reference is now made to the following description in conjunction with the attached figures, wherein: FIG. 1 is a schematic diagram of a laser sensing device scanning a target according to an embodiment of the present invention; FIG. 2 is a schematic diagram of the depth of field and the field of view of the lens of the image sensor of the laser sensing device according to an embodiment of the present invention; FIG. 3 is a plan view of the target; FIG. 4 is a schematic diagram of an object image of the target captured by the image sensor in the nearest field of view; FIG. 5 is a schematic diagram of an object image of the target captured by the image sensor in the farthest field of view; FIG. 6 is a schematic diagram of the process of the calibration method of the laser sensing device according to an embodiment of the present invention; and FIG. 7 is a schematic diagram of multiple depth values corresponding to the object image and the quantities corresponding to these depth values.

The following is a detailed discussion of embodiments of the present invention. However, it is understood that the embodiments provide many applicable concepts that can be implemented in a variety of specific contexts. The embodiments discussed and disclosed are for illustrative purposes only and are not intended to limit the scope of the present invention.

FIG1 is a schematic diagram of a laser sensing device 100 according to an embodiment of the present invention. The laser sensing device 100 is used to scan a target object 200 having a depth variation on the surface, and includes a laser source 110, an image sensor 120, and a processor 130.

As shown in FIG. 1 , the laser source 110 is configured to emit a line laser light toward the target 200 and move along the direction Y so that the line laser light completely scans the target 200, wherein the line laser light extends along the direction X perpendicular to the direction Y. The laser sensing device 100 may also include a moving mechanism (not shown); the moving mechanism drives the laser source 110 and the image sensor 120, and the laser source 110 and the image sensor 120 are able to perform laser sensing on the target 200 along the direction Y. In some embodiments, the target 200 may be placed on a moving platform; the moving platform drives the target 200 to move in a direction opposite to the direction Y, and the laser source 110 and the image sensor 120 are also able to perform laser sensing on the target 200 along the direction Y.

The image sensor 120 is configured to capture an object image facing the spatial area where the target 200 is located, wherein the object image is the image captured by the image sensor 120 after the line laser light completely scans the target 200. The image sensor 120 includes a lens and a photosensitive element (not shown), wherein the lens is used to focus light to the photosensitive element, and the photosensitive element is used to sense and receive the line laser light reflected from the target 200 and focused by the lens, thereby generating an image. The photosensitive element can be a charge coupled device (CCD), a complementary metal oxide semiconductor (CMOS) element, or other photosensitive elements.

The processor 130 is electrically connected to the laser source 110 and the image sensor 120. The processor 130 can control the laser source 110 to project a laser beam to the target 200, and receive an object image from the image sensor 120, and then calculate the three-dimensional structure information of the target 200 from the object image, wherein the three-dimensional structure information includes two-dimensional size information and depth information.

FIG2 is a schematic diagram of the depth of field r and the field of view of the lens of the image sensor 120 of the laser sensing device 100 according to an embodiment of the present invention. The range of the closest field of view FOV ( _d1 ) and the farthest field of view FOV ( _d2 ) corresponding to the depth of field r of the lens of the image sensor 120 is trapezoidal. As shown in FIG1 and FIG2, if the distance (depth) between the target object 200 and the lens (in the direction Z) is within the range of the depth of field r, the linear laser light reflected from the target object 200 can be focused by the lens to the photosensitive element to form an image. However, as the distance between the target object 200 and the lens changes, the resolution of the object image captured by the image sensor 120 in the direction X will also change and produce deformation.

FIG. 3 is a plan view of the target object 200 , and FIG. 4 and FIG. 5 are schematic diagrams of the object images of the target object 200 captured by the image sensor 120 in the range of the closest field of view FOV (d ₁ ) and the farthest field of view FOV (d ₂ ), respectively. In the range of the closest field of view FOV (d ₁ ), the length of the object image in the direction X, l _cx (the length of the object image in the closest field of view in the direction X) is longer than the length of the target 200 in the direction X, l _ox (the length of the target in the direction X), and the length of the object image in the direction Y, l _cy (the length of the object image in the closest field of view in the direction Y) is equal to the length of the target 200 in the direction Y, l _oy (the length of the target in the direction Y); in the range of the farthest field of view FOV (d ₂ ), the length of the object image in the direction X, l _fx (the length of the object image in the farthest field of view in the direction X) is shorter than the length of the target 200 in the direction X, l _ox , and the length of the object image in the direction Y, l _fy (the length of the object image in the farthest field of view in the direction Y) is equal to the length of the target 200 in the direction Y, l Therefore, the length of the object image in the direction X will produce different proportions of deformation as the distance between the target object 200 and the lens _changes .

FIG6 is a flowchart of a laser sensing device calibration method 300 according to an embodiment of the present invention. The laser sensing device calibration method 300 described below can be executed by the processor 130 in the laser sensing device 100, but a person skilled in the art can also use other processors to execute the laser sensing device calibration method 300 according to the following description. First, in step S310, a reference object distance z ₁ of the target object 200 relative to the image sensor 120 of the laser sensing device 100 is calculated from the object image.

FIG. 7 is a schematic diagram of multiple depth values (distances) corresponding to an object image and the quantities corresponding to these depth values. Further, in step S310, multiple depth values corresponding to the object image may be captured first. That is, these depth values of the object image that completely scans the target object 200 are captured. Then, the depth value corresponding to the largest quantity is selected from these depth values as the reference object distance z ₁ . That is, the distance between the target object 200 and the image sensor 120 is determined from the concentration of these depth values of the object image, thereby finding the reference object distance z ₁ . For example, these depth values of the object image can be substituted into a Gaussian model or a Gaussian mixture model to find the maximum value of the statistical histogram using the Gaussian model or the Gaussian mixture model, thereby obtaining a reference object distance z ₁ . As shown in FIG7 , these depth values corresponding to the object image present a Gaussian distribution, and the reference object distance z ₁ is the high point of the Gaussian distribution, i.e., the depth value with the largest number of corresponding values among these depth values.

其中，FOV(z₁)、z₁、FOV(d₁)、FOV(d₂)以及r分別代表參考平面視野、參考物距、最近視野、最遠視野以及景深。 With reference to FIG. 2 , then, in step S320 , the reference plane field of view FOV (z ₁ ) corresponding to the reference object distance z ₁ is calculated according to the reference object distance z 1 and the closest field of view FOV (d _{1 ), the farthest field of view FOV (d 2 )} and the depth of field r of the image sensor 120 . The formula of the reference plane field of view is as follows:

Here, FOV(z ₁ ), z ₁ , FOV(d ₁ ), FOV(d ₂ ) and r represent the reference plane field of view, the reference object distance, the closest field of view, the farthest field of view and the depth of field, respectively.

In step S330, the length of the object image in the direction X is scaled and adjusted according to the reference field of view, the reference plane field of view FOV (z ₁ ), the target image aspect ratio of the reference image corresponding to the target 200 in the reference field of view, and the physical aspect ratio of the target 200, wherein the reference field of view is the nearest field of view FOV (d ₁ ) or the farthest field of view FOV (d ₂ ), the reference image is the image of the target 200 captured by the image sensor 120 when in the reference field of view, the target image aspect ratio is substantially equal to the ratio of the length of the reference image in the direction X to the length in the direction Y, and the physical aspect ratio is substantially equal to the ratio of the length of the physical body of the target 200 in the direction X to the length in the direction Y.

For example, when the reference field of view is the closest field of view FOV( _d1 ), the processor 130 scales and adjusts the length ratio of the object image in the direction X by the following formula: R=(FOV( _z1 )/FOV( _d1 ))( _D0 / _Dd1 ), wherein R, _D0 and _Dd1 respectively represent the ratio, the entity aspect ratio and the target image aspect ratio of the reference image corresponding to the reference image when the reference field of view is the closest field of view.

Therefore, by scaling and adjusting the length of the object image in the direction _X using the ratio of the reference plane field of view FOV (z ₁ ) to the benchmark field of view (closest field of view FOV (d 1 )) and the ratio _of the entity aspect ratio Do to the target image aspect ratio D _d1 (target image aspect ratio D _d1 at the closest field of view FOV (d ₁ )), the object image can be adjusted in two-dimensional size to the same ratio as the entity of the target 200 .

For example, in Figure 3, the physical aspect ratio _Do (length l _ox /length l _oy ) of the target display "ABC" is 3/1. In Figure 4, the target image aspect ratio D _d1 (length l _cx /length l _cy ) of the reference image "ABC" corresponding to the closest field of view FOV (d ₁ ) is 6/1. To adjust the target image aspect ratio D _d1 of the reference image corresponding to the closest field of view FOV (d ₁ ) to be consistent with the physical aspect ratio _Do, it needs to be reduced by two times.

When the reference field of view is the farthest field of view FOV( _d2 ), the processor 130 scales and adjusts the length ratio of the object image in the direction X by the following formula: R=(FOV( _z1 )/FOV( _d2 ))( _D0 / _Dd2 ), wherein _Dd2 represents the target image aspect ratio of the reference image corresponding to the farthest field of view.

Similar to the above description, the length of the object image in the direction _X is scaled and adjusted using the ratio of the reference plane field of view FOV (z ₁ ) to the benchmark field of view (the farthest field of view FOV (d 2 )) and the ratio _of the entity aspect ratio Do to the target image aspect ratio D _d2 (the target image aspect ratio D _d2 at the farthest field of view FOV (d ₂ )). The object image can be adjusted in two-dimensional size to the same ratio as the entity of the target 200.

For example, in FIG5 , the target image aspect ratio D _d2 (length l _fx /length l _fy ) of the reference image "ABC" corresponding to the farthest field of view FOV (d ₂ ) is 1.5/1. To adjust the target image aspect ratio D _d2 of the reference image corresponding to the farthest field of view FOV (d ₂ ) to be consistent with the physical aspect ratio D _o , it is necessary to magnify it twice.

It should be noted that the depth values captured by the processor 130, as well as the calculation of the reference plane field of view FOV (z ₁ ), the scaling ratio R, etc., can be directly stored and calculated in bits or only the relative ratio can be calculated without first converting to actual physical values (absolute spatial distance) for processing, thereby greatly reducing the processing time of the processor 130.

In summary, the laser sensing device and the calibration method of the laser sensing device of the present invention can avoid the tedious calculation process of precise calibration by calculating the absolute distance value in space, and can calibrate the deformation of the image in the horizontal direction when referring to the object distance, and is easier to maintain and introduce for use.

Although the present disclosure has been disclosed as above by way of embodiments, it is not intended to limit the present disclosure. Any person with ordinary knowledge in the relevant technical field may make some changes and modifications within the spirit and scope of the present disclosure. Therefore, the protection scope of the present disclosure shall be subject to the scope defined by the attached patent application.

100: Laser sensor device

110: Laser source

120: Image sensor

130: Processor

200: Target

Claims (9)

A laser sensing device comprises: A laser source configured to emit a line of laser light toward a target object and move along a first direction; An image sensor configured to capture an object image toward a spatial region where the target object is located; and A processor configured to perform the following operations: Calculate a reference object distance of the target object relative to the image sensor from the object image; Calculate a reference plane field of view based on the reference object distance and a nearest field of view, a farthest field of view and a depth of field of the image sensor; and The length of the object image in a second direction perpendicular to the first direction is scaled and adjusted according to a reference field of view, the reference plane field of view, a target image aspect ratio of a reference image corresponding to the reference field of view, and a physical aspect ratio of the target, wherein the target image aspect ratio is substantially equal to the ratio of the length of the reference image in the second direction to the length in the first direction, and the physical aspect ratio is substantially equal to the ratio of the length of a physical body of the target in the second direction to the length in the first direction; wherein the processor calculates the reference object distance including: capturing a plurality of depth values corresponding to the object image; and selecting the depth value corresponding to the largest number from the depth values as the reference object distance. The laser sensing device as claimed in claim 1, wherein when the reference field of view is the closest field of view, the processor scales and adjusts a ratio of the length of the object image in the second direction ; wherein FOV(z ₁ ), FOV(d ₁ ), D _o and D _d1 respectively represent the reference plane field of view, the closest field of view, the entity aspect ratio and the target object image aspect ratio of the reference image corresponding to the reference image when the reference field of view is the closest field of view. The laser sensing device as claimed in claim 1, wherein when the reference field of view is the farthest field of view, the processor scales and adjusts a ratio of the length of the object image in the second direction ; wherein FOV(z ₁ ), FOV(d ₂ ), D _o and D _d2 respectively represent the reference plane field of view, the farthest field of view, the entity aspect ratio and the target image aspect ratio of the reference image corresponding to the farthest field of view. The laser sensing device as described in claim 1, wherein the processor selects the depth value corresponding to the largest number from the depth values, including: Substituting the depth values into a Gaussian model or a Gaussian mixture model to obtain the reference object distance. The laser sensing device as claimed in claim 1, wherein the reference plane field of view calculated by the processor ; wherein z ₁ , FOV(d ₁ ), FOV(d ₂ ) and r represent the reference object distance, the closest field of view, the farthest field of view and the depth of field respectively. A calibration method for a laser sensing device, comprising: Using a laser sensing device to emit a line of laser light toward a target object, and moving along a first direction, and capturing an object image in a spatial region where the target object is located; Calculating a reference object distance of the target object relative to the laser sensing device from the object image; Calculating a reference plane field of view based on the reference object distance and a closest field of view, a farthest field of view, and a depth of field of the laser sensing device; and The length of the object image in a second direction perpendicular to the first direction is scaled and adjusted according to a reference field of view, the reference plane field of view, a target image aspect ratio of a reference image corresponding to the reference field of view, and a physical aspect ratio of the target, wherein the target image aspect ratio is substantially equal to the ratio of the length of the reference image in the second direction to the length in the first direction, and the physical aspect ratio is substantially equal to the ratio of the length of a physical body of the target in the second direction to the length in the first direction; Wherein calculating the reference object distance includes: Capturing a plurality of depth values corresponding to the object image; and Selecting the depth value corresponding to the largest number from the depth values as the reference object distance. The calibration method of the laser sensing device as described in claim 6, wherein when the reference field of view is the closest field of view, the method further comprises: scaling and adjusting a ratio of the length of the object image in the second direction ; wherein FOV(z ₁ ), FOV(d ₁ ), D _o and D _d1 represent the reference plane field of view, the closest field of view, the entity aspect ratio and the target image aspect ratio of the reference image corresponding to the target in the closest field of view respectively. The calibration method of the laser sensing device as described in claim 6, wherein when the reference field of view is the farthest field of view, the method further comprises: scaling and adjusting a ratio of the length of the object image in the second direction ; wherein FOV(z ₁ ), FOV(d ₂ ), D _o and D _d2 respectively represent the reference plane field of view, the farthest field of view, the entity aspect ratio and the target image aspect ratio of the reference image corresponding to the farthest field of view. A method for calibrating a laser sensing device as described in claim 6, wherein the reference plane field of view calculated ; wherein z ₁ , FOV(d ₁ ), FOV(d ₂ ) and r represent the reference object distance, the closest field of view, the farthest field of view and the depth of field respectively.

Images