TWI879332B - Method for calculating actual distance and angle of target object of tracking device - Google Patents

Abstract

The present invention discloses a method for calculating the actual distance and angle of a target object with a tracking device. The method comprises the following steps: an image capture step, an image segmentation step, a coordinate transformation step, a coordinate positioning step, a coordinate mapping step, and a coordinate system transformation step. Initially, the method involves capturing a target-following image containing the target object using the tracking device. Subsequently, the image undergoes segmentation, coordinate transformation, coordinate positioning, coordinate mapping, and coordinate system transformation. Through these steps, the method enables the application of an imaging device to capture images of the target being followed and perform image coordinate transformation without the need for complex computations. This facilitates the precise calculation of the relative positional coordinates between the actual target and the device. Consequently, the method calculates the actual distance and angle, allowing the device to automatically follow individuals or movable target objects, thereby assisting in the handling of heavy objects.

Description

Method for calculating actual distance and angle of target object by following device

The present invention relates to a calculation method of a following device, and more particularly to a method of calculating the actual distance and angle of a target object by a following device.

In some automation fields, such as automatic following systems, automatic traffic signal management systems, and automatic watering systems, it is often necessary to automatically calculate the actual distance and angle between the imaging device and a target object in real time, so that the automatic following system can automatically drive the vehicle to follow the target object, the traffic signal of the automatic traffic signal management system can automatically determine the distance of the car on the road to switch to the appropriate signal under the premise of safety, and the automatic watering system can determine the relative distance and angle of the plants to be watered within the range to water them.

There are many different methods for calculating the actual distance and angle of a target object. For example, Taiwan Patent No. I407081 discloses a distance measuring device, which reduces the influence of the background light and the flash light by removing the part corresponding to the background light and the flash light from the light sensing signal generated by an image sensor in the distance measuring device, and can calculate a correction parameter that can correct an assembly error angle of the distance measuring device according to an imaging position of a reflected light obtained by measuring a calibration object with a known distance, and then correctly calculate the distance to be measured according to the correction parameter. However, the known technology is to calculate the energy received after the light-emitting element emits detection light to the object to be measured to generate reflected light to reversely infer the distance. Therefore, the known technology can only calculate the relative distance but cannot know the relative angle.

Another known method, such as Taiwan Patent No. I777791, discloses a method for verifying the dynamic virtual image display distance of a human-machine interface, comprising the steps of: establishing a test base map database; displaying a first test base map in the test base map database by a display element; projecting a first image to be tested onto a superimposed image element by a human-machine interface module, wherein the first image to be tested corresponds to a first virtual image display distance to be tested, and the first virtual image to be tested corresponds to a first virtual image display distance to be tested. The image display distance is the same as the first reference virtual image display distance corresponding to the first test base map; the first test base map and the first image to be tested are captured by an image capture module; the reliability of the first image to be tested and the first test base map is evaluated by a recognition module; and the overlap rate of the first image to be tested and the first test base map is calculated by a processing module to verify the accuracy of the first virtual image display distance to be tested of the human-machine interface. However, the known technology mainly uses an image capture module to capture the first test background image and the first image to be tested, and then uses a recognition module to calculate the overlap rate of the first image to be tested and the first test background image to calculate the distance of the object to be tested. Therefore, the known technology can only calculate the distance of the object to be tested, and cannot calculate the relative angle between the two.

Another known method, such as Taiwan Patent No. I746234, discloses a method for measuring the distance and positioning of a target on the sea surface by an unmanned helicopter, which includes: an altitude pressure sensing module, a GPS module, an image module, a signal processing module, a flight control module, and a transmission module. These components are integrated into the unmanned aircraft. When the target is found, the altitude pressure sensing module can know the height of the unmanned aircraft from the sea level; the image module The plane distance between the drone and the target can be calculated by the angle at which the camera illuminates the target, and the angle of the target can be obtained by the angle difference of the rotation of the camera lens of the GPS module and the image module. The actual GPS coordinate position of the target can be calculated from the GPS coordinate position of the drone itself, the plane distance, and the angle between the target and the drone, and then the relative coordinate position and distance between the target and the drone can be grasped. However, the known technology mainly calculates the relative position distance by illuminating the target with the camera and locating the target with GPS, and then sending the coordinates back to the carrier device. The required equipment is relatively complex and cannot be used in places without GPS signals or with poor signals.

In view of the above and other problems, the prior art needs to be improved.

The purpose of this invention is to provide a method for calculating the actual distance and angle of a target object by a tracking device to improve the shortcomings of the prior art.

Another purpose of this invention is to provide a method for calculating the actual distance and angle of a target object by a tracking device. By using the image frame obtained by the camera to perform image coordinate conversion, the relative position coordinates between the actual target and the body can be calculated, and then the actual distance and angle can be calculated.

Another purpose of this invention is to provide a method for calculating the actual distance and angle of a target object by a following device, which can accurately calculate the actual distance and angle without complicated calculations, making it convenient for the following device to move and keep following.

Another purpose of this invention is to provide a method for calculating the actual distance and angle of a target object by a following device, which can be applied to unmanned intelligent devices such as cargo robots, golf ball pickers, golf equipment carriers or automatic lawn mowers, and can allow the device to automatically follow a person or a movable target object through image recognition.

To achieve the above and other purposes, the present invention provides a method for calculating the actual distance and angle of a target object by a tracking device, comprising: an image capture step, in which a camera unit of a tracking device captures a tracking target image, wherein the tracking device has a device coordinate, and the tracking target image is a two-dimensional plane image containing a target object, and the tracking target image has a first origin coordinate; an image segmentation step, identifying the position of the target object in the tracking target image, performing image segmentation on the tracking target image, and obtaining a comparison image, wherein the image of the target object is located in the middle of the comparison image; a coordinate conversion step; A step of linearly transforming the first origin coordinates by using the image sizes of the tracking target image and the comparison image to obtain a second origin coordinate in the comparison image; a coordinate positioning step of setting a positioning coordinate in the comparison image by using the coordinate value of the second origin coordinate; a coordinate correspondence step of calculating the coordinate correspondence relationship between the device coordinates and the positioning coordinates through the curve fitting relationship between the axis distances of the device coordinates and the positioning coordinates; and a coordinate system transformation step of calculating the actual distance and actual angle between the tracking device and the target object according to the coordinate correspondence relationship between the positioning coordinates and the device coordinates.

In one embodiment, it further includes: a following step, according to the coordinate correspondence between the device coordinates and the positioning coordinates, the positioning coordinate value corresponding to the origin of the device coordinates is calculated, and the positioning coordinate value is converted into an actual distance and angle, and an automatic driving function of the following device is provided to perform corresponding driving direction and distance corrections, and the following device automatically drives to follow the target object.

In one embodiment, in some embodiments, the device coordinates, the first origin coordinates, the second origin coordinates, and the positioning coordinates are rectangular coordinate systems.

In one embodiment, the coordinate origin of the coordinate positioning step is set at the bottom middle position of the comparison image.

In one embodiment, the target object is a movable object.

In one embodiment, the coordinate corresponding step includes pre-measuring the actual distance between a lens of the camera unit and a number of the positioning coordinates at different positions.

The directions or similar terms described throughout the present invention, such as front, back, left, right, top, bottom, inside, outside, side, etc., are mainly with reference to the directions of the drawings. Each direction or similar terms are only used to assist in explaining and understanding the various embodiments of the present invention, and are not used to limit the present invention.

The articles used in the elements and components described in the present invention, such as a or the, are only for the convenience of use or simplified description and should be interpreted as including one or at least one, and a single concept also includes a plural concept unless it is obvious that there is a different meaning.

In order to make the above and other purposes, effects, and features more clearly understood, some preferred embodiments are specifically cited below, and detailed descriptions are made with reference to the attached drawings. Without departing from the spirit of the creation, this creation has a variety of implementation methods, and is not limited to the details specifically described in the implementation methods below, and the drawings may not be drawn according to the actual scale, but are provided only for illustrating the embodiments.

FIG1 is a flowchart of an embodiment of a method for calculating the actual distance and angle of a target object by the creative follower device. Referring to FIG1, the method for calculating the actual distance and angle of a target object in the embodiment comprises: an image capturing step S10, an image segmentation step S11, a coordinate conversion step S12, a coordinate positioning step S13, a coordinate corresponding step S14 and a coordinate conversion step S15. The image capturing step S10 is performed by a controller installed in the follower device. A camera unit 11 of the device 10, such as a lens, captures a tracking target image 20 of a target object that the tracking device 10 is to follow in real time, wherein the tracking device 10 defines a device coordinate system, and the tracking target image 20 is a two-dimensional plane image containing the image of the target object, and the upper left corner of the tracking target image 20 is a first origin coordinate C1. The image segmentation step S11 is to identify the position of the target object in the tracking target image 20, perform image segmentation on the tracking target image 20, and obtain a comparison image 30 containing the image of the target object, and the image of the target object is located in the middle position of the comparison image 30. The coordinate conversion step (S12) utilizes the linear conversion relationship between the image sizes of the tracking target image 20 and the comparison image 30 to linearly convert the first origin coordinate C1, and obtain a second origin coordinate in the comparison image 30. The coordinate positioning step S13 utilizes the coordinate value of the second origin coordinate C2 to set a positioning coordinate C3 in the comparison image 30. The coordinate correspondence step S14 pre-measures the actual distance between a lens of the camera unit 11 and the positioning coordinates at different positions, and calculates the coordinate correspondence relationship between the device coordinates and the positioning coordinates through the curve fitting relationship between the X and Y axis distances of the device coordinates and the positioning coordinates. The coordinate system transformation step S15 calculates the actual distance and actual angle between the following device 10 and the target object according to the coordinate correspondence between the positioning coordinates and the device coordinates.

FIG2 is a flow chart of an embodiment of a method for automatically following the actual distance and angle of the target object calculated by the tracking device of the present invention. Please refer to FIG2. The present embodiment mainly includes the application of the aforementioned coordinate system conversion step S15 after calculating the actual distance and actual angle between the tracking device 10 and the target object, that is, after the coordinate system conversion step S15, it further includes: a following step S16 is to use an automatic driving function of the tracking device, according to the coordinate correspondence between the device coordinates and the positioning coordinates, calculate the positioning coordinate value corresponding to the origin of the device coordinates, and convert the positioning coordinate value (such as a rectangular coordinate system) into an actual distance and angle (such as a polar coordinate system ( )) provides the automatic driving system with a corresponding correction of the driving direction and distance, and the automatic driving system follows the movable target object while maintaining a preset distance with the following device.

In other words, in the tracking target image captured by the camera unit 11, the position of the target object has a certain correlation with the actual angle and actual distance of the tracking target from the camera unit 11. By finding this correlation, the actual angle and distance can be further found through the information of the position of the target object in the tracking target image captured by the camera. Since the specifications and resolutions of each camera lens are different, it is necessary to find the curve fitting formula for the captured image of the camera lens used by the camera unit 11 in combination with the actual distance measurement value.

FIG3 is a schematic diagram of the relationship between the coordinates of the creation device, the first origin coordinates, the second origin coordinates and the positioning coordinates. As shown in FIG3, the image pixel origin of the tracking target image 20, that is, the first origin coordinate C1 is the upper left corner of the image, and the coordinate system of the tracking target image 20 uses ( ) is defined, or called the image pixel large coordinate system ( ).

However, since the space captured by the actual image does not need to use the entire image, the image pixels close to the center of the tracking device are generally used, that is, the part near the small comparison box in the lower half of Figure 3, whose upper left corner is the second origin coordinate C2. The comparison image coordinate system uses ( ) is defined, or called the image pixel small coordinate system ( ).

Image pixel coordinate system ( ) and the image pixel small coordinate system ( ) is a linear transformation relationship. Once the sizes of the large image and the small image are determined, the transformation relationship between the two can be determined.

However, since the following device 10 is defined using the device coordinate system (x, y) or referred to as the device coordinate system (x, y), the image pixel small coordinate system ( ) is preferably further converted to the positioning coordinate system ( ), or image ( ) coordinate system, that is, the image pixel small coordinate system ( ) and images ( ) coordinate systems, as long as the bottom center of the image is determined as the positioning coordinate origin C3, the conversion relationship can be obtained, as shown in Figure 3.

Next, determine the device coordinate system (x, y) and image ( ) coordinate system, it can determine the relationship between the device coordinate system (x, y) and the image pixel large coordinate system ( )’s relevance.

FIG4 is a diagram showing the image position and actual distance verification of the target object of the present invention, and FIG5 is a diagram showing the curve fitting relationship between the device coordinate system (x, y) and the measured distance and the positioning coordinate system of the present invention. Please refer to FIG4 and FIG5 to find out the relationship between the device coordinate system (x, y) and the image pixel large coordinate system ( ) can be used to verify the image position and distance as shown in Figure 4. For a known distance between the camera unit (11) and the venue (F), the relationship between the device coordinate system (x, y) and the image ( ) coordinate systems.

Table 1 shows the X distance and image of the device coordinate system of the tracking device ( ) coordinate systems. Table 1 X(cm) =0 =80 =160 =240 =320 =400 =360 0 30.2 60.5 90.7 121 151.2 =300 0 21.2 42.5 63.7 84.9 106.2 =240 0 15.6 31.3 46.9 62.6 78.2 =180 0 12.3 24.6 36.9 49.2 61.5 =120 0 10.2 20.4 30.6 40.9 51.1 =60 0 8.8 17.6 26.5 35.3 44 =0 0 7.75 15.5 23.5 31 38.75

According to the relationship in Table 1, it is found that the X distance and the image ( ) The coordinate system relationship can be assumed as follows: Mathematical formula 1 =

Table 2 shows the When the The actual measured distance is shown in Table 3. The curve fitting value when the diameter is 80 cm: Table 2 Measured distance(cm) 360 30.2 300 21.2 240 15.6 180 12.3 120 10.2 60 8.8 0 7.75 Table 3 Curve Fitting Value 80 360 28.214 80 300 21.32 80 240 15.866 80 180 11.852 80 120 9.278 80 60 8.144 80 0 8.45

According to Table 2, Table 3 and Figure 5, the device coordinate system X distance and image of the tracking device ( ) The relationship between the curve fitting of the coordinate system can be found: Mathematical formula 2 Mathematical formula 3

Table 4 shows the Y distance and image of the device coordinate system of the tracking device ( ) coordinate systems. Table 4 (=95)(cm) =0 =80 =160 =240 =320 =400 =0 0 0 0 0 0 0 =60 13 13 13 13 13 13 =120 30 30 30 30 30 30 =180 55 55 55 55 55 55 =240 95.5 95.5 95.5 95.5 95.5 95.5 =300 163 163 163 163 163 163

According to the relationship in Table 4, it is found that the Y distance and the image ( ) The coordinate system relationship can be assumed as follows: Mathematical formula 4

The mathematical formula 4 represents the Y distance of the tracking device coordinate system and the image coordinate system. The impact is not relevant and can be discarded without consideration.

FIG6 is a diagram showing the Y distance and image of the device coordinate system of the tracking device ( ) coordinate system. Table 5 shows the curve fitting results between Table 6 shows the fitting curve values corresponding to Table 5. (cm) Measured distance(cm) 0 0 60 13 120 30 180 55 240 95.5 300 163 Table 6 Curve Fitting Value 3.55 8.38 26.17 56.92 100.63 157.3

Through the data in Table 4, Table 5 and Table 6, we can fit the mathematical formula 5

From equations 2, 3, and 5, we can find the device coordinate system (x, y) and the image ( ) coordinate systems. That is, as long as we know the image ( ) coordinates, we can get the (x,y) coordinates of the device coordinate system. This is further used to control the ( ) The polar coordinate system can also be directly converted and calculated from the rectangular coordinate system, and then a command is given to the following device 10 to complete the required driving direction and distance.

Some features of the present invention are: in some embodiments, an automatic following device or vehicle can be provided. For example, when a heavy object needs to be transported from place A to place B, the person responsible for transporting the object does not need to operate the vehicle. He only needs to set the following target (such as the person responsible for transporting the object) and place the object on the following vehicle. When the person responsible starts to move from place A to place B, the following device vehicle will lock the image of the person responsible to track the movement. In addition, this application can also be applied to the needs of receiving, sending and placing documents or components in offices or large factories.

The present invention has been disclosed by using the above preferred embodiments, but they are not used to limit the present invention. Those with ordinary knowledge in the technical field to which the present invention belongs can clearly understand that the present invention is not limited to the details of the above illustrative implementation methods. The implementation methods are only for explaining the present invention, not for limiting the present invention. The present invention is based on the scope of the patent application, not on the above description. The meaning of the scope of the patent application and its equivalent scope are all within the scope of the patent right of the present invention.

(S10~S16): Steps (10): Following device (11): Camera unit (20): Following target image (30): Comparison image (C1): First origin coordinates (C2): Second origin coordinates (C3): Positioning coordinate origin (F): Venue

FIG. 1 is a flowchart of an embodiment of a method for calculating the actual distance and angle of a target object by the creative tracking device. FIG. 2 is a flowchart of an embodiment of a method for automatically following the actual distance and angle of a target object calculated by the creative tracking device. FIG. 3 is a schematic diagram of the associated positions of the device coordinates, the first origin coordinates, the second origin coordinates and the positioning coordinates of the creative device. FIG. 4 is a schematic diagram of the image position and actual distance verification of the target object of the creative device. FIG. 5 is a schematic diagram of the curve fitting relationship between the measured distance of the device coordinate system X and the positioning coordinate system of the first embodiment of the creative device. FIG. 6 is a schematic diagram of the curve fitting relationship between the measured distance of the device coordinate system Y and the positioning coordinate system of the first embodiment of the creative device.

(S10~S15): Steps

Claims (5)

A method for calculating the actual distance and angle of a target object by a tracking device comprises: an image capturing step (S10), wherein a camera unit (11) of a tracking device (10) captures a tracking target image (20), wherein the tracking device (10) has a device coordinate (x, y), the tracking target image (20) is a two-dimensional plane image containing a target object, and the tracking target image (20) has a first origin coordinate ( )(C1); an image segmentation step (S11), identifying the position of the target object in the tracking target image (20), performing image segmentation on the tracking target image (20), and obtaining a comparison image (30), wherein the image of the target object is located in the middle of the comparison image (30); a coordinate conversion step (S12), using the image sizes of the tracking target image (20) and the comparison image (30), converting the first origin coordinate ( After linear transformation (C1), a second origin coordinate is obtained in the comparison image ( )(C2); a coordinate positioning step (S13), using the second origin coordinate ( )(C2) coordinate value( ) sets a positioning coordinate ( )(C3); A coordinate correspondence step (S14), through the distance of each axis of the device coordinate (x, y) and the positioning coordinate ( )(C3) to calculate the device coordinates (x, y) and the positioning coordinates ( )(C3) coordinate correspondence; and a coordinate system transformation step (S15), according to the positioning coordinates ( The actual distance and actual angle between the following device (10) and the target object are calculated by using the coordinate correspondence between (C3) and the device coordinate (x, y). The above calculation is based on mathematical formulas 1 to 5. Mathematical formula 1 is = , Mathematical formula 2 is , Mathematical formula 3 is , Mathematical formula 4 is , mathematical formula 5 is . The method for calculating the actual distance and angle of a target object by a tracking device as described in claim 1 further comprises: a following step (S16), according to the device coordinates (x, y) and the positioning coordinates ( )(C3) coordinate correspondence relationship, calculate the positioning coordinate value corresponding to the device coordinate origin, and convert the positioning coordinate value into actual distance and angle, provide an automatic driving function of the following device to perform corresponding driving direction and distance correction, and automatically drive the following device to follow the target object. The method for calculating the actual distance and angle of a target object by a tracking device as described in claim 1, wherein the device coordinates (x, y), the first origin coordinates ( )(C1), the second origin coordinates ( )(C2) and the positioning coordinates ( )(C3) is a rectangular coordinate system. A method for calculating the actual distance and angle of a target object by a tracking device as described in claim 1, wherein the origin of the positioning coordinates in the coordinate positioning step (S13) is set at the bottom center position of the comparison image. A method for calculating the actual distance and angle of a target object by a tracking device as described in claim 1, wherein the target object is a movable object.

Images