TWI866318B - Dual-vehicle heterogeneous collaborative navigation system and navigation method thereof - Google Patents

Abstract

This invention includes a land vehicle, an unmanned aerial vehicle (UAV) and a pre-built database. When the land vehicle moves on a ground, it can capture a real-time moving image. When the UAV flies above the ground, it can capture a real-time flying image. A wireless communication can be transmitted between the land vehicle and the UAV. The pre-built database includes road-viewing images, road corresponding geographical coordinates, fly-viewing images, and aerial corresponding geographical coordinates. By comparing the characteristics of the real-time moving image with the characteristics of the road-viewing images, the most consistent one can be found. Hence, the corresponding road corresponding geographical coordinates can be obtained. Similarly, by comparing the characteristics of the real-time flying image with the characteristics of the fly-viewing images, the most consistent one can be found. Hence, the corresponding aerial corresponding geographical coordinates can be obtained as well. By collaborating road corresponding geographical coordinates and the aerial corresponding geographical coordinates, it can achieve the dual-vehicle heterogeneous collaborative navigation and positioning. The speed for navigation and positioning is quick. The precision of this dual-vehicle heterogeneous collaborative navigation without reliance of GPS is high. In addition, the road mode and the aerial mode is switchable for such navigation and positioning.

Description

Dual-carrier heterogeneous cooperative positioning system and positioning method

The present invention relates to a dual-vehicle heterogeneous cooperative positioning system and a positioning method thereof, and in particular to a dual-vehicle heterogeneous cooperative positioning system and a positioning method thereof, which has a fast positioning speed, high dual-vehicle heterogeneous cooperative positioning accuracy, is independent of GPS, and can switch between road mode and air mode to achieve positioning.

The Global Positioning System (GPS) is a well-known positioning technology, but it has the disadvantage that it loses its positioning function when encountering signal shielding or interference.

There is also a known driving control system (driving control system, control method, and control device of the Republic of China invention publication No. TW202225888), which mainly includes a drone and a drone, and wireless transmission can be performed between them. The above-mentioned known positioning device mainly uses the drone to take aerial photos to determine whether there are obstacles in front of the drone and whether the route needs to be changed. It is not possible to perform the dual-carrier heterogeneous cooperative positioning of this case, nor can it switch between the road mode and the air mode to achieve positioning. In other words, if the GPS encounters signal shielding or interference, the above-mentioned known positioning device will not be able to locate.

In view of this, it is necessary to develop technology that can solve the above-mentioned shortcomings.

The purpose of the present invention is to provide a dual-vehicle heterogeneous cooperative positioning system and a positioning method thereof, which has the advantages of fast positioning speed, high accuracy of dual-vehicle heterogeneous cooperative positioning, no reliance on GPS, and the ability to switch between road mode and air mode to achieve positioning. In particular, the problem that the present invention intends to solve is that there is no dual-vehicle heterogeneous cooperative positioning system and a positioning method thereof so far.

The technical means to solve the above-mentioned problem is to provide a dual-vehicle heterogeneous cooperative positioning system and a positioning method thereof. The positioning system comprises: a road vehicle, which comprises a road moving part, a road image capturing part, a road wireless communication part and a road control part; the road moving part is used to carry the road image capturing part, the road wireless communication part and the road control part to move on a road surface; the road image capturing part is used to capture at least one real-time image during the movement; an aerial vehicle, which comprises an aerial moving part, an aerial image capturing part, an aerial wireless communication part and an aerial control part. control unit; the aerial mobile unit is used to carry the aerial image capture unit, the aerial wireless communication unit and the aerial control unit to fly in the air; the aerial image capture unit is used to capture at least one real-time image in flight in real time, and the aerial vehicle and the road vehicle move synchronously; and wirelessly transmit information through the road wireless communication unit and the aerial wireless communication unit; and a pre-built database is set on one of the road vehicle and the aerial vehicle; the pre-built database M includes a road database and an aerial database; the road database has corresponding multiple road perspective images and multiple road corresponding coordinates, the aerial database has corresponding multiple aerial perspective images and multiple aerial corresponding coordinates; thereby, the dual-vehicle heterogeneous cooperative positioning system is used to perform a road mode and an aerial mode; when the road mode is performed, the pre-built database captures the at least one in-flight real-time image and compares the image features with the multiple road perspective images. When the image features with the highest degree of consistency are matched, the corresponding road corresponding coordinates are output, thereby achieving road positioning; and when the aerial mode is performed, the pre-built database captures the at least one in-flight real-time image and compares the image features with the multiple road perspective images. The image features of multiple aerial perspective images are compared. When the image features with the highest degree of matching are matched, the corresponding aerial corresponding coordinates are output to achieve aerial positioning; the pre-built database further integrates the road surface corresponding coordinates outputted above with the aerial corresponding coordinates to achieve the dual-vehicle heterogeneous coordinated positioning system; Wherein, the at least one real-time image during the march has at least one road surface image feature; and the pre-built database compares the multiple road surface perspective images with the at least one road surface image feature; wherein the at least one road surface image feature is at least one of a signboard and a road sign.

The positioning method includes the following steps: 1. Preparation step; 2. Road mode execution step; 3. Air mode execution step; and 4. Dual-vehicle heterogeneous coordination synchronization.

The above-mentioned purposes and advantages of the present invention can be easily understood from the detailed description and accompanying drawings of the following selected embodiments.

The present invention is described in detail with the following embodiments and accompanying drawings:

10: Road vehicles

11: Road surface moving part

12: Road image capture unit

121: Real-time image while moving

12A: Road surface image features

13: Road wireless communication department

14: Road surface control department

20: Air Vehicles

21: Air Mobile Division

22: Aerial image capture unit

221: Real-time images during flight

22A: Aerial image features

23: Airborne Radio Communications Department

24: Air Control Department

91: Road surface

92: In the air

M: Pre-built database

M1: Road surface database

M2: Air database

M1A1, M1B1, M1C1: Road View Image

M1A2, M1B2, M1C2: road surface corresponding coordinates

M2A1, M2B1, M2C1: Aerial view images

M2A2, M2B2, M2C2: Corresponding coordinates in the air

H: Height

L: Distance

S1: Preparation steps

S2: Road mode steps

S3: Air mode steps

S4: Dual-carrier heterogeneous coordinated synchronization

Figure 1 is a schematic diagram of the positioning system of the present invention.

Figure 2 is a schematic diagram of an implementation example of Figure 1.

Figure 3A is a schematic diagram of the actual application example of the present invention.

Figure 3B is a schematic diagram of the positioning process of Figure 3A (buildings omitted).

Figure 4 is a schematic diagram of the captured image of the road vehicle of the present invention.

Figure 5 is a schematic diagram of the captured image of the aerial vehicle of the present invention.

Figure 6 is a flow chart of the positioning method of the present invention.

Referring to Figures 1, 2, 3A, 3B, 4 and 5, the present invention is a dual-vehicle heterogeneous cooperative positioning system and positioning method thereof, and the positioning system part includes: A road vehicle 10, which includes a road moving unit 11, a road image capturing unit 12, a road wireless communication unit 13 and a road control unit 14. The road moving unit 11 is used to carry the road image capturing unit 12, the road wireless communication unit 13 and the road control unit 14 to move on a road 91; the road image capturing unit 12 is used to capture at least one (actually multiple) real-time images 121 during movement.

An aerial vehicle 20 includes an aerial mobile unit 21, an aerial image capture unit 22, an aerial wireless communication unit 23, and an aerial control unit 24. The aerial mobile unit 21 is used to carry the aerial image capture unit 22, the aerial wireless communication unit 23, and the aerial control unit 24 to fly in an air 92 (as shown in FIG. 2, the aerial vehicle 20 has a height H with the ground 91, and has a distance L with the road vehicle 10 on the ground 91); the aerial image capture unit 22 is used to capture at least one (actually multiple) in-flight real-time image 221. The aerial vehicle 20 and the road vehicle 10 move synchronously; and wirelessly transmit messages through the road wireless communication unit 13 and the aerial wireless communication unit 23.

A pre-built database M is set on one of the road vehicle 10 and the aerial vehicle 20. The pre-built database M includes a road database M1 and an aerial database M2; the road database M1 has corresponding multiple road perspective images (as shown in Figure 4, for example, including M1A1, M1B1, M1C1, ...) and multiple road corresponding coordinates (for example, including M1A2, M1B2, M1C2, ...); the aerial database M2 has corresponding multiple aerial perspective images (as shown in Figure 5, for example, including M2A1, M2B1, M2C1, ...) and multiple aerial corresponding coordinates (for example, including M2A2, M2B2, M2C2, ...).

Thus, the dual-vehicle heterogeneous cooperative positioning system is used to perform a road mode and an aerial mode; when performing the road mode, the pre-built database M captures the at least one real-time image 121 during the flight, and compares the image features with the multiple road view images (as shown in Figure 4, for example, including M1A1, M1B1, M1C1, ...), and when the image feature with the highest degree of matching is matched, the corresponding road coordinates (for example, including M1A2, M1B2, M1C2, ...) are output, thereby achieving road positioning.

When the aerial mode is performed, the pre-built database M captures the at least one in-flight real-time image 221 and compares the image features with the multiple aerial view images (as shown in FIG. 5 , for example, including M2A1, M2B1, M2C1 , ...). When the image features with the highest degree of matching are matched, the corresponding aerial coordinates (for example, including M2A2, M2B2, M2C2, ...) are output to achieve aerial positioning; the pre-built database M further integrates the outputted road surface coordinates (for example, M1A2, M1B2 or M1C2) and the corresponding coordinates in the air (M2A2, M2B2 or M2C2) to achieve the dual-vehicle heterogeneous coordinated positioning system; wherein the at least one real-time image 121 during movement has at least one road image feature 12A; and the pre-built database M compares the multiple road view images (M1A1, M1B1, M1C1) with the at least one road image feature 12A; wherein the at least one road image feature 12A is at least one of a signboard and a road sign.

In practice, in this case, the pre-built database M is located on the road vehicle 10.

Refer to Figure 6, the dual-carrier heterogeneous coordinated positioning method includes the following steps:

1. Preparation step S1: Prepare a road vehicle 10, an aerial vehicle 20 and a pre-built database M. The road vehicle 10 includes a road moving unit 11, a road image capturing unit 12, a road wireless communication unit 13 and a road control unit 14. The road moving unit 11 is used to carry the road image capturing unit 12, the road wireless communication unit 13 and the road control unit 14 to move on a road surface 91; the road image capturing unit 12 is used to capture at least one (actually multiple) real-time images 121 during movement. The aerial vehicle 20 includes an aerial moving unit 21, an aerial image capturing unit 22, an aerial wireless communication unit 23 and an aerial control unit 24. The aerial mobile unit 21 is used to carry the aerial image capturing unit 22, the aerial wireless communication unit 23 and the aerial control unit 24 to fly in an air 92 (as shown in FIG. 2 , the aerial vehicle 20 is at a height H from the ground 91 and at a distance L from the road vehicle 10 on the ground 91); the aerial image capturing unit 22 is used to capture at least one (actually multiple) in-flight real-time image 221. The aerial vehicle 20 and the road vehicle 10 move synchronously; and wirelessly transmit information through the road wireless communication unit 13 and the aerial wireless communication unit 23. The pre-built database M is located on one of the road vehicle 10 and the aerial vehicle 20; the pre-built database M includes a road database M1 and an aerial database M2; the road database M1 has corresponding multiple road perspective images (such as shown in FIG. 4, including M1A1, M1B1, M1C1, ...) and multiple road corresponding coordinates (such as including M1A2, M1B2, M1C2, ...); the aerial database M2 has corresponding multiple aerial perspective images (such as shown in FIG. 5, including M2A1, M2B1, M2C1, ...) and multiple aerial corresponding coordinates (such as including M2A2, M2B2, M2C2, ...). The dual-vehicle heterogeneous coordinated positioning method is used to perform a road mode and an aerial mode.

2. Road surface mode step S2: When the road surface mode is performed, the pre-built database M is used to capture the at least one real-time image 121 during driving, and compare the image features with the multiple road surface view images (as shown in Figure 4, for example, including M1A1, M1B1, M1C1, ...). When the image feature with the highest degree of matching is matched, the corresponding road surface coordinates (for example, including M1A2, M1B2, M1C2, ...) are output, thereby achieving road surface positioning.

3. Step S3 of the aerial mode: When the aerial mode is performed, the pre-built database M is used to capture the at least one in-flight real-time image 221 and compare the image features with the multiple aerial view images (as shown in FIG. 5, for example, including M2A1, M2B1, M2C1, ...). When the image features with the highest degree of matching are matched, the corresponding aerial coordinates (for example, including M2A2, M2B2, M2C2, ...) are output, thereby achieving aerial positioning.

4. Dual-vehicle heterogeneous coordination synchronization S4: When the pre-built database M completes the positioning of the road mode and the air mode, the road corresponding coordinates (such as M1A2, M1B2 or M1C2) and the air corresponding coordinates (M2A2, M2B2 or M2C2) outputted above can be integrated to achieve the dual-vehicle heterogeneous coordination positioning system.

In practice, regarding the image feature comparison of the pre-built database M in step S2 of the road mode, for example: The at least one real-time image 121 captured in real time has at least one road image feature 12A, and the at least one road image feature 12A can be at least one of a store sign and a road sign.

Assume that at a certain time point, the at least one real-time image 121 during the journey may include 4 road image features 12A (for example, the shop sign on the left, another shop sign on the right, the road sign on the left, and the road sign on the right in the upper frame of Figure 4). Then when the pre-built database M compares the at least one real-time image 121 during the journey with the multiple road view images (for example, including M1A1, M1B1, M1C1, etc.), in fact, it is not the entire image that is compared (because the amount of calculation is large), but only the 4 road image features 12A are compared, so the amount of calculation can be greatly reduced, and when the one with the highest degree of matching is quickly matched, the corresponding road coordinates (for example, including one of M1A2, M1B2, M1C2, etc.) are output, thereby achieving road positioning.

For example, if the road surface perspective image M1B1 has the highest degree of conformity, the data of the corresponding road surface corresponding coordinates M1B2 (such as longitude, latitude and altitude) will be output, and the positioning will be completed.

Regarding the part of performing image feature comparison with the pre-built database M in step S3 of the aerial mode, similarly: the at least one real-time in-flight image 221 captured in real time has at least one aerial image feature 22A, and the at least one aerial image feature 22A can be the rooftop decoration of a building, etc.

Similarly, assuming that at a certain time point, the at least one in-flight real-time image 221 may include three aerial image features 22A (e.g., the rooftop decorations on the upper left, lower left, and right sides in the upper frame of Figure 5). Then, when the pre-built database M compares the at least one in-flight real-time image 221 with the multiple aerial view images (e.g., including M2A1, M2B1, M2C1, ...), in fact, the entire image is not compared (because the amount of calculation is large), but only the three aerial image features 22A are compared, so the amount of calculation can be greatly reduced, and when the highest degree of matching is quickly matched, the corresponding aerial corresponding coordinates (e.g., including one of M2A2, M2B2, M2C2, ...) are output, and the aerial positioning is achieved.

In short, referring to Figures 3A and 3B, assuming that the road vehicle 10, the aerial vehicle 20 and the pre-built database M are set in a certain area for positioning, the following situation may be encountered: When the satellite signal can be received smoothly, the positioning information of GPS is used normally.

When the GPS satellite signal is partially blocked or interfered, the traditional GPS positioning function fails, and this case will come into play.

That is, when the road vehicle 10 (capturing the real-time image 121 while moving) and the aerial vehicle 20 (capturing the real-time image 221 while flying) both encounter no obstacles and successfully capture images in real time during the forward process, the pre-built database M can successfully complete the positioning of the road mode and the aerial mode, and integrate the road corresponding coordinates (such as M1A2, M1B2 or M1C2) and the aerial corresponding coordinates (M2A2, M2B2 or M2C2), thereby improving the positioning accuracy and achieving dual-vehicle heterogeneous coordination.

In the event that one of the positioning of the road mode and the aerial mode fails to work, the positioning information can be obtained from the other mode. If the road mode encounters an obstruction and cannot successfully capture images in real time, the aerial mode can be switched to obtain the corresponding positioning information on the ground through geometric calculations, and vice versa.

When the aerial vehicle has a certain height relative to the ground and forms an angle with the ground vehicle, a typical right triangle geometric relationship is formed. Then, the coordinates can be converted or transformed using existing geometric calculation techniques and formulas (known techniques). Since this is a known technique (e.g., aerial surveying technology textbooks, Republic of China invention patent No. I444593 or other similar techniques), it will not be elaborated here.

The advantages and effects of the present invention can be summarized as follows:

[1] Fast positioning speed. The real-time image during movement of the present invention has at least one road image feature, and the real-time image during flight has at least one aerial image feature. It is sufficient to compare the at least one road image feature with the at least one aerial image feature, and the amount of calculation is small. Therefore, the positioning speed is fast.

[2] The dual-vehicle heterogeneous cooperative positioning is highly accurate and does not rely on GPS. The pre-built database can integrate the road corresponding coordinates and the aerial corresponding coordinates to improve positioning accuracy and achieve dual-vehicle heterogeneous coordination without relying on traditional GPS. Therefore, the dual-vehicle heterogeneous cooperative positioning is highly accurate and does not rely on GPS.

[3] The road mode and the aerial mode can be switched to achieve positioning. When the road image features of the road vehicle (road mode) are insufficient or blocked, the aerial mode (air corresponding coordinates) can be switched to be the main mode. When the aerial vehicle (air mode) encounters obstacles such as dark clouds, wires, flying objects, etc. when moving and needs to avoid obstacles, the road mode (road corresponding coordinates) is switched to be the main mode. Therefore, the road mode and the aerial mode can be switched to achieve positioning. If the road mode encounters an obstruction and cannot successfully capture images in real time, the aerial mode is switched to obtain the corresponding ground positioning information through geometric calculations, and vice versa.

The above is only a detailed description of the present invention through a preferred embodiment. Any simple modification and changes made to the embodiment do not deviate from the spirit and scope of the present invention.

10: Road vehicles

11: Road surface moving part

12: Road image capture unit

121: Real-time image while moving

13: Road wireless communication department

14: Road surface control department

20: Air Vehicles

21: Air Mobile Division

22: Aerial image capture unit

221: Real-time images during flight

23: Airborne Radio Communications Department

24: Air Control Department

91: Road surface

92: In the air

M: Pre-built database

M1: Road surface database

M2: Air database

Claims (8)

A dual-vehicle heterogeneous cooperative positioning system includes: a road vehicle, including a road mobile unit, a road image capture unit, a road wireless communication unit and a road control unit; the road mobile unit is used to carry the road image capture unit, the road wireless communication unit and the road control unit to move on a road; the road image capture unit is used to capture at least one real-time image during the journey; an aerial vehicle, including an aerial mobile unit, an aerial image capture unit, an aerial wireless communication unit and an aerial control unit; the aerial mobile unit is used to carry the aerial image capture unit, The airborne wireless communication unit and the airborne control unit fly in the air; the airborne image capture unit is used to capture at least one real-time image in real time, and the airborne vehicle and the road vehicle move synchronously; and wirelessly transmit information through the road wireless communication unit and the airborne wireless communication unit; and a pre-built database is set on one of the road vehicle and the airborne vehicle; the pre-built database includes a road database and an airborne database; the road database has corresponding multiple road perspective images and multiple road corresponding coordinates, and the airborne database has corresponding multiple air The dual-vehicle heterogeneous cooperative positioning system is used to perform a road mode and an air mode; when the road mode is performed, the pre-built database captures the at least one in-flight real-time image and compares the image features with the multiple road view images. When the image features with the highest degree of consistency are matched, the corresponding road coordinates are output to achieve road positioning; and when the air mode is performed, the pre-built database captures the at least one in-flight real-time image and compares the image features with the multiple air view images. When the image feature with the highest matching degree is matched, the corresponding aerial coordinates are output to achieve aerial positioning; the pre-built database further integrates the road surface corresponding coordinates outputted above and the aerial corresponding coordinates to achieve the positioning system of the dual-vehicle heterogeneous coordination and not relying on GPS; wherein the at least one real-time image during the movement has at least one road surface image feature; and the pre-built database compares the multiple road surface view images with the at least one road surface image feature; wherein the at least one road surface image feature is at least one of a signboard and a road sign. The dual-vehicle heterogeneous coordinated positioning system as described in claim 1, wherein the pre-built database is located on the road vehicle. A dual-vehicle heterogeneous cooperative positioning method includes: 1. Preparation step: preparing a road vehicle, an aerial vehicle and a pre-built database; the road vehicle includes a road moving part, a road image capturing part, a road wireless communication part and a road control part; the road moving part is used to carry the road image capturing part, the road wireless communication part and the road control part to move on a road surface; the road image capturing part is used to capture at least one real-time image during the movement; the aerial vehicle includes an aerial moving part, an aerial image capturing part, an aerial wireless communication part and a road control part; The aerial mobile unit is used to carry the aerial image capture unit, the aerial wireless communication unit and the aerial control unit to fly in the air; the aerial image capture unit is used to capture at least one real-time image in flight in real time, and the aerial vehicle and the road vehicle move synchronously; and wirelessly transmit information through the road wireless communication unit and the aerial wireless communication unit; the pre-built database is set on one of the road vehicle and the aerial vehicle; the pre-built database includes a road database and an aerial database, and the road database has a corresponding plurality of road perspective image and multiple road surface corresponding coordinates; the aerial database has corresponding multiple aerial perspective images and multiple aerial corresponding coordinates, and the dual-carrier heterogeneous cooperative positioning method is used to perform a road mode and an aerial mode; 2. Road mode execution step: When performing the road mode, the pre-built database is used to capture the at least one real-time image during the journey, and compare the image features with the multiple road perspective images. When the image features with the highest degree of matching are matched, the corresponding road surface corresponding coordinates are output, thereby achieving road surface positioning; 3. Aerial mode Steps: When the aerial mode is performed, the pre-built database is used to capture the at least one real-time image during flight and compare the image features with the multiple aerial view images. When the image features with the highest degree of matching are matched, the corresponding aerial coordinates are output to achieve aerial positioning; and 4. Dual-vehicle heterogeneous coordination synchronization: When the pre-built database completes the positioning of the road mode and the aerial mode, the road corresponding coordinates and the aerial corresponding coordinates outputted above can be integrated to achieve the dual-vehicle heterogeneous coordination and positioning system that does not rely on GPS. The dual-vehicle heterogeneous cooperative positioning method as described in claim 3, wherein the pre-built database is located on the road vehicle. The dual-vehicle heterogeneous cooperative positioning method as described in claim 3, wherein: the at least one real-time image during the movement has at least one road image feature; and the pre-built database is compared with the multiple road view angle images and the at least one road image feature. The dual-carrier heterogeneous coordinated positioning method as described in claim 5, wherein the at least one road image feature is at least one of a signboard and a road sign. The dual-vehicle heterogeneous cooperative positioning method as described in claim 3, wherein: the at least one inter-flight real-time image has at least one aerial image feature; and the pre-built database compares the multiple aerial view images with the at least one aerial image feature. The dual-carrier heterogeneous coordinated positioning method as described in claim 7, wherein the at least one aerial image feature is a rooftop decoration.

Images