WO2022180276A1 - Autonomous precision landing system, method and program for drones - Google Patents

Abstract

Disclosed are an autonomous precision landing system, method and program for drones. The system comprises a camera (910) installed on board a drone (1010) and directed towards the ground; a landing pad detection unit (920) on board the drone (1010) and designed to detect, in an image (502) captured by the camera (910), a landing pad (400) formed by a plurality of concentric circular rings (402) having decreasing thickness, by detecting a predetermined number N of concentric circles in the image (502); a command system (930) on board the drone (1010) and designed to autonomously land the drone (1010) on the landing pad (400), using, as a reference, positions (508) of the landing pad (400) detected in images (502) captured by the camera (910) during landing. The invention allows precise landing on the landing pad (400).

Description

SYSTEM. PRECISION LANDING PROGRAM METHOD AND PRODUCT

AUTONOMOUS FOR DRONES

DESCRIPTION

field of invention

The present invention falls within the field of autonomous landing systems for drones.

Background of the invention

Currently, there are autonomous landing systems for drones that use different technologies to land in a specific landing zone, among which the following stand out:

• Infrared light detection: it is based on the use of an array of infrared LEDs with respect to which to position oneself looking for its center (e.g. the “IRIock” autonomous landing system). The problem with using infrared is the contamination of the light footprint that may exist, especially with direct sunlight, which makes it difficult to make precise landings in certain environmental conditions.

• Code detection: uses for landing the detection of a code (e.g. ArUco code used by the “Flytbase” autonomous landing system) printed on the landing zone, which can be used to identify the landing zone (to check for example, which is the one that corresponds to the drone) as well as as a navigation reference during landing. However, the current landing systems based on code detection (e.g. ArUco, QR, ARtag) have the disadvantage that they can only detect the code when there is a complete view of it, so they do not work correctly when the code is partially detected, This happens very frequently when the drone is at a low altitude during the landing operation, close to the landing zone, or when external elements (such as fallen leaves from the trees) partially cover the code. .

In the field of precise landing systems for drones, where the drone needs to land in a small and specific space, for example, on a small landing platform so that the drone can be loaded automatically, in the Currently, to make landings with errors of a few centimeters, an RTK-type GNSS navigation data source is used. But an RTK system has the disadvantage of the need to use external systems, such as GNSS constellations (which do not provide guarantees of operational persistence for critical systems, as is the case of drone safe landing systems) and the telemetry link with the base to receive RTK corrections , which can also fail. This dependence on external systems can cause failures (eg in data communication), as well as requiring additional hardware elements.

The present invention proposes an autonomous precision landing system that solves these problems.

Description of the invention

The invention relates to an autonomous precision landing system and method for drones. The invention is based on the detection of a certain landing template. The main components of the system are a landing template detector and a system to command corrections to the drone.

According to one embodiment, the autonomous precision landing system for drones of the present invention comprises the following elements:

A camera installed on board a drone and oriented towards the ground.

A landing template detection unit, on board the drone and configured to detect, in an image captured by the camera, a landing template formed by a plurality of concentric circular crowns of decreasing thickness by detecting a predetermined number N of concentric circles in the image. The landing template detection unit analyzes an input image and returns whether or not the template is present and the coordinates or relative position of the center of the template on the input image.

A command system, embedded in the drone and configured to perform an autonomous landing of the drone on the landing template using as reference the positions of the landing template detected in images captured by the camera during landing.

Advantageously, the landing template detection unit can be used in real environments in a drone, with detection speeds in real time greater than 25Hz, which is important to be able to counteract movements of the drone in its displacement, both by the corrections that are applied to it as for external movements caused by example by the wind.

The template detection algorithm can be run on a small, inexpensive device such as the Nvidia Jetson Nano. The detection carried out provides robustness against changing light conditions, being able to work with clear skies, with clouds and conditions in which shadows are produced on part of the landing template.

The template detection of the present invention provides robustness against occlusions, being able to resist small elements that can cover the landing template, such as leaves that fall on top of the template. Detection is also robust against drone vibrations, as drone flight will induce vibrations on the camera mount. Furthermore, the degree of robustness of the detection can be adapted by regulating the predetermined number N of concentric circles in the image to be detected; for example, detections can be made more robust by enforcing a larger number N of concentric detections. By using a visual pattern (unlike RTK-based autonomous landing systems), the landing system can ensure the necessary operational persistence to ensure accurate landings.

Regarding the detection distance, the landing template detection unit is capable of detecting the template at a minimum distance of about 15cm (which is necessary to be able to establish a guidance up to a sufficiently low height) and at a maximum distance higher to 10m, which is necessary to be able to start the automated control when the drone's return position has a static error of a couple of meters.

The landing template detection procedure allows it to work with a simple camera (RGB monocular) and light (less than 50gr), so as not to reduce the load capacity or autonomy of the drone.

The size of the detection template is limited by the dimensions of the landing platform where you want to land the drone. In one embodiment, for example, a size of 80x80cm is considered. The minimum size of the detection template will depend on the maximum distance at which detections are to be made; for example, in an embodiment where the size is reduced to 60x60cm, detections at a distance of 10m could be guaranteed. Detection distance and template size are related in a linear proportional manner. In case the required sensing distance is less, the size may reduce. The absolute minimum size of the detection template is the one that allows the circular crowns not to be coupled, which can be around 20x20cm.

Regarding the drone command system, it is in charge of making the automatic landing in a safe way, generating an appropriate set of movements. To do this, it uses a valid communication system (e.g. communication through MAVROS) with the chosen autopilot (e.g. PX4). The command system is capable of executing the control procedure at a real-time rate greater than 25Hz.

The command system is capable of detecting the moment in which the drone must pass from the autopilot's own control (e.g. PX4) to the automatic precision landing control of the present invention. To do this, it evaluates that a series of conditions are met.

The command system is responsible for repeatedly managing drone descent actions and drone centering actions during landing, since even if an initial centering of the drone is performed with respect to the landing template before starting the descent, during During descent, alterations in the centering of the drone may occur (produced, for example, by small gusts of wind) that make new corrections necessary for the correct centering of the drone at the moment of landing.

Likewise, the command system is capable of managing recovery actions, such as ascending to a predetermined level in the event that the landing template is no longer visible (e.g. lateral displacement caused by the wind, large object covering the template, etc. ), or alternative actions to precision landing (e.g. landing in an alternative zone, passing manual control to the pilot, etc.) when the landing process exceeds a pre-established time limit.

The present invention also relates to an autonomous precision landing method for drones, comprising the following steps:

- Repeatedly acquire images using a camera installed on board a drone and oriented towards the ground.

Detect, in each image captured by the camera, a landing template formed by a plurality of concentric circular crowns of decreasing thickness by detecting a predetermined number N of concentric circles in the image. Perform an autonomous landing of the drone on the landing template using as reference the positions of the landing template detected in images captured by the camera during landing.

A further aspect of the present invention relates to a program product comprising program instruction means for performing the autonomous precision landing method for drones.

Brief description of the drawings

Next, a very brief description is given of a series of drawings that help to better understand the invention and that are expressly related to an embodiment of said invention that is presented as a non-limiting example of it.

Figure 1 shows a first embodiment of a circular shaped landing template.

Figure 2 illustrates a second embodiment of a circular shaped landing template.

Figure 3 represents a third embodiment of a circular shaped landing template.

Figure 4 represents a fourth embodiment of a circular shaped landing template.

Figure 5A shows a landing template detection procedure in which the landing template of Figure 4 is used. Figure 5B illustrates, according to one embodiment, the detection process of N concentric circles in the image.

Figures 6A-6D show an example of the treatment and analysis of an image captured by a drone camera for the detection of the landing template.

Figure 7 shows an example of cropping an image to eliminate a circumference detected in it.

Figure 8 shows, according to one embodiment, an autonomous precision landing method for drones.

Figure 9 illustrates an autonomous precision landing system for drones according to a possible realization.

Figure 10 shows a drone prepared to initiate a precision landing on a landing template arranged on a landing pad.

Figures 11 and 12 show different checks that are carried out, according to a possible embodiment, before initiating a precision landing.

Figure 13 shows a status check of the drone performed during a precision landing according to a possible embodiment.

Detailed description of the invention

The present invention relates to an autonomous precision landing system for drones based on the detection of a landing template used as a visual reference for landing. The landing template must be identified by the drone during landing in order to land safely on top of it.

The autonomous precision landing system comprises a landing template detection unit (or landing template detector) configured to detect the presence and position of the landing template in images captured by a ground-facing drone camera, and a drone command system in charge of executing the maneuvers and movements of the drone during landing and commanding the drone the appropriate corrections for the autonomous landing of the drone on the landing template based on the detections made by the landing template detector, and in particular taking as reference the positions of the landing template detected in images captured by the camera during the landing sequence.

In the field of shape detection techniques, there are multiple methods to detect shapes or shapes of elements present in a digital image. In the case of circles, the most common classical computer vision methods are based on edge extraction followed by processing, such as contour extraction and the Hough Transform. There are alternatives such as the extraction of related components or the search for patterns by comparing templates (“Template Matching”), among others. Another alternative is to use artificial vision techniques based on artificial intelligence, such as neural networks in the form of a convolutional network. As for the shapes to detect in the template, some of the most effective are squares (eg, ArUco, QR, ARtag labels or codes) and circles. When no information needs to be encoded inside the template, circles have the advantage of being invariant against rotations, as well as being easier to distinguish when not the entire template is visible.

Figures 1 to 4 present different embodiments of landing templates, and algorithms used for their detection, using circles as shapes to detect.

Figure 1 shows a circular landing template 100, formed by two concentric circular annuluses (102,104), used by a landing template detector based on YOLO ("You Only Look Once"), an object detection system in real time. For this realization, the Tiny YOLOv2 detection system is used, a convolutional neural network (CNN) for object detection and classification. Tiny YOLOv2 is a smaller version of YOLOv2, with shorter execution times.

In order to detect the template, it is necessary to retrain the neural network, which is achieved by generating a set of examples in which the template is labeled and a class is assigned to it, in this case the only class to be detected. Algorithms like YOLO have the advantage of allowing good resistance to noise and occlusions, but they have the following problems:

• It needs to run on a device with a GPU (e.g. the Nvidia Jetson family of products), ruling out the possibility of using it on other simpler computers like an Odroid.

• You need neural network optimization libraries (TensorRT) that change frequently and are not stable to work in production.

• The initialization time of the detector is long, since it must load a large amount of data in memory. From when you want to launch the detector until it is loaded, it can take more than 8 seconds.

• Limited execution speed, with values that in the Jetson Nano are around 10-12 Hz.

• Need to generate a set of training and test data and perform training validations, which is time consuming. Figure 2 shows, according to a second embodiment, a landing template 200 based on circular shapes and colors. The landing template detector is responsible for the detection of concentric circular rings or annuluses (202,204,206) of different colors in the landing template 200. The colors of the different rings allow filtering of false positives. For the detection of the circles, edge extraction and shape checks are performed.

The main advantages of this embodiment are:

• Low number of false positives when using proper color management.

• Good execution speed (+20 FPS) on CPU.

• You only need to rely on the OpenCV library.

• Good resistance to occlusions.

The problems it presents are:

• Difficulty interpreting colors in real lighting conditions (changes in white balance and exposure), which means that automatic parameter search methods have to be implemented for each situation and that can lead to an increase in false positives.

• Template scalability. Considering a "closed" design, making the template large enough to be detected at a distance of 10m by the drone causes it to lose detection when the drone is close (since only the inner circles are visible). To solve this problem, checks must be added or removed from the algorithm to adapt it to the final size.

Figure 3 represents a third embodiment of the landing template 300, formed by a plurality of concentric circular crowns (302,304,306,308) of a dark tone, preferably black. The landing template detector is in this case based on detection of circular shapes and filtering with comparison of templates (“Template Matching”). The template comparison technique consists of searching an image for an input template, being a common technique to perform sign and license plate detection.

The main problems with this technique are:

• Manage the scale of the template (it must be very similar to that of the object present in the image to be analyzed).

• Manage occlusions, as these generate low confidence in the metric used to compare the template with the input image section.

In the event that templates of multiple sizes or with predefined occlusions are sought to help the algorithm, the execution times become too high. Furthermore, it is a method that is not rotation invariant, so adding additional shapes in the template to increase robustness complicates the implementation to a great extent.

Figure 4 depicts a fourth embodiment of the landing template 400, in which the landing template detector is based on recursive detection of circular shapes. The landing template 400 comprises a plurality of concentric annuluses 402, of decreasing thickness as the annuluses are closer to their center 406. The thickness of the annuluses, defined by the difference between the outer radius and the inner radius of the annulus circular, is selected so that the annuluses can be clearly identifiable when the landing template 400 is a certain distance away from the drone camera.

The concentric circular crowns 402 have a dark tone, preferably black, and are separated by spaces 404 in a light tone (white or another color with little intensity) and of decreasing thickness, in order to ensure that there is no overlapping optical effect. between the black concentric annuluses 402 as the landing template 400 is moved away. Alternatively, the landing template 400 could also be considered to be formed by a plurality of consecutive 2-tone concentric rings of decreasing thickness: first concentric annuluses 402 in a dark tone (for example, in black) separated from each other by second concentric circular annuluses (or spaces 402) in a light tone (for example, in white).

This particular design of the landing template 400 allows the number of concentric annuluses 402 to be increased or decreased to make it larger or smaller to suit the required situation without having to change the detection procedure used by the landing template detector or with some minimum parameter changes, such as establishing the maximum and minimum radius (in pixels) of the circles to search.

The total number of concentric circular crowns 402 allows to control the robustness against occlusions of some of the circular crowns (leaves, reflection produced by the Sun, etc.) as well as when only part of the template is visible because it is in one of the edges of the picture. A greater number of concentric annulus 402 offers greater robustness.

In the example shown in Figure 4, in which a maximum landing template size of 80x80cm is used and a minimum detection distance of about 15cm and a maximum detection distance greater than 10m are required, 7 crowns have been considered. concentric annuluses 402. To maximize the landing template 400, the concentric annuluses 402 are designed so that the outermost annulus utilizes the 40cm maximum radius allowed for the template. The inner and outer radii of the different circular crowns selected in the example of Figure 4 is as follows (units in cm):

Landing jig 400 can obviously use a different number of concentric annuluses 402 and thicknesses, depending on the particular application (landing jig 400 maximum size, minimum sensing distance, maximum sensing distance, etc.).

The basis of the landing template detector's operation is to be able to detect a predetermined number N of concentric annuluses 402, where N is equal to or preferably less than the total number of concentric annuluses 402 in the landing template 400. In this way, the algorithm manages to detect both far and near distances.

According to one embodiment, the number N of concentric circular annuluses 402 that the detector must search for is fixed for the entire process, both for near and far distances. However, since N is a parameter, the value of said parameter can be modified in real time, for example, to require a different number of concentric circles to be detected depending on the distance between the drone and the landing template 400 (eg the vertical distance measured by a height sensor and correcting for the drone's tilt angles). The minimum value of the parameter N will be N=2 (that is, to detect a pair of concentric circles) and the maximum value of N will depend on the number of visually identifiable circles in an image extracted from the drone's camera, which therefore depends on of the initial image resolution, the size of the template and the distance from the drone to the template. Increasing the parameter N would imply a decrease in the overall performance of the algorithm by having to perform a greater number of checks, but since the image to be analyzed is getting smaller and smaller (due to the successive cuts of the images), the cost of each additional check is lower. to linear complexity growth.

To perform the detection of concentric circular crowns of the landing template 400 of Figure 4, the Hough transform algorithm applied to circumferences is used. The basis of the operation of the algorithm is as follows:

1. Detect circumferences in the image captured by the drone's camera.

2. For each circumference detected, carry out N-1 recursive searches for concentric circles inside the detected circumference.

The main advantages of this technique are:

• High robustness against partial occlusions, since detections are achieved up to 50% occlusion when cut along one of the axes. In the event that the template is cut with both axes, detections are made when only 30-40% of the template is visible.

• Good robustness against light conditions. The detection algorithm works correctly in multiple real conditions, with direct light and with shadow.

• High execution speed. It is a computationally simple technique capable of running at a real-time rate greater than 25 Hz (e.g. greater than 40 Hz on the Nvidia Jetson Nano).

• It works correctly with different camera models without having to modify parameters, so its expansion to different cameras is simple or direct.

• The template is easily scalable to different sizes, depending on the maximum distances to be detected.

To increase robustness against false positives, the detection algorithm starts at the moment the drone is positioned above the landing zone and the landing sequence is to start. In addition, this can be complemented with additional code checks, such as ArUco or QR tags, present in the landing zone, which also allow a certain drone to be linked to a certain landing zone. Figure 5A depicts, according to one embodiment, a flowchart of a landing template detection procedure 500 executed by the landing template detector when landing template 400 of Figure 4 is used. First, a camera installed on board the drone and preferably oriented along the positive Z axis (axis directed downwards in normal flight attitude) in the body axis system of the aircraft acquires 501 an image 502. In normal flight attitude, the camera it is oriented towards the ground; It is not necessary for the camera to be perpendicular to the ground in normal flight attitude, that is, the camera can be oriented at a certain angle with respect to the positive Z axis of the body axis system. In the event that the camera is not perpendicular to the ground but oriented at a small angle to the ground when it acquires the image, the circles on the landing template 400 will start to look like ellipses in the captured image, but the Hough transform will still represent it as circles and the detected center will be valid. As the tilt of the camera relative to the ground at the time of capturing the image increases, the shape of the landing template circles 400 will appear on the image as ellipses in which the semi-major axis and semi-minor axis differ each time. larger in size, reaching a point from which the Hough transform will not be able to detect the circles.

Figure 6A shows, by way of example, an image 502 captured by a drone camera at the initial moment of landing (for example, at an initial height greater than 10 meters), before beginning the landing process. When the drone is in a landing start position above the landing zone, at a certain height with respect to the landing template 400 (e.g. at 10m height), the image will contain the entire landing template 400, in addition of other details and forms (601,603,605) present in the landing zone.

Next, a detection process of N concentric circles in the image 503 is started, and the detection 504 of at least N concentric circles in the image, where N is a predetermined number, is checked. In the event that at least N concentric circles are detected, detection 506 is determined on the image of the landing template 400, and the position (Xc.Yc) of the center 406 of the landing template is determined 508, which it is positioned in the center of the concentric circles detected. If at least N concentric circles are not detected, it is considered that the landing template has not been detected 510 and is therefore not present in the image. The detection of the circumferences in the image is preferably carried out by means of the Hough transform applied to circles.

Figure 5B shows a flow diagram of the detection process 503 of N concentric circles in the image, according to a preferred embodiment. First, circumferences 520 are detected in the image 502 captured by a drone camera by means of the Hough transform, obtaining detected circumferences 522. In the example of Figure 6A, the Hough transform would detect, as shown in Figure 6B , the circumferences 522a and 522b, which correspond respectively to the outer contour of the landing template 400 and the outer contour of the circle 601. In Figure 6B the first 402a, second 402b and third 402 outermost circular crowns of the template are represented numbered landing template 400, and the first 404a, second 404b, and third 404c outermost spaces of the landing template 400.

The use of the Hough transform favors the detection of the outermost circumference in the case of concentric circumferences (as is the case with landing template 400), since the outer circumference is the first to receive positive votes. Since the first step in applying the Hough transform is to perform edge extraction on the image, depending on the lighting conditions the strongest edge of each black annulus could be either the inner edge or the outer edge. The algorithm works in both cases. In the example of Figure 6B, the Hough transform applied to image 502 provides the outermost circumference 522a of landing template 400, corresponding to the outer edge of outermost annulus 402a.

For each circumference detected (522a and 522b in the example of Figure 6B), a recursive search is performed for the N-1 remaining inner concentric circumferences in order to detect the N required concentric circumferences. The recursive search comprises cropping 524 the image 502 to remove the detected circumference 522, obtaining a cropped image 526, as shown in Figure 6C when the crop is applied to the detected circumference 522a corresponding to the landing template 400.

Once the first cropped image 526 has been obtained, the following steps are repeated iteratively until N-1 concentric circles inside the detected circle are detected (or until inside concentric circles are no longer detected, in the event that there are fewer than N circles). concentric for the detected circumference - 522a, 522b- being analyzed):

Detect 528 an additional circumference 530 in each cropped image 526 using the Hough transform. If no circumference (i.e. additional circumference 530) is found in the cropped image, the landing template is considered not detected 510.

Check 532 that each additional circumference 530 is concentric to the detected circumference 522, within a margin of error. If it is not, it is considered that the landing template has not been detected 510.

Crop 536 each cropped image (526,538) to eliminate the corresponding additional circumference 530, obtaining an additional cropped image 538 (which will be analyzed again to check if an additional circumference 530 is detected 528).

During the iterative process, it is checked 534 (for example, after checking 532 the concentricity of the additional circumference 530 with respect to the detected circumference 522) if N-1 additional circumferences 530 have already been detected (through, for example, a counter of detected additional circumferences or an iteration number counter), in which case it is determined that the landing template has been detected 506; otherwise, the cropped image is cropped 536 and the iterative process is restarted, starting again with the detection 528 of an additional circumference 530 in the cropped image 538.

Applying this iterative process to the cropped image 526 of Figure 6C for N=3, a first additional circumference 530a is detected 528, which would correspond to the outer edge of the outermost circular annulus 402b (and which is complete, not cropped) in the image. cropped image 526, said annulus 402b being the second outermost annulus of landing template 400, as illustrated in Figure 6B. Next, it is verified 532 that the first additional circumference 530a is indeed concentric to the detected circumference 522a and that 534 N-1 (ie 2) additional circumferences have not yet been detected, with which the cropped image 526 (Figure 6C) is displayed. re-crops 536, obtaining a new cropped image 538 (Figure 6D), in which a second additional circumference 530b concentric to the detected circumference 522a is detected, which corresponds to the outer edge of the outermost circular annulus 402c in the cropped image 536 (third outermost crown of landing template 400, Figure 6B). In this way the iterative process is finished, having detected the two additional circumferences (530a, 530b) required, and detection 506 of landing template 400 is determined.

The landing template detection unit also determines 508 the position (Xc.Yc) of the center 406 of the landing template 400. The coordinates (Xc.Yc) of the center of the landing template 400 can be mapped to the coordinates of the center (606a, 606b, 606c) of any of the concentric circles (522a, 530a, 530b) or can be calculated from them, for example as an average value of the coordinates of the centers (606a, 606b, 606c ) of the concentric circles (522a, 530a, 530b) detected.

The cuts (524,536) of the images are made in such a way that the detected circumference is eliminated, to avoid detecting the same element again when the Hough transform is applied again to the cut image (526,538). For example, in Figure 6C the cropped image 526 is obtained by removing, at least partially, the outermost annulus 402a, so that none of its edges (inner or outer) can be detected when the Hough transform is applied. to the cropped image 526. The specific cropping to be applied once a circumference is detected and the next one is to be detected can be adjusted based on the ratios of the distances between the concentric circular crowns 402 and the spaces 404.

An example of cropping an image 700 (rectangular array) to remove a detected circumference 702 is shown in Figure 7. The cropped image 704 (or sub-image) is selected such that the detected circumference 702 is removed to prevent it from being detect the same element again in the cropped image 704. The detected circumference is defined by the parameters (x, y, r), where 'x' is the position of the center 706 of the circumference 702 for the X axis, 'y' is the position of the center 706 of the circle 702 for the Y axis, and the value Ύ is the radius of the circle 702.

To apply the crop, the rectangle 708 (or square, when the camera is arranged perpendicular to the landing template 400) containing the detected circumference 702 is calculated. The points that define the four corners of the rectangle 708 are calculated from as follows: x1 = max(x - r, 0) x2 = min(x + r, W) y1 = max(y - r, 0) y2 = min(y + r, H)

Where x1, x2, y1 , y2 are the minimum and maximum positions of the rectangle on the X and Y axes, W is the width of the 700 image in pixels, and Ή' is the height of the 700 image in pixels.

Based on the distance between the annuluses 402 and the gaps 404, a crop function is applied to the rectangle 708 to remove the detected circumference 702, reducing the size of the image 700 by a certain percentage. For example, in the cutouts shown in Figures 6C, 6D and 7, the chosen cutout is such that the cutout image 704 maintains a certain percentage (e.g. 70%) of the dimensions of the rectangle 708 that encompasses the circumference 702. Figure 7 shows half (To/2) of the initial size (To) of the rectangle 708 on the X axis, half (Ti/2) of the final size (Ti) of the cropped image 704, and the percentage of crop (%R) applied .

Once the image 700 has been cut, the search for a circumference is carried out again. In addition, in each recursive step the cropped image 704 is smaller, so the computational cost of applying the Hough transform is reduced, since in the new sub image the maximum radius to be searched is reduced compared to the previous image and also there are fewer pixels on which to apply edge and circumference detection.

If an internal circumference is identified, it is calculated with a predetermined margin of error if it is concentric. This can be done by projecting the center of the circumferences found on the general coordinates of the input image, calculating the discrepancy of its position and verifying that it is less than a margin of error considered acceptable. The projection is done by adding for each axis the coordinates of the new detected center to the coordinates of the sub-image clipping rectangle.

If any of the internal search steps fail, the landing template detection algorithm discards the circumference. If it is possible to validate the conditions for the required N total circumferences, the landing template detection is taken as valid.

Landing template 400 can be supplemented by adding more internal shapes, such as squares or diamonds, for greater robustness against false positives.

Figure 8 shows the flowchart of an autonomous precision landing method 800 for drones according to an embodiment of the present invention, which employs the process of landing template detection explained in Figures 5 to 7. The autonomous precision landing method 800 comprises the following steps:

- Acquire 501 images 502 repeatedly during the landing sequence using a camera installed on board a drone and oriented towards the ground.

Detect 506, in each image 502 captured by the camera, a landing template 400 formed by a plurality of concentric circular crowns 402 of decreasing thickness by detecting 503 a predetermined number N of concentric circumferences in the corresponding image 502.

Carry out the autonomous landing 810 of the drone on the landing template 400 using as reference the positions 508 of the landing template 400 detected in images captured by the camera during landing.

Figure 9 illustrates the elements that make up an autonomous precision landing system 900 for drones according to an embodiment of the present invention. The autonomous precision landing system 900 includes elements mounted on a drone and responsible for the execution of the autonomous precision landing method 800 (FIG. 8) on board the drone during the landing sequence.

The autonomous precision landing system 900 comprises the following elements: a ground-facing camera 910, a landing template detection unit 920, and a command system 930. As shown in Figure 9, the detection unit landing template detection 920 and the command system 930 can be incorporated in a control unit 940 (implemented for example in a CPU, a processor or a microcontroller), or they can be totally independent elements on board the drone, not included in a common device. The autonomous precision landing system 900 may also comprise a height sensor 912 to provide the command system 930 with the HD height of the drone relative to the ground. The direction of the height measurement would preferably be along the plumb line or along the Down axis of a NED (North East Dow) system centered on the drone's center of mass.

The landing template detection unit 920 is configured to detect, in each image 502 received from the camera 910, a landing template 400 formed by a plurality of concentric circular annuluses 402 of decreasing thickness by detecting a predetermined number N of concentric circumferences in the corresponding image 502. Each time it detects a landing template 400 in a received image 502, the landing template detection unit 920 calculates the position 508 of the landing template, determined for example by the position (Xc.Yc) of the center 406 of the landing template 400, and provides it to the command system 930.

The command system 930 is configured to perform the autonomous landing of the drone on the landing template 400 using as reference the positions 508 of the landing template 400 detected by the landing template detection unit 920 in images captured by the camera 910. during landing.

In Figure 10 a drone 1010 is shown that incorporates the autonomous precision landing system 900 of Figure 9, where the camera 910 is focused towards a landing zone 1020 in which a landing platform 1030 is arranged that supports a landing template 400 arranged horizontally and oriented upwards. Landing template 400 may be attached or adhered in any way (e.g. by means of attachment or an adhesive element) to landing pad 1030, or painted on its upper outer surface. In one embodiment, the autonomous precision landing system 1000 comprises, in addition to the elements on board the drone and described in Figure 9, the landing template 400 and/or the landing platform 1030.

The drone 1010 is ready to initiate a precision landing 1002, in a position and at a height where the camera 910 can capture the landing template 400, which will be used as a visual aid during the execution of the precision landing.

According to the embodiment shown in Figure 9, the command system 930 is in turn made up of several modules: a command start management unit 932, a movement control unit 934, in charge of the general control of movements of the drone 1010 based on the inputs, and a speed controller 936 configured to generate the necessary corrections to center the drone with respect to the landing template 400 according to the detections of the landing template 400 made by the detection unit of landing template 920 during landing.

The 930 command system remains on standby until a series of conditions are met that allow the precision landing 1002 to be initiated. Said precision landing initiation conditions are automatically detected by the 932 command initiation management unit. In the event that any of the modules required for the operation is not available, for example due to connection errors, the program ends. The flow chart in Figure 11 shows different checks carried out by the start management unit of the command 932 before starting a precision landing 1002. First, the connection 1102 with the autopilot and the connection 1004 with the autopilot are checked. with the landing template detection unit 920, which is responsible for managing the camera 910, detecting the landing template 400 and returning the results related to its detection and its position. Such checks may include a maximum number of connection attempts.

It is then checked whether the conditions for switching to flight mode with automatic control (OFFBOARD conditions '1106) are met. In one embodiment, shown in the flowchart of Figure 12, said conditions include the following checks:

1. Being close to the takeoff point (HomePos) and flying in the RTL 1202 return home mode (“Return to Launch”, is a mode that allows the drone to return to the area from which it took off and land automatically).

2. Not be moving 1204 (or with a minimum movement, within a threshold), to know if it is already in the final position. This is done by evaluating the speeds on each axis of the 1010 drone.

3. Be in a similar orientation to the orientation it had at takeoff 1206 (takeoff orientation), as RTL mode leaves the drone facing the same place it took off.

4. Optionally, it is checked 1208 whether it is necessary or not to detect a reference code (e.g. ArUco tag, QR, ARtag, or similar) established, present in the landing zone 1020, as an additional check, and in which case it is checked if the reference code has been detected 1210 in an image (502) captured by the camera (910). The reference code can be incorporated into the landing pad 400 itself (exterior to the concentric annuluses 402) or, as shown in Figure 10 (reference code 1050), into the landing pad 1030 itself. of the reference code, any detection method known in the state of the art can be used.

All OFFBOARD' conditions 1106 have to be fulfilled one by one, for at least a specific time, according to a chain check, following a certain flow of checks (for example, according to the flow chart of Figure 12). In the event that any of the conditions fail, all the checks are repeated from the beginning. Returning to the flowchart of Figure 11, once it is determined that all the OFFBOARD' conditions are met 1106, it is checked 1108 if some conditions are met to finally proceed with the precision landing (precision landing conditions), which It is determined based on whether the wind speed levels exceed a certain limit or if there have been gusts of wind in the last moments (eg a few minutes) prior to landing.

In the event that the conditions for precise landing are met (e.g. wind intensity below a certain threshold and/or without wind gusts for a certain time prior to landing), precision landing 1002 is initiated. If the precision landing conditions are not met, an alternative landing 1110 is initiated, in which the drone is directed automatically, using guidance techniques already known in the state of the art, to an alternative place to land, for example to a point defined by the user in the system configuration. This alternative point or location can be defined by setting a distance from the takeoff point (HomePos) to the desired point and an orientation (North, South, East, West or a more specific heading in degrees). Once the alternate landing site is reached, the drone goes into autopilot landing mode. This precision landing, with maximum errors of a few centimeters, allows the integration of other systems to automate the maintenance of the drone or leave it protected, such as the integration with the so-called Nests for drones ("Drone Nest") or landing in spaces with limited dimensions, while a non-precision landing system or alternate landing are aimed at landing safely, but for a use case where it is acceptable to assume the 3 m errors that can be had with a GNSS receiver conventional.

A flowchart of the state control performed by the motion control unit 934 during a precision landing 1002 is shown in Figure 13. The drone initiates the precision landing 1002 in a stationary state 1302 (no movement, according to the check made in 1204). The detection of the landing template by the landing template detection unit 920 is checked 1304 by analyzing at least one image captured by the camera 910 based on the previously described detection algorithm. If the landing template is not detected, it is checked 1306 whether the drone is positioned at a given maximum height; in the affirmative case, it returns to the stationary state 1302 to check again 1304 the detection of the landing template, and in Otherwise, the drone goes to an ascension state 1308 to reach the maximum height determined from which it can detect the landing template and start the landing sequence. In the event that the drone remains hovering 1302 for a certain time without detecting the landing template, the drone abandons the precision landing 1002 and initiates an alternate landing 1110 instead. Similarly, if from the start precision landing 1002 an internal time counter reaches a certain maximum value, precision landing 1002 is canceled and alternate landing 1110 proceeds.

If landing template 400 is detected, the drone enters a horizontal approach state 1310 in order to center the drone horizontally with respect to landing template 400 (i.e., center drone 1010 at coordinates (Xc.Yc ) of the center 406 of the landing template 400). Once the drone enters said horizontal approach state 1310, it checks 1312 the detection of the landing template in at least one image captured by the camera 910. If it does not detect the landing template 400 for a certain period of time (during the checks the drone remains stationary), it is checked 1306 if the drone is positioned at a maximum height.

In the event that the landing template is detected, it proceeds to check 1314 if the drone is centered on it. If it is not centered, it returns to the horizontal approach state 1310 to give the appropriate speed commands that allow the drone 1010 to center on the landing template 400 detected. This horizontal approach process is repeated until the drone is centered on the landing template, at which point the drone enters a vertical approach state 1316 in order to gradually descend and vertically approach the landing template 400.

During the vertical approach state 1316 the drone descends vertically, repeatedly checking during the descent if the template is still detected 1318 and if the drone is still centered 1320. In case the landing template is no longer detected (for example, when a strong crosswind gust has occurred that has substantially displaced the drone and prevents it from detecting the landing template at its current height), the drone enters climb state 1308 until template 1304 detections are obtained again or the landing template is reached. the established maximum flight height 1306, in which case it will return to the stationary state 1302, as in the beginning of the precision landing 1002. If during the descent it is verified that the drone is no longer centered, it returns to a state of horizontal zoom 1310 to center the drone on the landing template. As the drone 1010 descends and remains centered on the landing template 400, it is checked 1322 whether the drone is positioned at the landing height (eg at the height of the landing template), at which point which enters a landing state 1324 and terminates the precision landing sequence 1002.

The motion control unit 934 uses the speed controller 936 in the horizontal approach state 1310 to center the drone 1010 on the landing template 400 during precision landing 1002. The speed controller 936 is responsible for generating the speeds of correction necessary to center the drone, depending on the position (Xc.Yc) of the landing template detected at each instant. The speed controller 936 is also responsible for generating the vertical ascent (in climb state 1308) and descent (in vertical approach state 1316) speeds of the drone.

The implemented speed controller 936 is preferably based on a proportional controller with maximum and minimum output cutoff. This cutoff can be set based on the input offset instead of being a fixed preset value.

In one embodiment, the speed controller 936 inputs are as follows:

Offset on the X-axis (offset _x ) in pixels between the center of the image 502 and the center (Xc.Yc) of the landing template 400.

Offset on the Y axis (offset _y ) in pixels between the center of the image 502 and the center (Xc.Yc) of the landing template 400.

- Height ( HD ) of the drone with respect to the ground, acquired by a height sensor 912 installed on the drone, such as a small laser rangefinder.

Proportional gain ( g ).

Percentage ( MaxSpeedPer ) to set the maximum and minimum step value based on the input deviation.

The maximum output speed value on the horizontal X axis ( MaxSpeed _x ) and on the horizontal Y axis ( MaxSpeed _y ) is calculated as:

MaxSpeed _x = abs(deviation _x ) + MaxSpeedPer x abs(deviation _x )

MaxSpeed _y = abs(deviation _y ) + MaxSpeedPer x abs (deviation _y ) The speed to command on each horizontal axis (X,Y) is calculated as follows:

1. Calculate the physical distance between the drone 1010 and the center (Xc.Yc) of the landing template 400 on each horizontal axis from the offset _x and the offset _y . This distance will be what is called Physical Error. This is done by calculating the Ground Sampling Distance (GSD). This calculation manages to establish, from a height value H _D and the properties of the camera (resolution, field of view -FOV- and focal length of the lens), how much physical distance is equivalent to each pixel of the image, so knowing the difference in pixels ( offset _x , offset _y ) between the center of the image 502 and the center (Xc.Yc) of the landing template 400, it is possible to calculate the physical distance in the horizontal plane (axes X and Y) . The accuracy of the calculation largely depends on the quality of the height measurement as well as the perpendicularity between the camera 910 (the main lens axis) and the landing template 400. The distance measured by the height sensor 912 is corrects when the drone 1010 is not perpendicular to the plane of the landing template 400. In real landing situations, the pitch angles of the drone 1010 when moving to apply the corrections or due to the need to overcome the wind are reduced, so they do not pose a problem for the calculation of the Physical Error with the camera.

2. The horizontal speed (V _x , V _y ) of the output to be commanded in each horizontal axis (X,Y) is calculated by applying the proportional controller:

V _x = PhysicalError _x xg

V _y = ErrorPhysicalOy xg

3. If the speed to be commanded on an axis (OutputVelocity _x , OutputVelocity _y ) exceeds the maximum output speed ( MaxSpeed _x , MaxSpeed _y ) or minimum (opposite value) for said axis, the speed is limited to the output speed maximum corresponding to said axis.

Claims

1. Autonomous precision landing system for drones, characterized in that it comprises: a camera (910) installed on board a drone (1010); a landing template detection unit (920), mounted on the drone (1010) and configured to detect, in an image (502) captured by the camera (910), a landing template (400) formed by a plurality of concentric circular annuluses (402) of decreasing thickness by detecting a predetermined number N of concentric circles in the image (502); a command system (930), embedded in the drone (1010) and configured to perform an autonomous landing of the drone (1010) on the landing template (400) using as reference the positions (508) of the landing template (400 ) detected in images (502) captured by the camera (910) during landing. 2. System according to claim 1, characterized in that for the detection of a predetermined number N of concentric circumferences in an image (502) captured by the camera (910), the landing template detection unit (920) is configured to : detect circumferences in the image (502); for each circumference detected (522), performing a recursive search for N-1 inner concentric circumferences, where the recursive search comprises: cropping the image (502) to eliminate the detected circumference (522), obtaining a cropped image (526); iteratively until detecting N-1 concentric circles inside the detected circle (522): detect an additional circle (530) in each cropped image (526,538), check that each additional circle (530) is concentric, considering a margin of error , to the detected circumference (522), and crop each cropped image (526,538) to eliminate the corresponding additional circumference (530), obtaining a cropped image (538). System according to claim 2, characterized in that the landing template detection unit (920) is configured to detect circumferences (522,530) by means of the Hough transform. 4. System according to any of claims 2 to 3, characterized in that the cropping of each image (502,526,538) comprises reducing the rectangle (708) that contains the circumference (702) by a certain percentage. System according to any of the preceding claims, characterized in that the landing template detection unit (920) is configured to dynamically determine the number N of concentric circumferences. 6. System according to claim 5, characterized in that the number N of concentric circles is determined dynamically as a function of the distance between the drone (1010) and the landing template (400). 7. System according to any of the preceding claims, characterized in that the command system (930) comprises a speed controller (936) configured to generate horizontal speed corrections (V _x , V _y ) to center the drone (1010 ) relative to the landing template (400). 8. System according to claim 7, characterized in that it further comprises a height sensor (912) configured to obtain the height (HD) of the drone (1010) with respect to the ground; and because the speed controller (936) is configured to: determine the distance between the drone (1010) and the center (Xc.Yc) of the landing template (400) in each horizontal axis by calculating the distance of soil sample, GSD, using the height (HD) of the drone (1010); and control the horizontal speed (V _x , V _y ) of the drone (1010) proportional to the distance to the center (Xc.Yc) of the landing template (400) determined in each axis, limiting the horizontal speed (V _x , V _y ) on each axis if it exceeds a certain limit (MaxSpeed _x , MaxSpeed _y ). 9. System according to claim 8, characterized in that the maximum horizontal speed limit (MaxSpeed _x , MaxSpeed _y ) on each axis (X, Y) is set dynamically based on the distance to the center (Xc.Yc) of the landing template (400) determined in the corresponding axis (X,Y). 10. System according to any of claims 8 to 9, characterized in that the command system (930) comprises a movement control unit (934) configured to control a descent of the drone (1010) towards the landing template (400) and, during descent, repeatedly detecting the landing template (400) in images (502) captured by the camera (910) and centering the drone (1010) with respect to the detected landing template (400) using the speed controller (936). 11. System according to any of the preceding claims, characterized in that the command system (930) comprises a command initiation management unit (932), configured to detect the fulfillment of conditions to initiate a precision landing (1002) . System according to claim 11, characterized in that the conditions for initiating a precision landing (1002) comprise: checking (1202) the proximity to the takeoff point and the activation of the RTL return home mode; check (1204) that the drone is not moving; checking (1206) that the drone is in an orientation similar to the takeoff orientation; checking (1108) that the wind speed does not exceed a certain limit and/or that there have been no gusts of wind during a certain time. 13. System according to claim 12, characterized in that the conditions for initiating a precision landing (1002) additionally comprise the detection (1210) of a reference code (1050) in a landing zone (1020) in which it is located. the landing template (400). System according to any of the preceding claims, characterized in that it comprises a landing platform (1030) that supports the landing template (400) arranged horizontally. 15. Drone with autonomous precision landing, characterized in that it includes the system autonomous precision landing gear according to any of claims 1 to 13. 16. Autonomous precision landing method for drones, characterized in that it comprises: repeatedly acquiring (501) images (502) by means of a camera (910) installed on board a drone (1010); detect (506), in each image (502) captured by the camera (910), a landing template (400) formed by a plurality of concentric circular crowns (402) of decreasing thickness by detecting (503) a predetermined number N of concentric circles in the image (502); perform (810) an autonomous precision landing (1002) of the drone (1010) on the landing template (400) using as a reference the positions (508) of the landing template (400) detected in images (502) captured by the camera (910) during landing. 17. Method according to claim 16, characterized in that the detection (503) of a predetermined number N of concentric circumferences in an image (502) captured by the camera (910) comprises: detecting (520) circumferences in the image (502) ; for each circumference detected (522), perform a recursive search for N-1 inner concentric circumferences, where the recursive search comprises: cropping (524) the image (502) to eliminate the detected circumference (522), obtaining a cropped image (526 ); iteratively until detecting N-1 concentric circles inside the detected circle (522): detect (528) an additional circle (530) in each cropped image (526,538), check (532) that each additional circle (530) is concentric , considering a margin of error, to the detected circumference (522), and cropping (536) each cropped image (526,538) to eliminate the corresponding additional circumference (530), obtaining a cropped image (538). 18. Method according to claim 17, characterized in that the detection of circumferences (522,530) is carried out by means of the Hough transform. 19. Method according to any of claims 17 to 18, characterized in that cropping each image (502,526,538) comprises reducing the rectangle (708) that contains the circumference (702) by a certain percentage. 20. Method according to any of claims 16 to 19, characterized in that the number N of concentric circumferences is dynamically determined at each detection. 21. Method according to claim 20, characterized in that the number N of concentric circumferences is dynamically determined at each detection as a function of the distance from the drone (1010) to the landing template (400). 22. Method according to any of claims 16 to 21, characterized in that it comprises generating corrections in the horizontal speed (V _x , V _y ) to center the drone (1010) with respect to the landing template (400). 23. Method according to claim 22, characterized in that the corrections in the horizontal speed (V _x , V _y ) are determined according to the following steps: obtain the height (H _D ) of the drone (1010) with respect to the ground by a height sensor (912); determine the distance between the drone (1010) and the center (Xc.Yc) of the landing template (400) on each horizontal axis by calculating the ground sample distance, GSD, using the height (HD) of the drone (1010); and control the horizontal speed (V _x , V _y ) of the drone (1010) proportional to the distance to the center (Xc.Yc) of the landing template (400) determined in each axis, limiting the horizontal speed (V _x , V _y ) on each axis if it exceeds a certain limit (MaxSpeed _x , MaxSpeed _y ). 24. Method according to claim 23, characterized in that the maximum horizontal speed limit (MaxSpeed _x , MaxSpeed _y ) in each axis (X, Y) is set dynamically as a function of the distance to the center (Xc.Yc) of the template landing point (400) determined in the corresponding axis (X, Y). 25. Method according to any of claims 23 to 24, characterized in that it comprises performing a descent of the drone (1010) towards the landing template (400) and, during the descent, repeatedly detect the landing template 400 in images 502 captured by the camera 910 and center the drone 1010 with respect to the detected landing template 400 by generating the corrections in the horizontal velocity (V _x , V _y ). 26. Method according to claim 25, characterized in that it comprises checking, during the descent of the drone (1010) towards the landing template (400), the detection of the landing template (400) and, in the event that it is not detected the landing template (400), perform a drone ascent (1010) until the landing template (400) is detected again or until reaching a maximum height. 27. Method according to claim 26, characterized in that it comprises, in the event that the landing template (400) is not detected for a certain time, initiating an alternative landing (1110) in an area outside the landing template (400). ). 28. Method according to any of claims 16 to 27, characterized in that it comprises detecting the fulfillment of conditions to initiate a precision landing (1002). 29. Method according to claim 28, characterized in that the conditions for initiating a precision landing (1002) comprise: checking (1202) the proximity to the takeoff point and the activation of the RTL return home mode; check (1204) that the drone is not moving; checking (1206) that the drone is in an orientation similar to the takeoff orientation; checking (1108) that the wind speed does not exceed a certain limit and/or that there have been no gusts of wind during a certain time. 30. Method according to claim 29, characterized in that the conditions for initiating a precision landing (1002) additionally comprise the detection (1210) of a reference code (1050) in a landing zone (1020) in which it is located the landing template (400). 31. A program product comprising program instruction means for carrying out the method defined in any one of claims 16 to 28 when the program runs on a processor. 32. A program support medium, which stores the program product according to claim 31.