KR102772340B1 - Image-based vertiport detection system and landing guidance method for precision landing of air mobility - Google Patents

Abstract

A method for operating an aircraft may include: obtaining at least one image in which a marker for guiding a landing point of the aircraft is captured by a video capturing device; recognizing the marker based on a shape of the marker in the at least one image; estimating relative pose information including relative position information and direction information between the marker and the aircraft based on a result of recognizing the marker; and providing guide information for guiding the aircraft to land on the landing point based on the relative pose information.

Description

{IMAGE-BASED VERTIPORT DETECTION SYSTEM AND LANDING GUIDANCE METHOD FOR PRECISION LANDING OF AIR MOBILITY}

It relates to aircraft and methods of operating aircraft.

Urban Air Mobility (UAM) is a type of personal aircraft capable of vertical takeoff and landing, and is emerging as a next-generation mobility solution that maximizes transportation efficiency in urban areas by using the sky as a new transportation route.

Urban air traffic requires separate takeoff and landing facilities, and requires technology to safely approach the takeoff and landing facilities and make precise landings.

U.S. Patent No. US 11,783,575

We aim to provide guidance so that aircraft can safely approach the landing point and land precisely. In particular, we aim to enable precise landing of urban air traffic at vertiports, which are takeoff and landing airfields for urban air traffic.

According to one aspect, a method for operating an aircraft is provided, including: a step of obtaining at least one image in which a marker for guiding a landing point of an aircraft is captured through an image capturing device; a step of recognizing the marker based on a shape of the marker in the at least one image; a step of estimating relative pose information including relative position information and direction information between the marker and the aircraft based on a result of recognizing the marker; and a step of providing guide information for guiding the aircraft to land on the landing point based on the relative pose information.

According to one embodiment, the step of recognizing the marker may include a step of recognizing the marker by asymmetrically dividing the marker based on a shape of the marker within the at least one image, into a first symbol for calculating coordinates of a landing point of the aircraft, and a second symbol that serves as a reference for estimating the relative pose information.

According to one embodiment, the step of recognizing the marker may include the step of extracting a surrounding area including the marker within the at least one image as a filtering area, and recognizing the first symbol and the second symbol based on a result of applying a binarization algorithm to the filtering area.

According to one embodiment, the step of recognizing the marker may further include the step of recognizing a lighting device located around the landing point through the imaging device.

According to one embodiment, the relative position information between the marker and the aircraft may include relative distance information and relative altitude information between the marker and the aircraft.

According to one embodiment, the step of estimating the relative pose information may include the step of calculating a length of the second symbol in an image plane based on pixel information of the at least one image; and the step of calculating the relative distance information based on a ratio of the actual length of the second symbol and the length of the second symbol in the image plane.

According to one embodiment, the step of estimating the relative pose information may include the step of calculating the relative altitude information based on a ratio of an actual width of the second symbol and an width of the second symbol in an image plane and a focal length of the image capturing device.

According to one embodiment, the step of estimating the relative pose information may include the step of calculating the direction information based on an angle between a first axis directed from the center of the second symbol to the center of the first symbol and a second axis directed from the center of the first symbol to the center of the aircraft.

According to one embodiment, the step of providing the guide information based on the relative pose information may include the step of visually displaying the relative position information and the direction information.

According to one embodiment, the step of providing the guide information may further include the step of indicating an expected path of the aircraft based on the relative position information and the direction information.

According to one embodiment, the step of providing the guide information based on the relative pose information may include the step of adjusting and displaying the size of at least one of the model representing the landing point and the model of the aircraft based on at least one of the relative position information and the direction information.

According to another aspect, an aircraft is provided, comprising: a communication device; an image capturing device; a user operating device; a processor; and a memory storing commands executable by the processor, wherein the processor, by executing the commands, obtains at least one image in which a marker for guiding a landing point of the aircraft is captured through the image capturing device, recognizes the marker based on a shape of the marker in the at least one image, estimates relative pose information including relative position information and direction information between the marker and the aircraft based on a result of recognizing the marker, and provides guide information for guiding the aircraft to land on the landing point based on the relative pose information.

According to one embodiment, the processor, by executing the instructions, can asymmetrically segment the marker based on a shape of the marker within the at least one image, and recognize it as a first symbol for calculating coordinates of a landing point of the aircraft and a second symbol that serves as a reference for estimating the relative pose information.

According to one embodiment, the processor, by executing the instructions, can calculate a length of the second symbol within an image plane based on pixel information of the at least one image, calculate relative distance information based on a ratio of the actual length of the second symbol and the length of the second symbol within the image plane, and calculate relative altitude information based on a ratio of the actual width of the second symbol and the width of the second symbol within the image plane and a focal length of the image capturing device.

According to one embodiment, the processor, by executing the instructions, can calculate the direction information based on an angle between a first axis directed from the center of the second symbol to the center of the first symbol and a second axis directed from the center of the first symbol to the center of the aircraft.

According to one embodiment, the processor can visually display the relative position information and the direction information through the user operation device by executing the instructions.

According to another aspect, a computer program stored in a computer-readable storage medium for executing a method for operating an aircraft, the method for operating an aircraft includes: a step of obtaining at least one image in which a marker for guiding a landing point of the aircraft is captured by an image capturing device; a step of recognizing the marker based on a shape of the marker in the at least one image; a step of estimating relative pose information including relative position information and direction information between the marker and the aircraft based on a result of recognizing the marker; and a step of providing guide information for guiding the aircraft to land on the landing point based on the relative pose information.

It can provide guidance so that aircraft can safely approach the landing point and land precisely. In particular, it can guide urban air traffic to land precisely at a vertiport, which is an urban air traffic takeoff and landing field.

By allowing urban air traffic to safely take off and land at vertiports, accessibility within the city center can be secured.

Additionally, urban air traffic can improve service convenience, such as battery charging, passenger boarding, and shortened turnaround times.

The present disclosure can be readily understood by the combination of the following detailed description and the accompanying drawings, in which reference numerals refer to structural elements.
FIG. 1a is a conceptual diagram illustrating a method of operating an aircraft for landing using a marker, according to one embodiment.
FIG. 1b is a diagram illustrating parameters used when calculating a relative pose between an aircraft and a marker, according to one embodiment.
Figure 2 is a flowchart showing a method of operating an aircraft according to one embodiment.
FIG. 3 is a diagram for explaining a process of recognizing a marker and estimating relative pose information according to an embodiment.
FIG. 4 is a drawing for explaining a process for calculating the relative distance between a marker and an aircraft, according to one embodiment.
FIG. 5 is a drawing for explaining a process for calculating the relative altitude between a marker and an aircraft, according to one embodiment.
FIG. 6 is a drawing for explaining a process for calculating the direction between a marker and an aircraft, according to one embodiment.
FIGS. 7A and 7B are drawings for explaining guide information displayed on a user operation device according to one embodiment.
Figure 8 is a block diagram illustrating the configuration of an aircraft according to one embodiment.

Hereinafter, various embodiments will be described in detail with reference to the drawings. The embodiments described below may be modified and implemented in various different forms. In order to more clearly explain the features of the embodiments, detailed descriptions of matters that are widely known to those skilled in the art to which the embodiments below belong will be omitted.

Meanwhile, when it is said in this specification that a certain configuration is "connected" to another configuration, this includes not only the case where it is "directly connected" but also the case where it is "connected with another configuration in between." In addition, when it is said that a certain configuration "includes" another configuration, this means that, unless specifically stated otherwise, it does not exclude other configurations but may further include other configurations.

Additionally, terms including ordinal numbers, such as "first" or "second," used herein may be used to describe various components, but the components should not be limited by the terms. The terms are used only for the purpose of distinguishing one component from another.

In this specification, "aircraft" may refer to a vehicle or machine with wings that can fly in the air. In addition, the aircraft may refer to a manned aircraft or an unmanned aircraft. In addition, the aircraft may refer to an aircraft that is directly piloted by boarding and controlling the aircraft or that is operated by automatically/semi-automatically controlling the flight by a program on the ground without a pilot on board. In addition, the aircraft may correspond to urban air mobility (UAM).

In this specification, a "marker" may be an object displayed to guide a landing point of an aircraft. The marker may be installed within an airfield where an aircraft takes off and lands. The marker may be for identifying a vertiport for urban air traffic to take off and land. The marker may be installed in a preset shape. Here, the preset shape may be a shape that complies with standard specifications.

Hereinafter, specific embodiments of the present invention described above will be described with reference to FIGS. 1a to 8.

FIG. 1a is a conceptual diagram illustrating a method of operating an aircraft for landing using a marker, according to one embodiment.

Referring to FIG. 1a, the aircraft (10) can capture a marker (20) for guiding a landing point through an image capturing device within the aircraft (10) and obtain an image including the marker (20).

In order for the aircraft (10) to safely approach and land at a landing point, the aircraft (10) can provide guide information to the user based on a marker (20) recognized within the captured image.

Here, the guide information may include relative position information and direction information between the center point of the marker (20) and the center point of the aircraft (10). The relative position information may include relative distance information in the x-axis direction and the y-axis direction derived from a two-dimensional image captured by an image capturing device. In addition, the relative position information may include relative altitude information on which the aircraft (10) is located with respect to the actual plane on which the marker (20) is located.

Additionally, the guide information may include guide line information that enables the aircraft (10) to land precisely based on relative position information and direction information.

FIG. 1b is a drawing for explaining parameters used when calculating a relative pose between an aircraft (10) and a marker (20), according to one embodiment.

Referring to the image (110) of Fig. 1b, a marker (20) is positioned on an actual plane at a point where an aircraft (10) lands, and a relative position between the marker (20) and the aircraft (10) can be calculated based on a distance between the center of the aircraft (10) and the center of the marker (20). The relative position can be expressed by a relative distance on a two-dimensional plane and a relative altitude of the aircraft (10) with respect to the actual plane.

In addition, referring to the image (120) of Fig. 1b, the direction information of the aircraft (10) with respect to the marker (20) can be calculated by the angle between a line connecting the center of the marker (20) and the center of the aircraft (10) and a predetermined axis.

Figure 2 is a flow chart showing an operation method of an aircraft (10) according to one embodiment.

Referring to FIG. 2, in step S210, the aircraft (10) can obtain at least one image in which a marker (20) for guiding the landing point of the aircraft (10) is captured through an image capturing device.

In step S220, the aircraft (10) can recognize the marker (20) based on the shape of the marker (20) within at least one image.

For example, the aircraft (10) can asymmetrically segment the marker (20) based on the shape of the marker (20) within at least one image. Specifically, the aircraft (10) can recognize the marker (20) by segmenting it into a first symbol for calculating the coordinates of the landing point and a second symbol that serves as a reference for estimating relative pose information.

In addition, the aircraft (10) can recognize the marker (20) in at least one image using a marker recognition model. The marker recognition model may be a model that recognizes the marker (20) using a histogram-based image binarization algorithm. For example, the marker recognition model may analyze the histogram of the image to determine an optimal threshold value for binarizing the image. Here, the threshold value may be a reference value for distinguishing the marker (20) and may be a value for separating the pixels corresponding to the area of the marker (20) and the pixels corresponding to the area other than the marker (20). The marker recognition model may binarize the image based on the threshold value. For example, the marker recognition model may be a learning model that recognizes the marker (20) from an image in which the marker (20) is captured. In the marker recognition model, when an image in which the marker (20) is captured is input as an input value, the result of recognizing the marker (20) may be output as an output value.

In addition, the marker recognition model can divide the captured image into a grid, and set the area around the marker as a filtering area based on the presence or absence of the marker in each grid cell. The marker recognition model can recognize the marker (20) by using a histogram-based image binarization algorithm for the filtering area. By setting the filtering area, the range to which the binarization algorithm is applied can be reduced, and the influence of illumination changes that may occur when performing binarization can be reduced.

In addition, the marker recognition model can extract an area within a preset range including the marker (20) within the captured image as a filtering area. For example, the area within the preset range can be a minimum area including a symbol within the marker (20). The marker recognition model can asymmetrically divide the marker (20) and recognize a first symbol and a second symbol based on the result of applying a histogram-based image binarization algorithm to the filtering area. The first symbol is a symbol for calculating the coordinates of the landing point of the aircraft (10), and the second symbol can be a symbol that serves as a reference for estimating relative pose information. The process of recognizing the marker (20) is described in FIG. 3.

In addition, the aircraft (10) can recognize a marker (20) located at the landing point through an imaging device. For example, the imaging device can be an infrared (IR) camera to support precision landing of the aircraft (10) even at night or under low visibility conditions. The infrared (IR) camera can recognize an IR sensor attached to a marker (20) located at the landing point.

In step S230, the aircraft (10) can estimate relative pose information including relative position information and direction information between the marker (20) and the aircraft (10) based on the result of recognizing the marker (20). Here, the relative position information between the marker (20) and the aircraft (10) can include relative distance information and relative altitude information between the marker (20) and the aircraft (10).

For example, the aircraft (10) can calculate the length of the second symbol within the image plane based on pixel information of at least one image. The aircraft (10) can calculate relative distance information based on a ratio of the actual length of the second symbol and the length of the second symbol within the image plane. A process of calculating relative distance information is described in FIG. 4.

For example, the aircraft (10) can calculate relative altitude information based on the ratio of the actual width of the second symbol and the width within the image plane of the second symbol and the focal length of the imaging device. The process of calculating relative altitude information is described in Fig. 5.

For example, the aircraft (10) can calculate direction information based on the angle between the first axis from the center of the second symbol to the center of the first symbol and the second axis from the center of the first symbol to the center of the aircraft (10). The process of calculating direction information is described in Fig. 6.

In step S240, the aircraft (10) can provide guide information to guide the aircraft (10) to land at a landing point based on relative pose information.

For example, the aircraft (10) can visually display relative position information and direction information. Specifically, the aircraft (10) can display an expected path of the aircraft (10) based on the relative position information and direction information. In addition, the aircraft (10) can display an expected path of the aircraft (10) based on at least one of the relative position information, direction information, the size and shape of the aircraft (10), and external environmental factors around the aircraft (10). Additionally, the aircraft (10) can display an expected path of the aircraft (10) using a navigation system and a control algorithm.

For example, the aircraft (10) can adjust and display the size of at least one of a model representing a landing point and a model of the aircraft (10) based on at least one of relative position information and direction information.

FIG. 3 is a drawing for explaining a process of recognizing a marker (20) and estimating relative pose information according to an embodiment.

Referring to FIG. 1b and FIG. 3, the marker (20) may be configured as a polygon. Depending on the position of the reference line dividing the marker (20), the sub-symbols divided from the marker (20) may be symmetrical or asymmetrical to each other. By utilizing asymmetrical symbols, the relative position information or direction information of the aircraft (10) with respect to the marker (20) can be easily calculated.

A processor within an aircraft (10) can recognize a marker (20) based on at least one image captured by an image capturing device. Based on the result of recognizing the marker (20), the processor can estimate relative pose information between the marker (20) and the aircraft (10).

Specifically, in step S310, the processor may detect a first symbol (301) from the marker (20). The first symbol (301) may be a symbol for calculating coordinates of a landing point of the aircraft (10). The number of line segments constituting the first symbol (301) may be greater than the number of line segments constituting the second symbol (302). In step S312, the processor may select a shape with a maximum area from a polygon corresponding to the first symbol. For example, the shape with a maximum area may be a circle. In step S314, the processor may calculate a center of the circle.

In step S320, the processor can detect a second symbol (302) from the marker (20). The second symbol (302) can be a symbol that serves as a reference for estimating relative pose information. A polygon corresponding to the second symbol (302) can be characterized by an angle of 90 degrees. In step S322, the processor can select a rectangle that is a characteristic of the polygon having an angle of 90 degrees. In step S324, the processor can calculate the center of the rectangle.

The operations of steps S310 to S314 and the operations of steps S320 to S324 may be performed simultaneously or sequentially.

In step S330, the processor can estimate force information using the center of the circle and the center of the rectangle. The process of estimating force information is described in FIGS. 4 to 6.

FIG. 4 is a drawing for explaining a process of calculating the relative distance between a marker (20) and an aircraft (10) according to one embodiment.

Image (410) of Fig. 4 represents an image plane captured by an imaging device, and image (420) of Fig. 4 represents an actual plane at the landing point of the aircraft (10).

A processor within the aircraft (10) can calculate a length (l') within the image plane of the second symbol (302) based on pixel information of the image (410). Additionally, the processor can calculate a distance from the center of the first symbol (301) to the x-axis in the image (410) as △y'.

The processor can calculate relative distance information between the marker (20) and the aircraft (10) based on a ratio of the actual length (l) of the second symbol (302) and the length (l') of the second symbol (302) within the image plane. Specifically, the processor can calculate △y, which is a distance from the center of the first symbol (301) to the x-axis in the actual plane, by mathematical expression 1.

FIG. 5 is a drawing for explaining a process of calculating the relative altitude between a marker (20) and an aircraft (10) according to one embodiment.

Referring to FIG. 5, the object (410) represents an image capturing device, the plane (520) represents an image plane, and the plane (530) represents an actual plane at the landing point of the aircraft (10).

In addition, the focal length (f) may represent the distance from the imaging device to the image plane (520), and the altitude (h) may represent the distance from the imaging device to the actual plane (530). The relationship between the distance (d') within the image plane (520) and the distance (d) within the actual plane (530) is as shown in mathematical expression 2.

The processor in the aircraft (10) can calculate relative altitude information based on the ratio of the actual width (A) of the second symbol and the width (A') of the second symbol within the image plane and the focal length (f) of the image capturing device. Specifically, the processor can calculate relative altitude information by mathematical expression 3.

FIG. 6 is a drawing for explaining a process of calculating the direction between a marker (20) and an aircraft (10) according to one embodiment.

Referring to FIG. 6, the processor in the aircraft (10) can calculate direction information based on the angle between the first axis (610) that runs from the center of the second symbol (302) to the center of the first symbol (301) and the second axis (620) that runs from the center of the first symbol (301) to the center of the aircraft (10). Here, the range of the θ value representing the direction information can exceed -180 degrees and be within 180 degrees.

FIGS. 7A and 7B are drawings for explaining guide information displayed on a user operation device according to one embodiment.

The aircraft (10) can visually display relative position information and direction information. For example, as illustrated in FIG. 7a, a user operation device in the aircraft (10) can display an instrument panel interface (710) that reflects relative position information and direction information between the marker (20) and the aircraft (10). A model (711) of the aircraft (10) and a model (712) of the marker (20) can be displayed on the instrument panel interface (710), and the model (711) of the aircraft (10) and the model (712) of the marker (20) can be displayed in consideration of directionality. In addition, the user operation device can reflect relative position information and direction information on the instrument panel interface (710) in real time as the aircraft (10) moves.

For example, the user operating device can visually display relative position information and direction information. Specifically, as the aircraft (10) gets closer to the marker (20), the size of the model (712) of the marker (20) on the instrument panel interface (710) can be displayed to become larger. In this case, the size of the model (711) of the aircraft (10) can also be displayed to become larger.

Additionally, the user operating device may display numerical information (720) for the aircraft (10) to land at a landing point. For example, the numerical information (720) may include relative distance information (△x, △y) and direction information (△θ) calculated in two dimensions.

In addition, the user operation device can display the expected path of the aircraft (10) based on the relative position information and direction information. In addition, the aircraft (10) can display the expected path of the aircraft (10) based on at least one of the relative position information, direction information, the size and shape of the aircraft (10), and external environmental factors around the aircraft (10). In addition, the aircraft (10) can display the expected path of the aircraft (10) using a navigation system and a control algorithm.

Referring to FIG. 7b, the user operating device can display an instrument panel interface (730) reflecting relative position information and direction information. Specifically, the user operating device can display a model of an aircraft (10) reflecting relative position information and direction information centered on a marker (20) on the instrument panel interface (730).

Additionally, the user operating device can display visual information (740) and direction information (750) of the aircraft (10) as separate interfaces.

FIG. 8 is a block diagram illustrating the configuration of an aircraft (10) according to one embodiment.

Referring to FIG. 8, the aircraft (10) may include a communication device (810), an image capturing device (820), a user operating device (830), a memory (840), and a processor (850). However, not all of the illustrated components are essential components. The aircraft (10) may be implemented with more components than the illustrated components, and may be implemented with fewer components than the illustrated components. Hereinafter, the above components will be described. The aircraft (10) illustrated in FIG. 8 may correspond to the aircraft (10) described in FIGS. 1 to 7b in the same manner.

The communication device (810) can perform communication with an external device. For example, the communication device (810) can perform communication with an external device by being connected to a network by wire or wirelessly. Here, the external device can be an electronic device.

The communication device (810) may include a communication module that supports one of various wired and wireless communication methods. The communication module may be a short-range communication module or a wired communication module.

The image capturing device (820) can acquire image frames such as still images or moving images in a capturing mode. Images captured by the image capturing device (820) can be processed by the processor (850). For example, the image capturing device (820) can be a camera. A plurality of cameras can be installed in the aircraft (10).

Image frames processed in the video capturing device (820) can be stored in memory (840) or transmitted externally through a communication device (810).

The user operating device (830) may mean a device that receives a signal or input from a user to control the aircraft (10).

The processor (850) can control the user operation device (830) to generate and output a user interface screen for receiving a predetermined command or signal from a user. The user operation device (830) can include an input device for receiving an input for controlling the operation of the aircraft (10) and an output device for displaying information such as the operation of the aircraft (10) and the status of the aircraft (10). For example, the user operation device (830) can include an operation panel for receiving a user input, a display panel for displaying a screen, etc.

For example, the input device may include devices capable of receiving various forms of user input, such as a touch screen or a microphone. In addition, the output device may include, for example, a display panel or a speaker. However, the user operation device (830) is not limited thereto and may include devices that support various input/output.

The memory (840) can store at least one program for executing an operation method of the aircraft (10) that recognizes a marker (20) based on an image acquired through an image capturing device (820), estimates relative pose information of the aircraft (10) with respect to the marker (20) based on a result of recognizing the marker (20), and provides guide information for guiding the landing of the aircraft (10). The at least one program stored in the memory (840) can be classified into a plurality of modules according to function.

The processor (850) controls the overall operation of the aircraft (10) and may include at least one processor, such as a CPU. The processor (850) may include at least one specialized processor corresponding to each function, or may be a processor integrated into one.

The processor (850) may execute a program stored in the memory (840), read data or a file stored in the memory (840), or store a new file in the memory (840). In addition, the processor (850) may execute instructions stored in the memory (840).

The video recording device (820) can capture a marker (20) for guiding the landing point of the aircraft (10) and obtain at least one image.

The processor (850) can recognize the marker (20) based on the shape of the marker (20) within at least one image.

For example, the processor (850) may asymmetrically segment the marker (20) based on the shape of the marker (20) within at least one image. Specifically, the processor (850) may recognize the marker (20) by segmenting it into a first symbol for calculating the coordinates of the landing point and a second symbol that serves as a reference for estimating relative pose information.

Additionally, the processor (850) can recognize a sensor attached to a marker through an image capturing device (820).

In addition, the processor (850) can recognize the marker (20) in at least one image using a marker recognition model. The marker recognition model may be a model that recognizes the marker (20) using a histogram-based image binarization algorithm. For example, the marker recognition model may analyze the histogram of the image to determine an optimal threshold value for binarizing the image. Here, the threshold value may be a reference value for distinguishing the marker (20) and may be a value for separating the pixel corresponding to the area of the marker (20) and the pixel corresponding to the area other than the marker (20). The marker recognition model may binarize the image based on the threshold value. For example, the marker recognition model may be a learning model that recognizes the marker (20) from an image in which the marker (20) is captured. In the marker recognition model, when an image in which the marker (20) is captured is input as an input value, the result of recognizing the marker (20) may be output as an output value.

For example, a marker recognition model can divide a captured image into a grid, and set the area around a marker as a filtering area based on the presence or absence of a marker in each grid cell. The marker recognition model can recognize a marker (20) by using a histogram-based image binarization algorithm for the filtering area. By setting a filtering area, the range to which the binarization algorithm is applied can be reduced, and the influence of illumination changes that may occur when performing binarization can be reduced.

For example, the marker recognition model can extract a surrounding area including a marker (20) in a captured image as a filtering area. The marker recognition model can asymmetrically divide the marker (20) and recognize a first symbol and a second symbol based on the result of applying a histogram-based image binarization algorithm to the filtering area. The first symbol can be a symbol for calculating the coordinates of the landing point of the aircraft (10), and the second symbol can be a symbol that serves as a reference for estimating relative pose information.

The processor (850) can estimate relative pose information including relative position information and direction information between the marker (20) and the aircraft (10) based on the result of recognizing the marker (20). Here, the relative position information between the marker (20) and the aircraft (10) can include relative distance information and relative altitude information between the marker (20) and the aircraft (10).

For example, the processor (850) can calculate the length of the second symbol within the image plane based on pixel information of at least one image. The processor (850) can calculate relative distance information based on a ratio of the actual length of the second symbol and the length of the second symbol within the image plane.

For example, the processor (850) can calculate relative elevation information based on a ratio of the actual width of the second symbol to the width within the image plane of the second symbol and the focal length of the imaging device (820).

For example, the processor (850) can calculate direction information based on the angle between a first axis from the center of the second symbol to the center of the first symbol and a second axis from the center of the first symbol to the center of the aircraft (10).

The processor (850) can provide guide information to guide the aircraft (10) to land at a landing point based on relative pose information.

For example, the user operation device (830) can visually display relative position information and direction information. Specifically, the user operation device (830) can display an expected path of the aircraft (10) based on the relative position information and direction information. The user operation device (830) can display an expected path of the aircraft (10) based on at least one of the relative position information, direction information, the size and shape of the aircraft (10), and external environmental factors around the aircraft (10). Additionally, the user operation device (830) can display an expected path of the aircraft (10) using a navigation system and a control algorithm.

For example, the user operation device (830) can adjust and display the size of at least one of a model representing a landing point and a model of an aircraft (10) based on at least one of relative position information and direction information.

The aircraft (10) described in the present disclosure may be implemented by hardware components, software components, and/or a combination of hardware components and software components. In addition, the present disclosure may be provided in the form of a computer program stored in a computer-readable storage medium so as to perform the operating method of the aircraft (10). In addition, the present disclosure may be written as a program that can be executed on a computer, and may be implemented in a general-purpose digital computer that operates such a program using a computer-readable storage medium.

Such computer-readable storage media may be read-only memory (ROM), random-access memory (RAM), flash memory, CD-ROMs, CD-Rs, CD+Rs, CD-RWs, CD+RWs, DVD-ROMs, DVD-Rs, DVD+Rs, DVD-RWs, DVD+RWs, DVD-RAMs, BD-ROMs, BD-Rs, BD-R LTHs, BD-REs, magnetic tape, floppy disks, magneto-optical data storage devices, optical data storage devices, hard disks, solid-state disks (SSDs), and any device capable of storing instructions or software, related data, data files, and data structures, or providing instructions or software, related data, data files, and data structures to a processor or a computer so that the processor or the computer may execute the instructions.

Although the embodiments have been described in detail above, the scope of the present invention is not limited thereto, and various modifications and improvements made by those skilled in the art using the basic concept of the present invention defined in the following claims also fall within the scope of the present invention.

Claims (17)

A step of acquiring at least one image of a marker for guiding a landing point of an aircraft within a vertiport, which is a takeoff and landing airfield for urban air traffic, using a video capturing device;
A step of recognizing the marker based on the shape of the marker within at least one image;
A step of estimating relative pose information including relative position information and direction information between the marker and the aircraft based on the result of recognizing the marker; and
Based on the above relative pose information, a step of providing guide information for guiding the aircraft to land at the landing point is included.
The step of recognizing the above marker is:
A step of recognizing a first symbol for calculating coordinates of a landing point of the aircraft and a second symbol as a reference for estimating the relative pose information by asymmetrically dividing the marker based on the shape of the marker within at least one image; and
Including a step of recognizing a lighting device located around the landing point through the video recording device,
The above marker is composed of a polygon,
The number of line segments constituting the first symbol is greater than the number of line segments constituting the second symbol,
The polygon corresponding to the second symbol above has an angle of 90 degrees,
How the aircraft operates. delete In the first paragraph,
The step of recognizing the above marker is:
A method for operating an aircraft, further comprising the step of extracting a minimum area including a symbol within a marker within at least one image as a filtering area, and recognizing the first symbol and the second symbol based on a result of applying a binarization algorithm to the filtering area. In the first paragraph,
The step of recognizing the above marker is:
A method of operating an aircraft, further comprising the step of recognizing a sensor attached to the marker through the video recording device. In the first paragraph,
The relative position information between the above marker and the above aircraft is,
A method of operating an aircraft, comprising relative distance information and relative altitude information between the marker and the aircraft. In paragraph 5,
The step of estimating the above relative pose information is,
A step of calculating a length within the image plane of the second symbol based on pixel information of at least one image; and
A method of operating an aircraft, comprising the step of calculating the relative distance information based on a ratio of an actual length of the second symbol and a length within an image plane of the second symbol. In Article 6,
The step of estimating the above relative pose information is,
A method of operating an aircraft, comprising the step of calculating the relative altitude information based on a ratio of an actual width of the second symbol and an width within an image plane of the second symbol and a focal length of the image capturing device. In the first paragraph,
The step of estimating the above relative pose information is,
A method for operating an aircraft, comprising the step of calculating the direction information based on an angle between a first axis directed from the center of the second symbol to the center of the first symbol and a second axis directed from the center of the first symbol to the center of the aircraft. In the first paragraph,
Based on the above relative pose information, the step of providing the guide information is:
A method of operating an aircraft, comprising the step of visually displaying the relative position information and the direction information. In Article 9,
The steps for providing the above guide information are:
A method for operating an aircraft, further comprising the step of displaying an expected path of the aircraft based on the relative position information and the direction information. In Article 9,
Based on the above relative pose information, the step of providing the guide information is:
A method for operating an aircraft, comprising the step of adjusting and displaying the size of at least one of a model representing the landing point and a model of the aircraft based on at least one of the relative position information and the direction information. communication device;
video recording device;
User-operated device;
processor; and
comprising a memory storing instructions executable by the processor;
The above processor, by executing the above instructions,
At least one image is obtained by capturing a marker for guiding the landing point of an aircraft within a vertiport, which is an urban air traffic takeoff and landing field, through the above video recording device,
Based on the shape of the marker within the at least one image, the marker is asymmetrically divided, and the marker is recognized as a first symbol for calculating the coordinates of the landing point of the aircraft and a second symbol that serves as a reference for estimating relative pose information, based on the shape of the marker within the at least one image.
Recognize the lighting devices located around the landing point through the above video recording device,
Based on the result of recognizing the marker, the relative pose information including relative position information and direction information between the marker and the aircraft is estimated,
Based on the above relative pose information, the aircraft provides guide information to guide landing at the landing point.
The above marker is composed of a polygon,
The number of line segments constituting the first symbol is greater than the number of line segments constituting the second symbol,
The polygon corresponding to the second symbol above has an angle of 90 degrees,
Aircraft. delete In Article 12,
The above processor, by executing the above instructions,
Based on pixel information of at least one image, the length of the second symbol within the image plane is calculated,
Based on the ratio of the actual length of the second symbol and the length within the image plane of the second symbol, relative distance information is calculated,
An aircraft that calculates relative altitude information based on a ratio of the actual width of the second symbol and the width within the image plane of the second symbol and the focal length of the image capturing device. In Article 12,
The above processor, by executing the above instructions,
An aircraft that calculates the direction information based on the angle between a first axis directed from the center of the second symbol to the center of the first symbol and a second axis directed from the center of the first symbol to the center of the aircraft. In Article 12,
The above processor, by executing the above instructions,
An aircraft that visually displays the relative position information and the direction information through the user operation device. A computer program stored in a computer-readable storage medium for executing a method of operating an aircraft, wherein the method of operating an aircraft comprises:
A step of acquiring at least one image of a marker for guiding a landing point of an aircraft within a vertiport, which is a takeoff and landing airfield for urban air traffic, using a video capturing device;
A step of recognizing the marker based on the shape of the marker within at least one image;
A step of estimating relative pose information including relative position information and direction information between the marker and the aircraft based on the result of recognizing the marker; and
Based on the above relative pose information, a step of providing guide information for guiding the aircraft to land at the landing point is included.
The step of recognizing the above marker is:
A step of recognizing a first symbol for calculating coordinates of a landing point of the aircraft and a second symbol as a reference for estimating the relative pose information by asymmetrically dividing the marker based on the shape of the marker within at least one image; and
Including a step of recognizing a lighting device located around the landing point through the video recording device,
The above marker is composed of a polygon,
The number of line segments constituting the first symbol is greater than the number of line segments constituting the second symbol,
The polygon corresponding to the second symbol above has an angle of 90 degrees,
A computer program stored on a computer-readable storage medium.