JP2024069229A - Imaging system and imaging method - Google Patents

Abstract

To provide an imaging system and an imaging method that can easily and efficiently image an object.SOLUTION: An imaging system that uses a camera mounted on a flight device to image a long structure in a vertical direction includes a flight control unit which executes: the first imaging step in which a flight device flies by circling the structure while autonomously descending from an upper side to a lower side of the structure, and images the structure with the camera at any imaging angle that is fixed in advance; and the second imaging step in which the flight device flies by autonomously circling the structure at a height position when the flight device descends to any height position of the structure in the first imaging step, displaces the camera to the lower side of the structure to change the imaging angle of the camera in the first imaging step and images the structure.SELECTED DRAWING: Figure 6

Description

The present invention relates to an imaging system and imaging method, and in particular to an imaging system and imaging method for imaging a vertically long structure using a camera mounted on a flying device.

In recent years, flying devices such as drones or multicopters that fly by rotating multiple propellers are often used to observe objects from high places or to take aerial photographs of the ground from above. Patent Document 1 discloses the generation of three-dimensional images from images of objects captured by a camera mounted on the flying device.

JP 2018-10630 A

However, when capturing an image of an object using a camera mounted on a flying device as in Patent Document 1, an operator operates the flying device to capture the image with the camera. However, if the flight of the flying device and the capture of the image of the object from the flying device can be automatically controlled, images of the object can be obtained simply and efficiently.

The present invention has been made in consideration of the above circumstances, and aims to provide an imaging system and imaging method that can capture images of an object simply and efficiently.

To achieve the above object, the imaging system of the present invention is an imaging system that images a vertically long structure with a camera mounted on a flying device, and is characterized by having a flight control unit that executes a first imaging step in which the flying device autonomously flies around the structure while descending from the top to the bottom of the structure, and images the structure with the camera at a pre-fixed arbitrary imaging angle, and a second imaging step in which, when the flying device descends to an arbitrary height position of the structure in the first imaging step, the flying device autonomously flies around the structure at that height position, displacing the camera below the structure to change the imaging angle of the camera in the first imaging step and images the structure.

The second imaging step of this imaging system is characterized in that the flying device flies around the structure at any height position of the structure, making multiple orbits, and when the flying device transitions to the next orbit, the camera is displaced below the structure to change the imaging angle of the structure by the camera.

Furthermore, the imaging system is characterized in that the arbitrary height position of the structure that transitions from the first imaging step to the second imaging step is set based on the height positions of surrounding structures that exist around the structure.

With this imaging system, the flying device autonomously executes the first imaging step and the second imaging step under the control of the flight control unit, so that images of the structure below can be captured easily and efficiently from above to below the structure, and even at any height position set based on the height positions of surrounding structures around the structure beyond which the flying device cannot descend.

Furthermore, the first imaging step of the imaging system is characterized in that the camera is positioned toward the bottom of the structure at a height position of the structure that corresponds to the flight altitude of the flying device.

This makes it possible to prevent unnecessary information from appearing in the frame of the captured image when unnecessary information is present at the upper side of the frame of the captured image captured in the first imaging step.

This imaging system is characterized by the fact that it models the structure as an approximately cylindrical shape that encloses all sides of the structure, based on an arbitrary distance from the center position of the structure as the radius and the height of the structure.

Therefore, even if the structure has a complex shape, the structure can be grasped in its simple form, making it easy to set up a circular flight using the flying device.

This imaging system is characterized in that the area of the airspace captured by the flight trajectory of the flying device flying around the structure in the first imaging step is the same as the area of the airspace captured by the flight trajectory of the flying device flying around the structure in the second imaging step.

As a result, the image quality of the image captured in the first imaging step and the image quality of the image captured in the second imaging step can be made uniform, thereby improving the quality of the captured image.

Furthermore, the flight control unit of this imaging system is characterized in that the flying device flies multiple orbits above the structure, and when the flying device transitions to the next orbit, it executes an aerial imaging step in which the camera is displaced below the structure to change the imaging angle of the structure by the camera and capture the structure.

Moreover, the flight control unit of this imaging system is characterized in that it executes an aerial imaging step before executing the first imaging step. However, the above-mentioned first imaging step, second imaging step, and aerial imaging step may be executed in this order, or in another order.

This aerial imaging step allows images to be captured that include the structure's surrounding environment, from above to below, as well as surrounding structures that exist around the structure.

This imaging system is characterized in that, when the flying device reaches a preset flight condition, the orbital flight of the flying device and the imaging of the structure being performed by the flying device are interrupted, the position on the structure where the orbital flight and imaging were interrupted is stored, and when the flight condition is released, the flying device returns to the stored position on the structure and resumes the orbital flight and imaging of the structure from the returned position on the structure.

To achieve the above object, the imaging method of the present invention is an imaging method for imaging a vertically long structure with a camera mounted on a flying device, characterized in that it comprises a first imaging step in which the flying device flies around the structure while descending from the top to the bottom of the structure, and images the structure with the camera at a pre-fixed arbitrary imaging angle, and a second imaging step in which, when the flying device descends to an arbitrary height position of the structure in the first imaging step, the flying device flies around the structure at that height position, displacing the camera below the structure to change the imaging angle of the camera in the first imaging step, and images the structure.

This invention allows images of structures to be captured simply and efficiently using an autonomous flying device.

1 is a diagram illustrating an outline of a configuration of an imaging system according to an embodiment of the present invention. FIG. 2 is a block diagram illustrating the hardware configuration of the flying device according to the present embodiment. FIG. 2 is a block diagram illustrating the software configuration of a flight controller of the flying device according to the present embodiment. FIG. 2 is a block diagram illustrating a hardware configuration of a server of the imaging system according to the present embodiment. FIG. 2 is a block diagram illustrating a software configuration of a server of the imaging system according to the present embodiment. FIG. 11 is a block diagram outlining the processing executed by the flight control unit of the server of the imaging system according to this embodiment. FIG. 13 is a diagram for explaining an outline of the aerial imaging step executed by the flight control unit of the server of the imaging system according to this embodiment. FIG. 11 is a diagram outlining the first imaging step executed by the flight control unit of the server of the imaging system according to this embodiment. This figure also explains an outline of the second imaging step executed by the flight control unit of the server of the imaging system of this embodiment. FIG. 11 is a diagram for explaining an outline of the first imaging step and the second imaging step executed by the flight control unit of the server of the imaging system according to this embodiment. FIG. 13 is a diagram for explaining an outline of a procedure for imaging a structure using the imaging system according to the present embodiment. FIG. 13 is a diagram for explaining an outline of a procedure for imaging a structure using the imaging system according to the present embodiment. FIG. 13 is a diagram for explaining an outline of a procedure for imaging a structure using the imaging system according to the present embodiment. FIG. 13 is a diagram for explaining an outline of a procedure for imaging a structure using the imaging system according to the present embodiment. FIG. 13 is a diagram for explaining an outline of a procedure for imaging a structure using the imaging system according to the present embodiment. FIG. 13 is a diagram for explaining an outline of a procedure for imaging a structure using the imaging system according to the present embodiment.

Next, an imaging system according to an embodiment of the present invention will be described with reference to Figures 1 to 16.

In this embodiment, the structure captured by the imaging system is an example of a vertically long steel tower, high-rise building, or other structure erected on the ground.

Figure 1 is a diagram illustrating an outline of the configuration of an imaging system according to this embodiment. As shown in the figure, the imaging system 10 includes a flying device 20 and a server 30 that is connected to the flying device 20 via a communication network 40 so that they can communicate with each other.

This imaging system 10 creates a three-dimensional model of the tower 1 based on multiple captured images of the tower 1 taken by the flying device 20, analyzes the captured images to detect abnormalities, and maps the detected abnormalities on the three-dimensional model.

Figure 2 is a block diagram explaining the hardware configuration of the flight device 20 according to this embodiment. As shown in the figure, the flight device 20 includes a transceiver 21, a flight controller 22 connected to the transceiver 21, a battery 23 that supplies power via the flight controller 22, a speed control unit (Electronic Speed Controller: ESC) 24 controlled by the flight controller 22, a motor 25, and four propellers 26 driven by the motor 25.

Furthermore, the flying device 10 is equipped with a camera 27 that is fixed to the aircraft and captures images of part or all of the tower 1.

The transmitter/receiver unit 21 is a communication interface configured to transmit and receive data from multiple external devices, such as a transmitter/receiver (radio transmitter), an information terminal, a display device, or other remote control devices, and in this embodiment, it mainly communicates with the server 30.

This transmission/reception unit 21 can utilize multiple communication networks, such as a local area network (LAN), a wide area network (WAN), infrared, wireless, Wi-Fi, a point-to-point (P2P) network, a telecommunications network, cloud communication, etc.

Furthermore, the transceiver unit 21 transmits and receives multiple data, such as various acquired data, processing results generated by the flight controller 22, various control data, and user commands from a terminal or a remote controller.

The flight controller 22 mainly comprises a processor 22A, memory 22B, and sensors 22C.

In this embodiment, the processor 22A is configured, for example, by a CPU (Central Processing Unit) and controls the operation of the flight controller 22, controls the transmission and reception of data between each element, and performs processing necessary for program execution.

Memory 22B includes a main memory device consisting of a volatile memory device such as a DRAM (Dynamic Random Access Memory), and an auxiliary memory device consisting of a non-volatile memory device such as a flash memory or a HDD (Hard Disc Drive).

This memory 22B is used as a working area for the processor 22A, and also stores various configuration information such as logic, code, or program instructions that can be executed by the flight controller 22.

Furthermore, the memory 22B may be configured so that data acquired from the camera 27, sensors 22C, etc. is directly transmitted and stored therein.

In this embodiment, the sensors 22C are composed of a GPS sensor 22Ca that receives radio waves from GPS satellites, a pressure sensor 22Cb that measures atmospheric pressure, a temperature sensor 22Cc that measures temperature, and an acceleration sensor 22Cd.

The camera 27 can change the imaging angle by using a gimbal depending on the imaging direction in which the tower 1 is imaged, and in this embodiment, it captures an RGB image capturing visible light. On the other hand, it may also capture a thermal image capturing infrared light, or it may capture both an RGB image and a thermal image simultaneously or sequentially.

Figure 3 is a block diagram illustrating the software configuration of the flight controller 22 of the flying device 20 according to this embodiment. As shown in the figure, the flight controller 22 includes an instruction receiving unit 22Ba, an aircraft control unit 22Bb, a position and orientation information acquisition unit 22Bc, an image capture processing unit 22Bd, an image capture information transmission unit 22Be, a position and orientation information storage unit 22Bf, and an image capture information storage unit 22Bg.

The instruction receiving unit 22Ba, the aircraft control unit 22Bb, the position and orientation information acquisition unit 22Bc, the image capture processing unit 22Bd, and the image capture information transmission unit 22Be are realized by the processor 22A executing a program stored in the memory 22B.

On the other hand, the position and orientation information storage unit 22Bf and the imaging information storage unit 22Bg are realized as storage areas provided by the memory 22B.

The instruction receiving unit 22Ba accepts various commands (hereinafter referred to as "flight operation commands") that instruct the operation of the flying device 20. In this embodiment, the flight operation commands are received from the server 30, but it may also be configured to receive flight operation commands from a transceiver such as a radio control unit.

In this embodiment, the aircraft control unit 22Bb controls the operation of the flight device 20 in response to flight operation commands received by the instruction receiving unit 22Ba, and controls the motor 25 via the ESC 24 to adjust, for example, the spatial arrangement, speed, and/or acceleration of the flight device 20, which has six degrees of freedom (translational motions x, y, and z, and rotational motions θx, θy, and θz).

The motor 25 is driven by the control of the aircraft control unit 22Bb, causing the propeller 26 to rotate, generating lift for the flight device 20 to fly.

On the other hand, the aircraft control unit 22Bb can also execute various controls so that the flying device 20 flies autonomously without relying on flight operation commands.

The position and attitude information acquisition unit 22Bc acquires information (hereinafter referred to as "position and attitude information") indicating the current position and attitude of the flight device 20. In this embodiment, the position and attitude information includes the position of the flight device 20 on a map expressed by latitude and longitude, the flight altitude of the flight device 20, and the inclination of each of the x, y, and z axes of the flight device 20.

This position and attitude information acquisition unit 22Bc calculates the position of the flying device 20 on the map from the radio waves received by the GPS sensor 22Ca from GPS satellites.

The position and attitude information acquisition unit 22Bc calculates the flight altitude of the flight device 20 based on the difference between the atmospheric pressure measured by the atmospheric pressure sensor 22Cb before flight (hereinafter referred to as the "reference atmospheric pressure") and the atmospheric pressure measured by the atmospheric pressure sensor 22Cb during flight (hereinafter referred to as the "current atmospheric pressure"), and the air temperature measured by the temperature sensor 22Cc during flight.

Furthermore, the position and attitude information acquisition unit 22Bc obtains the attitude of the flying device 20 based on the output from the acceleration sensor 22Cd, and determines the optical axis (viewpoint axis) of the camera 27 from the attitude of the flying device 20.

The position of the flying device 20 on the map, the flight altitude of the flying device 20, and the attitude of the flying device 20 (the tilt of the optical axis of the camera 27) are stored in the position and attitude information storage unit 22Bf.

The imaging processing unit 22Bd controls the camera 27 to capture an image of part or all of the tower 1, and acquires the captured image captured by the camera 27.

In this embodiment, the imaging processing unit 22Bd captures images at a preset timing, and can capture images at any designated time interval, such as every 5 seconds or 30 seconds. On the other hand, it may be configured to capture images based on instructions from the server 30.

The imaging processing unit 22Bd generates imaging information by associating the acquired captured image with the imaging date and time, the latitude and longitude of the flying device 20 on the map at the time of imaging (imaging position), the flying altitude of the flying device 20 at the time of imaging (imaging altitude), and the attitude of the flying device 20 (tilt of the optical axis of the camera 27), and this imaging information is stored in the imaging information storage unit 22Bg.

The imaging information transmission unit 22Be transmits the images captured by the camera 27 to the server 30. In this embodiment, imaging information that associates the imaging date and time, imaging position, imaging altitude, and inclination with the captured image is transmitted to the server 30.

Figure 4 is a block diagram illustrating the hardware configuration of the server 30 according to this embodiment. As shown in the figure, the server 30 includes a CPU 31, a memory 32, a storage device 33, a communication device 34, an input device 35, and an output device 36.

The CPU 31 controls the operation of the server 30, controls the sending and receiving of data between each element that constitutes the server 30, and performs processing necessary for executing programs.

Memory 32 includes a main memory device consisting of a volatile memory device such as a DRAM, and an auxiliary memory device consisting of a non-volatile memory device such as a flash memory or HDD.

The storage device 33 is a storage medium that stores various data and programs, and is implemented, for example, by a HDD, SSD (Solid State Drive), or flash memory.

The communication device 34 communicates with other devices via the communication network 40, and in this embodiment, communicates with the flight device 20. The communication device 34 is configured to include, for example, an adapter for connecting to Ethernet (registered trademark), a modem for connecting to the public telephone line network, a wireless communication device for wireless communication, and a USB connector or RS232C connector for serial communication.

The input device 35 is an interface capable of inputting data, such as a keyboard, mouse, touch panel, button, microphone, etc., and the output device 36 is a device capable of outputting data, such as a display, printer, speaker, etc.

Figure 5 is a block diagram illustrating the software configuration of the server 30 according to this embodiment. As shown in the figure, the server 30 includes a flight control unit 33A, an imaging information receiving unit 33B, a three-dimensional model creation unit 33C, an abnormality detection unit 33D, a three-dimensional model display unit 33E, an image display unit 33F, an imaging information storage unit 33G, a three-dimensional model storage unit 33H, and an abnormality information storage unit 33I.

The flight control unit 33A, the imaging information receiving unit 33B, the three-dimensional model creation unit 33C, the abnormality detection unit 33D, the three-dimensional model display unit 33E, and the captured image display unit 33F are realized by the CPU 31 of the server 30 reading out a program stored in the storage device 33 into the memory 32 and executing it.

On the other hand, the imaging information storage unit 33G, the three-dimensional model storage unit 33H, and the anomaly information storage unit 33I are realized as part of the storage area provided by the storage device 33 of the server 30.

The flight control unit 33A is a module that controls the flight of the flight device 20, and in this embodiment, causes the flight device 20 to fly autonomously based on a preset program (data) related to the autonomous flight of the flight device 20. An outline of the processing in this flight control unit 33A will be described later.

The imaging information receiving unit 33B receives imaging information transmitted from the flying device 20 and stores the received imaging information in the imaging information storage unit 33G.

The three-dimensional model creation unit 33C creates a three-dimensional model that represents a three-dimensional structure from multiple captured images. In this embodiment, the world coordinate system of the three-dimensional model is expressed by latitude, longitude, and altitude, and the position and viewpoint direction of the camera 27 in the world coordinate system can be indicated by the imaging position, imaging altitude, and optical axis tilt contained in the imaging information.

This three-dimensional model creation unit 33C extracts feature points from the image data contained in the imaging information, matches the feature points extracted from the multiple image data based on the imaging position, imaging altitude, and inclination contained in the imaging information, and obtains a three-dimensional point group in the world coordinate system, also known as a point cloud.

The three-dimensional model (three-dimensional point cloud in this embodiment) created in this manner is stored in the three-dimensional model storage unit 33H.

The abnormality detection unit 33D analyzes the images captured by the flight device 20 to detect abnormalities in the tower 1.

Specifically, abnormalities in the tower 1 are detected using a trained model generated by machine learning such as a neural network to determine abnormalities based on images captured by the flying device 20, or by comparing images of the tower 1 when it is normal with the captured images.

This anomaly detection unit 33D identifies the position in the world coordinate system of the detected anomaly in the captured image.

For example, for each point included in the three-dimensional point cloud, the position on the image when the image is captured in the direction indicated by the tilt included in the imaging information from a camera installed at the imaging position and imaging height included in the imaging information is identified, and whether or not the identified position on the image constitutes an abnormality is determined based on whether or not this position is included in an area detected as an abnormality, and the coordinates of the point that constitutes the abnormality are identified as the position of the abnormality.

Information relating to the abnormality detected in this manner (hereinafter referred to as "abnormality information") is stored in the abnormality information storage unit 33I.

The three-dimensional model display unit 33E displays an image (hereinafter referred to as a "three-dimensional projected image") of the three-dimensional model created by the three-dimensional model creation unit 33C projected onto a plane. This three-dimensional projected image may be displayed using a point cloud (point cloud data), or the captured image may be mapped onto the three-dimensional model.

The captured image display unit 33F displays the captured image on a display connected to the server 30, for example.

Figure 6 is a block diagram outlining the processing executed by the flight control unit 33A according to this embodiment. As shown in the figure, the flight control unit 33A executes an aerial imaging step S1, a first imaging step S2, a second imaging step S3, and a flight condition processing step S4 based on a preset flight program.

Figure 7 is a diagram outlining the aerial imaging step S1. As shown in the figure, in the aerial imaging step S1, the flying device 20 flies multiple times above the tower 1 while maintaining a constant height position above the ground surface E based on a preset flight program, and captures an image of the tower 1 with the camera 27.

In this embodiment, the flying device 20 is set to perform, for example, a first orbital flight as indicated by flight trajectory f1 through a third orbital flight as indicated by flight trajectory f3, and is set to capture images of the tower 1 while changing the imaging angle of the camera 27 with a gimbal during multiple orbital flights.

In this aerial imaging step S1, the flying device 20 is configured to perform a first orbital flight indicated by flight trajectory f1, transition after the first orbital flight to a second orbital flight indicated by flight trajectory f2 in which the orbital flight radius is wider than that of the first orbital flight, and transition after the second orbital flight to a third orbital flight indicated by flight trajectory f3 in which the orbital flight radius is wider than that of the second orbital flight (orbital flight setting S1a).

In conjunction with the setting of the orbital flight, the camera 27 of the flight device 20 is positioned by a gimbal to a position where it is possible to image the upper side of the tower 1 with the camera 27 during the first orbital flight, the camera 27 of the flight device 20 is displaced by a gimbal below the tower 1 to change the imaging angle of the tower 1 by the camera 27 and positioned to image the middle part of the tower 1 during the second orbital flight, and the camera 27 of the flight device 20 is displaced by a gimbal further below the tower 1 to change the imaging angle of the tower 1 by the camera 27 and positioned to image the underside of the tower 1 during the third orbital flight (imaging angle setting S1b).

When setting this imaging angle, if the imaging angle of the camera 27 of the tower 1 is changed, the imaging angle is set so that continuous imaging can be performed so that at least a portion of the imaging area overlaps in the vertical direction of the tower 1.

On the other hand, when the flying device 20 orbits the tower 1 and captures images of the tower 1 with the camera 27, the imaging interval is set to an interval that allows continuous imaging so that at least a portion of the imaging area overlaps in the orbital direction (imaging interval setting S1c).

In this aerial imaging step S1, an image is acquired of the surrounding environment of the tower 1, from the top to the bottom, as well as surrounding structures around the tower 1, such as trees 2 and houses 3.

Figure 8 is a diagram outlining the first imaging step S2. This first imaging step S2 is executed following the aerial imaging step S1, and as shown in the figure, based on a preset flight program, the flying device 20 flies around the tower 1 while descending from the top to the bottom of the tower 1 to the descent limit position L, and at this time images of the tower 1 are captured by the camera 27.

In this embodiment, the flying device 20 is set to perform a circular flight as shown by flight trajectory f4 or a circular flight as shown by flight trajectory f10 while descending, for example, from the top to the bottom of the tower 1 to the lower descent limit position L, and captures an image of the tower 1 with a camera 27 fixed by a gimbal at an arbitrary imaging angle during the circular flight.

When the first imaging step S2 is performed, the lower descent limit position L is set by adding an arbitrary distance d1 in the height direction to the height position of the surrounding structure that is the highest of the surrounding structures present around the tower 2, such as trees 2 and houses 3 (setting of the lower descent limit position S2a).

In this embodiment, for ease of explanation, it is assumed that the tree 2 and the house 3 are at approximately the same height.

In the first imaging step S2, the vertical distance on the tower 1 between the circular flight shown by flight trajectory f4 and the circular flight shown by flight trajectory f5 by the flying device 20, and the vertical distance on the tower 1 when transitioning to the next circular flight between the circular flight shown by flight trajectory f5 and the circular flight shown by flight trajectory f10, are set to distances that allow the flying device 20 to fly around the tower 1 while descending from the top to the bottom of the tower 1, taking continuous images so that at least a portion of the imaging area overlaps in the vertical direction (circular flight setting S2b).

In the first imaging step S2, the camera 27 is positioned facing downward from the tower 1 at a height position on the tower 1 that corresponds to the flight altitude when the flying device 20 is flying in a circular flight as shown by flight trajectory f4, for example, and is set to be fixed in this state by a gimbal (setting of the imaging angle S2c).

The imaging angle is the angle between the center of the captured image and the top edge of the captured image, and in this embodiment, it is set to an optimal angle calculated depending on the situation. As a result, in cases where unnecessary information is present at the top of the frame of the captured image captured in the first imaging step S2 when creating a three-dimensional model, such as clouds floating above the steel tower 1, it is possible to prevent such unnecessary information from appearing in the frame of the captured image.

On the other hand, when the flying device 20 captures images of the tower 1 with the camera 27 while flying around the tower 1 in orbit, the imaging interval is set to an interval that allows continuous imaging so that at least a portion of the imaging area overlaps in the orbital direction (imaging interval setting S2d).

Figure 9 is a diagram outlining the second imaging step S3. This second imaging step S3 is executed when the flying device 20 descends to the lowermost descent position L in the first imaging step S1.

As shown, in the second imaging step S3, the flying device 20 flies around the tower 1 at the lowest descent position L multiple times based on a preset flight program, and captures an image of the tower 1 with the camera 27.

In this second imaging step S3, the flying device 20 is set to perform a circular flight indicated by flight trajectory f11 or a circular flight indicated by flight trajectory f14, for example, at the lower descent limit position L. (Circular flight setting S3a).

On the other hand, when transitioning from the circular flight indicated by flight trajectory f11 to the circular flight indicated by flight trajectory f12, and when transitioning to the next circular flight between the circular flight indicated by flight trajectory f12 and the circular flight indicated by flight trajectory f14, the imaging angle of the camera 27 is set to be changed by the gimbal to capture an image of the tower 1 (imaging angle setting S3b).

Specifically, when the flying device 20 performs the circular flight indicated by the flight trajectory f11, the camera 27 of the flying device 20 is displaced downward by a gimbal from the position determined in the first imaging step S2, changing the imaging angle of the camera 27 of the tower 1, and is set to be positioned in the imaging direction a1.

Next, when the flying device 20 performs the circular flight shown by the flight trajectory f12, the camera 27 of the flying device 20 is displaced further downward from the imaging direction a1 by the gimbal, changing the imaging angle of the tower 1 by the camera 27, and is set to be positioned at a position corresponding to the imaging direction a2.

Next, when the flying device 20 performs the circular flight shown by flight trajectory f13, the camera 27 of the flying device 20 is displaced further downward from the imaging direction a2 by the gimbal, changing the imaging angle of the tower 1 by the camera 27, and is set to be positioned at a position corresponding to the imaging direction a3.

Furthermore, when the flying device 20 performs the circular flight shown by the flight trajectory f12, the camera 27 of the flying device 20 is displaced further downward from the imaging direction a3 by the gimbal, changing the imaging angle of the tower 1 by the camera 27, and is set to be positioned at a position corresponding to the imaging direction a4.

When this imaging angle is set so that the imaging angle of the camera 27 of the tower 1 is changed to imaging directions a1 to a4, the imaging angle is set so that continuous imaging is possible so that at least a portion of the imaging area overlaps in the vertical direction of the tower 1.

On the other hand, the imaging interval when the flying device 20 captures images of the tower 1 with the camera 27 while flying around the tower 1 is set to an interval that allows continuous imaging so that at least a portion of the imaging area overlaps in the orbital direction (imaging interval setting S3c).

Figure 10 is a diagram outlining the first imaging step S2 and the second imaging step S3. As shown in the figure, in the first imaging step S2, the area s1 of the airspace grasped by the flight trajectories f4 to f10 along which the flying device 20 flies in an orbit around the pylon 1, and the area s2 of the airspace grasped by the flight trajectories f11 to f14 along which the flying device 20 flies in an orbit around the pylon 1, are the same in this embodiment.

As a result, the image quality of the image captured in the first imaging step S2 and the image quality of the image captured in the second imaging step S3 can be made uniform, thereby improving the quality of the captured image.

In flight condition processing step S4, when the flight device 20 reaches a preset flight condition, any step (aerial imaging step S1, first imaging step S2, second imaging step S3) being executed by the flight device 20 at the time of reaching the preset flight condition is interrupted, and the position on the tower 1 where any step was interrupted is stored in the memory (not shown) of the server 30 of the flight device 20.

When executing this flight condition processing step S4, in this embodiment, flight conditions are set such as "when the remaining charge of the battery 23 of the flight device 20 reaches 20%" or "when the flight time of the flight device 20 exceeds 20 minutes" (setting of flight conditions S4a).

Next, the procedure for imaging the tower 1 using the imaging system 10 according to this embodiment will be described with reference to Figures 11 to 16.

As shown in FIG. 11, first, it is confirmed whether the ground surface E on which the steel tower 1 to be imaged is erected is a flyable site E1 above which the flight device 20 is permitted to fly, and if it is, it is confirmed whether there are any surrounding structures around the steel tower 1.

As shown in the figure, when there are surrounding structures such as trees 2 and houses 3 around the steel tower 1, the flying device 20 is operated to adjust the flight altitude of the flying device 20 so that the flying device 20 is positioned at the height position of the surrounding structures, and the height position of the surrounding structures is obtained based on the flight altitude of the flying device 20 displayed on the operation screen, etc.

Similarly, the flight altitude of the flight device 20 is adjusted so that the flight device 20 is positioned at the height position of the steel tower 1, and the height position of the steel tower 1 is obtained based on the flight altitude of the flight device 20 displayed on the operation screen, etc. At this time, if the height of the steel tower 1 is known in advance, that height can be used, but if the height is unknown, it is sufficient to use the flight device 20 as in this embodiment.

In this embodiment, the acquired height positions of the surrounding structures and the height position of the tower 1 are input to the server 30.

Next, as shown in FIG. 11, the distance measured from the center position O of the tower 1 to the corner of the tower 1 is obtained, and this obtained distance is input to the server 30 as the radius r.

When the height position of the surrounding structure is input to the server 30, as shown in FIG. 12, in the flight space A in which the flight device 20 can fly, which is set above the flight site E1, an arbitrary distance d1 is added in the height direction to the height position of the surrounding structure, and a lower descent limit position L is set in the first imaging step S2 (setting of the lower descent limit position S2a).

On the other hand, when the height position and radius r of the tower 1 are input to the server 30, the tower 1 is modeled into an approximately cylindrical shape that surrounds all sides of the tower 1, and a tower model M is generated.

This allows the tower 1, which has a complex shape with multiple components such as cross arms, to be grasped in a simple form, making it easy to set up circular flight using the flying device 20.

Next, as shown in FIG. 13, the flying device 20 is placed at the center position O of the tower 1, the position where the flying device 20 is placed is acquired as GPS coordinates by the GPS sensor 22Ca, and the acquired GPS coordinates are input to the server 30.

The GPS coordinates input to the server 30 are understood as coordinates indicating the center position O of the tower 1, and the flight of the flying device 20 is controlled when it flies autonomously above and around the tower 1.

If it is not possible to place the flying device 20 at the center position O of the tower 1, it is also possible to place the flying device 20 at a diagonal position of the tower 1 in the horizontal direction, obtain the GPS coordinates of that position, and determine the center position of the diagonal line connecting the GPS coordinates as the center position O of the tower 1.

Then, the aerial imaging step S1 is set. In the aerial imaging step S1, the circular flight setting S1a, the imaging angle setting S1b, and the imaging interval setting S1c are set.

In this embodiment, in the circular flight setting S1a, the radius of the flight trajectory f3 during the third circular flight is set to be the height of the steel tower 1, and the height position (flight altitude) from the ground surface E at which the flight device 20 flies above the steel tower 1 is set to be the higher of 1.5 times the height of the steel tower 1 and the height of the steel tower plus a desired height that ensures a safe flight altitude.

In this embodiment, the circular flight setting S1a is set to perform three circular flights, but the number of circular flights can be set appropriately. Furthermore, it is also possible to set the flight path of the multiple circular flights to all have the same flight path radius without widening the radius of the flight path of the circular flight.

Next, the first imaging step S2 is set. In the first imaging step S2, the circular flight setting S2b, the imaging angle setting S2c, and the imaging interval setting S2d are performed.

The setting of the lower limit position S2a is performed by inputting the height position of the surrounding structure to the server 30, which eliminates the need to set it in the first imaging step S2.

In this embodiment, in the circular flight setting S2b, as shown in FIG. 14, the flying device 20 is set to perform the circular flight shown by flight trajectory f4 or the circular flight shown by flight trajectory f10 with a flight radius fr obtained by adding a desired distance d2 that ensures a safe flight distance between the tower 1 and the radius Mr of the tower model M.

In this embodiment, in the circular flight setting S2b, the flying device 20 is set to perform seven circular flights, but the number of circular flights can be set as appropriate.

Next, the second imaging step S3 is set. In the second imaging step S3, the circular flight setting S3a, the imaging angle setting S3b, and the imaging interval setting S3c are performed.

In this embodiment, in the circular flight setting S3a, the flying device 20 is set to perform the circular flight shown by the flight trajectory f11 or the circular flight shown by the flight trajectory f14 with the same flight radius fr as the flight radius fr set in the circular flight setting S2b of the first imaging step S2.

In this embodiment, the flight device 20 is set to perform four circular flights in the circular flight setting S3a, but the number of circular flights can be set as appropriate.

Next, flight condition processing step S4 is set. In flight condition processing step S4, flight condition setting S4a is set, and in this embodiment, "when the remaining charge of the battery 23 of the flight device 20 reaches 20%" is set as the flight condition.

By making the above settings, the flight program is set up, and the flight control unit 33A executes the aerial imaging step S1, the first imaging step S2, the second imaging step S3, and in some cases the flight condition processing step S4 based on the set flight program.

As shown in FIG. 15, by executing the aerial imaging step S1, the flying device 20 performs an orbital flight as indicated by flight trajectory f1, images the upper side of the tower 1 with the camera 27 positioned by the gimbal in a position that allows the upper side of the tower 1 to be imaged, and as the flight transitions to an orbital flight as indicated by flight trajectory f2 and flight trajectory f3, the imaging angle of the camera 27 of the tower 1 is changed to image the middle part and the underside of the tower 1.

As a result, an image is captured that includes the tower 1's surrounding environment, such as trees 2 and houses 3, which are structures that exist around the tower 1, making it possible to create a three-dimensional model that closely resembles the tower 1's actual environment.

After the aerial imaging step S1 is completed, the process moves to the first imaging step S2. By executing the first imaging step S2, the flying device 20 performs a circular flight from the top of the tower 1 to the bottom thereof as shown by flight trajectory f4 or flight trajectory f10 while descending to the lowermost descent position L, and during the circular flight, the tower 1 is imaged at the imaging angle of the camera 27 set in the imaging angle setting S2c.

In this embodiment, when the flight device 20 is flying in a circle as shown by flight trajectory f7, the flight conditions set in the flight condition setting S4a are reached ("when the remaining charge of the battery 23 of the flight device 20 reaches 20%), and therefore the first imaging step S2 being executed by the flight device 20 is interrupted in the circle as shown by flight trajectory f7 by the flight condition processing step S4, and the imaging of the tower 1 is also interrupted.

At this time, the GPS sensor 22Ca acquires the GPS coordinates of the position on the tower 1 where the circular flight and imaging of the tower 1, as shown by flight trajectory f7, were interrupted, and the acquired GPS coordinates are stored in the memory (not shown) of the server 30.

Then, the flight device 20 returns to a preset return position for the flight device 20, and the battery 23 is replaced with a fully charged battery, and the flight conditions are released.

When the flight conditions are released, the flight device 20 returns to the position on the tower 1 where the circular flight indicated by flight trajectory f7 and the imaging of the tower 1 were interrupted, based on the GPS coordinates stored in the memory of the server 30, and as shown in FIG. 16, performs a circular flight indicated by flight trajectory f8 or flight trajectory f10 from the position on the tower 1 where it has returned, and resumes imaging of the tower 1 from the position on the tower 1 where it has returned, using the imaging angle of the camera 27 set in the imaging angle setting S2c.

When the flight device 20 descends to the lowest descent position L and the first imaging step S2 is completed, the process moves to the second imaging step S3. In the second imaging step S3, the flight device 20 is at the lowest descent position L and performs an orbital flight as shown by flight trajectory f11 to flight trajectory f14. As the flight device 20 transitions from the orbital flight as shown by flight trajectory f11 to flight trajectory f14, the imaging angle of the camera 27 for the tower 1 is changed and the tower 1 is imaged from positions in imaging directions a1 to a4.

This second imaging step S3 allows the tower 1 to be imaged below the lower descent limit position L, below which the flying device 20 cannot descend any further, to obtain an image.

In this embodiment, the circular flight of the flying device 20 shown by flight trajectory f11 or the circular flight shown by flight trajectory f14 in the second imaging step S3 and the circular flight shown by flight trajectory f10 of the flying device 20 in the first imaging step S2 are performed at the lower descent limit position L, which is the same height position.

In this way, the flight device 20 autonomously executes the first imaging step S2 and the second imaging step S3 under the control of the flight control unit 33A of the server 30 of the imaging system 10, so that images of the tower 1 can be captured simply and efficiently from the top to the bottom, and further below the lower descent limit position L, which is set based on the height positions of the trees 2 and houses 3 surrounding the tower 1, beyond which the flying device 20 cannot descend.

In particular, the flight control unit 33A successively executes the aerial imaging step S1, the first imaging step S2, and the second imaging step S3, thereby enabling precise imaging of the tower 1 in all directions, including surrounding structures such as trees 2 and houses 3 that exist around the tower 1.

The present invention is not limited to the above embodiment, and various modifications are possible without departing from the spirit of the invention. In the above embodiment, the structure is a steel tower 1, but it is possible to capture images of any structure that is long in the vertical direction, such as a high-rise apartment building, a chimney, an antenna tower, a lighthouse, a windmill, a tree, or even a Kannon statue.

1. Steel tower (structure)
2. Trees (surrounding structures)
3. Houses (surrounding structures)
10 Imaging system 20 Flight device 22 Flight controller 22B Memory 22Ca GPS sensor 30 Server 33A Flight control unit f1 to f14 Flight trajectory L Lower descent limit position M Steel tower model O Center position r Radius S1 Sky imaging step S2 First imaging step S3 Second imaging step S4 Flight condition processing step

Claims (10)

In an imaging system for imaging a long structure in the vertical direction using a camera mounted on a flying device,
a first imaging step in which the flying device autonomously flies around the structure while descending from above to below the structure and images the structure with the camera at a pre-fixed arbitrary imaging angle;
a second imaging step in which, when the flying device descends to an arbitrary height position of the structure in the first imaging step, the flying device autonomously flies around the structure at the height position and displaces the camera below the structure to change the imaging angle of the camera in the first imaging step, thereby imaging the structure;
An imaging system comprising a flight control unit that executes the above. The second imaging step includes:
The imaging system described in claim 1, characterized in that the flying device flies multiple times around the structure at any of the height positions of the structure, and when the flying device transitions to the next orbit, the camera is displaced below the structure to change the imaging angle of the structure by the camera. The imaging system according to claim 1 or 2, characterized in that the arbitrary height position of the structure at which the first imaging step is transferred to the second imaging step is set based on the height positions of surrounding structures existing around the structure. The first imaging step includes:
4. The imaging system according to claim 1, wherein the camera is positioned facing downwardly of the structure at a height position of the structure corresponding to the flight altitude of the flying device. The imaging system according to any one of claims 1 to 4, characterized in that the structure is modeled as a roughly cylindrical shape that surrounds all sides of the structure, based on an arbitrary distance from the center position of the structure as a radius and the radius and the height of the structure. The imaging system according to any one of claims 1 to 5, characterized in that the area of the airspace captured by the flight trajectory of the flying device flying around the structure in the first imaging step is the same as the area of the airspace captured by the flight trajectory of the flying device flying around the structure in the second imaging step. The flight control unit:
The imaging system according to any one of claims 1 to 6, characterized in that the flying device flies multiple orbits above the structure, and when the flying device transitions to the next orbit, an aerial imaging step is executed in which the camera is displaced below the structure to change the imaging angle of the structure by the camera and image the structure. The flight control unit:
The imaging system according to claim 7 , wherein the aerial imaging step is performed before the first imaging step is performed. The imaging system according to any one of claims 1 to 8, characterized in that when the flying device reaches a preset flight condition, the orbital flight of the flying device and the imaging of the structure being performed by the flying device are interrupted and the position on the structure where the orbital flight and imaging were interrupted are stored, and when the flight condition is released, the flying device returns to the stored position on the structure and resumes the orbital flight and imaging of the structure from the returned position on the structure. An imaging method for imaging a vertically long structure using a camera mounted on a flying device, comprising:
a first imaging step in which the flying device autonomously flies around the structure while descending from above to below the structure and images the structure with the camera at a pre-fixed arbitrary imaging angle;
a second imaging step in which, when the flying device descends to an arbitrary height position of the structure in the first imaging step, the flying device autonomously flies around the structure at the height position and displaces the camera below the structure to change the imaging angle of the camera in the first imaging step, thereby imaging the structure;
An imaging method comprising: