[go: up one dir, main page]

CN107632615B - Autonomous flight four-rotor tunnel passing method based on visual inspection - Google Patents

Autonomous flight four-rotor tunnel passing method based on visual inspection Download PDF

Info

Publication number
CN107632615B
CN107632615B CN201710750773.5A CN201710750773A CN107632615B CN 107632615 B CN107632615 B CN 107632615B CN 201710750773 A CN201710750773 A CN 201710750773A CN 107632615 B CN107632615 B CN 107632615B
Authority
CN
China
Prior art keywords
tunnel
flight
rotors
autonomous
flying
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
CN201710750773.5A
Other languages
Chinese (zh)
Other versions
CN107632615A (en
Inventor
史莹晶
李�瑞
刘奇胜
蒋宏
张华林
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
University of Electronic Science and Technology of China
Original Assignee
University of Electronic Science and Technology of China
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by University of Electronic Science and Technology of China filed Critical University of Electronic Science and Technology of China
Priority to CN201710750773.5A priority Critical patent/CN107632615B/en
Publication of CN107632615A publication Critical patent/CN107632615A/en
Application granted granted Critical
Publication of CN107632615B publication Critical patent/CN107632615B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Landscapes

  • Control Of Position, Course, Altitude, Or Attitude Of Moving Bodies (AREA)

Abstract

本发明公开了一种基于视觉巡视的自主飞行四旋翼穿行隧道方法,利用自主飞行四旋翼自主寻找入口并进入隧道,在隧道内部,再通过底部摄像头对自主飞行四旋翼实现自主导航飞行,在飞行的同时利用摄像头对隧道内部电缆及周边情况进行实时检测,最后自主飞出隧道,整个过程无须人工干预,具有良好的自主飞行及导航能力,还具有检测范围广、速度快、精确度高等特点。

Figure 201710750773

The invention discloses a method for traveling through a tunnel with an autonomous flying quadrotor based on visual inspection. The autonomous flying quadrotor is used to autonomously find an entrance and enter the tunnel. Inside the tunnel, a bottom camera is used to realize autonomous navigation and flight for the autonomous flying quadrotor. At the same time, the camera is used to detect the cables inside the tunnel and the surrounding conditions in real time, and finally fly out of the tunnel autonomously. The whole process does not require manual intervention. It has good autonomous flight and navigation capabilities, and also has a wide detection range, high speed and high accuracy.

Figure 201710750773

Description

一种基于视觉巡视的自主飞行四旋翼穿行隧道方法A method of autonomous flying quadrotor traveling through tunnel based on visual inspection

技术领域technical field

本发明属于飞行控制技术领域,更为具体地讲,涉及一种基于视觉巡视的自主飞行四旋翼穿行隧道方法。The invention belongs to the technical field of flight control, and more particularly relates to a method for traveling through a tunnel based on an autonomous flying quadrotor based on visual inspection.

背景技术Background technique

近年来,在大型城市的发展中,以地下电缆方式取代传统的架空线路已经成为主流。统计表明,在许多现代化都市中,地下输电线路的比例已经超过70%。电缆隧道作为容纳电缆数量较多、有供安装和巡视的通道的全封闭地下构筑物,是地下电缆的最佳承载方式,随着地下电缆的普及,如何定期对封闭隧道进行检测并预防突发情况成为了急需解决的问题。In recent years, in the development of large cities, it has become mainstream to replace traditional overhead lines with underground cables. Statistics show that in many modern cities, the proportion of underground transmission lines has exceeded 70%. As a fully enclosed underground structure that accommodates a large number of cables and has passages for installation and inspection, cable tunnels are the best way to carry underground cables. With the popularization of underground cables, how to regularly detect closed tunnels and prevent emergencies has become an urgent problem.

由此人们提出了利用有轨机器人或者人工进行定期检测。其中,有轨机器人主要是依托轨道进行运动的,轨道铺设在隧道顶部或底部,机器人通过自身电机的旋转或者轨道的传动,沿着轨道在隧道内穿行,并且通过机器人搭载的相关传感器对隧道进行实时检测,有轨机器人通常需提前部署在目标隧道中,即一条隧道就需配备一个有轨机器人及配套的轨道设施,且仍需要定期派人下到隧道中检查维护有轨机器人自身,因此使用及维护成本较高,而无需提前部署,能够自主出入隧道,对隧道进行全自主,全面检测的无人装置还没有出现。Therefore, it has been proposed to use orbital robots or humans to perform periodic inspections. Among them, the tracked robot mainly relies on the track to move. The track is laid on the top or bottom of the tunnel. The robot travels through the tunnel along the track through the rotation of its own motor or the transmission of the track, and carries out the tunnel through the relevant sensors carried by the robot. For real-time detection, the rail robot usually needs to be deployed in the target tunnel in advance, that is, a tunnel needs to be equipped with a rail robot and supporting rail facilities, and it is still necessary to regularly send people down to the tunnel to check and maintain the rail robot itself, so use And the maintenance cost is high, without the need for advance deployment, the unmanned device that can enter and exit the tunnel autonomously, conduct full autonomous and comprehensive inspection of the tunnel has not yet appeared.

发明内容SUMMARY OF THE INVENTION

本发明的目的在于克服现有技术的不足,提供一种基于视觉巡视的自主飞行四旋翼穿行隧道方法,利用自主飞行四旋翼自主寻找入口并进入隧道,在隧道内部实现自主导航飞行,并实时检测隧道内部电缆及周边情况,最后自主飞出隧道。The purpose of the present invention is to overcome the deficiencies of the prior art, and to provide a method for traveling through a tunnel based on an autonomous flight quadrotor based on visual inspection. The autonomous flight quadrotor is used to autonomously search for an entrance and enter the tunnel, realize autonomous navigation and flight inside the tunnel, and detect in real time. The cables inside the tunnel and the surrounding conditions finally flew out of the tunnel autonomously.

为实现上述发明目的,本发明基于视觉巡视的自主飞行四旋翼穿行隧道方法,其特征在于,包括以下步骤:In order to achieve the above-mentioned purpose of the invention, the present invention based on the visual inspection of the autonomous flying quadrotor traveling through the tunnel method, is characterized in that, comprises the following steps:

(1)、沿着隧道中线添加黄色标识线,在自主飞行四旋翼的底部安装一个USB摄像头,并加装LED补光灯确保摄像头在隧道内部正常工作,在机头前后方及机身右侧处加装微型激光测距传感器;(1) Add a yellow marking line along the tunnel center line, install a USB camera at the bottom of the autonomous flying quadrotor, and install LED fill light to ensure that the camera works normally inside the tunnel, on the front and rear of the nose and the right side of the fuselage A miniature laser ranging sensor is installed;

(2)、控制自主飞行四旋翼驶入隧道井口(2) Control the autonomous flying quadrotor to drive into the tunnel wellhead

(2.1)、自主飞行四旋翼上电初始化;(2.1) Power-on initialization of the autonomous flight quadrotor;

(2.2)、手动起飞自主飞行四旋翼,使自主飞行四旋翼离地面距离h1,再手动遥控自主飞行四旋翼驶入隧道井口附近,此时,通过遥控器切换自主飞行四旋翼为全自主飞行模式;(2.2) Manually take off the autonomous flight quadrotor, make the autonomous flight quadrotor a distance h 1 from the ground, and then manually control the autonomous flight quadrotor to drive into the tunnel near the wellhead. At this time, switch the autonomous flight quadrotor to fully autonomous flight through the remote control model;

(2.3)、飞行控制模块计算自主飞行四旋翼的实时姿态角;(2.3), the flight control module calculates the real-time attitude angle of the autonomous flying quadrotor;

(2.4)、自主飞行四旋翼利用机头及机身右侧的激光测距传感器定位当前自主飞行四旋翼的实时位置信息,并与加速度计融合估计飞行速度,结合实时姿态角、位置信息和飞行速度,飞行控制模块控制自主飞行四旋翼飞向隧道井口中心位置;(2.4) The autonomous flying quadrotor uses the laser ranging sensor on the nose and the right side of the fuselage to locate the real-time position information of the current autonomous flying quadrotor, and fuses it with the accelerometer to estimate the flight speed, combining the real-time attitude angle, position information and flight speed speed, the flight control module controls the autonomous flying quadrotor to fly to the center of the tunnel wellhead;

(3)、控制自主飞行四旋翼从隧道井口中心位置处驶入隧道入口(3) Control the autonomous flying quadrotor to drive into the tunnel entrance from the center of the tunnel wellhead

飞行控制模块控制自主飞行四旋翼从隧道井口中心位置处垂直下降,当下降到隧道顶部平面时,改用机尾及机身右侧的激光测距传感器定位自主飞行四旋翼的实时位置信息,根据当前位置信息和实时姿态角继续垂直下降,当下降到设定的巡航高度时,达到隧道入口;The flight control module controls the autonomous flight quadrotor to descend vertically from the center of the tunnel wellhead. When it descends to the top plane of the tunnel, the laser ranging sensors on the tail and the right side of the fuselage are used to locate the real-time position information of the autonomous flight quadrotor. The current position information and real-time attitude angle continue to descend vertically, and when it descends to the set cruising altitude, it will reach the tunnel entrance;

(4)、飞行控制模块控制自主飞行四旋翼穿行隧道(4) The flight control module controls the autonomous flying quadrotor to travel through the tunnel

(4.1)、USB摄像头在LED补光灯的配合下周期性采集隧道内的环境图像;(4.1) The USB camera periodically collects the environmental images in the tunnel with the cooperation of the LED fill light;

(4.2)、开启自主飞行四旋翼的巡线线程,得到自主飞行四旋翼偏离黄色标识线的实际距离d与自主飞行四旋翼飞行方向与隧道走向的偏角θ;(4.2), open the line inspection thread of the autonomous flight quadrotor, and obtain the actual distance d of the autonomous flight quadrotor deviating from the yellow marking line and the deviation angle θ between the flight direction of the autonomous flight quadrotor and the tunnel direction;

(4.2.1)、将环境图像转换到HSV颜色空间,再根据黄色的HSV范围值对环境图像进行二值化处理,得到二值化图像;(4.2.1) Convert the environmental image to the HSV color space, and then perform a binarization process on the environmental image according to the yellow HSV range value to obtain a binarized image;

(4.2.2)、利用开运算滤除二值化图像中的白色噪点,再利用闭运算连接二值化图像中离散的白色区域,得到理想二值化图;(4.2.2), use the open operation to filter out the white noise in the binarized image, and then use the closed operation to connect the discrete white areas in the binarized image to obtain an ideal binarized image;

(4.2.3)、取理想二值化图像上方前n行图像,对于第一行,从理想二值化图最左侧开始向右寻找白色区域左边沿,以及从最右侧开始向左寻找白色区域右边沿,得到该行图像的白色区域,再找出该白色区域中心;对于剩下的n-1行图像,在其上一行寻找所得左右边沿位置附近m个像素范围内寻找本行的左右边沿,得到本行图像的白色区域,再找出该行图像对应的白色区域中心;(4.2.3), take the first n rows of images above the ideal binarized image, for the first row, start from the leftmost left of the ideal binarized image to the right to find the left edge of the white area, and start from the rightmost to the left For the right edge of the white area, get the white area of the line of the image, and then find the center of the white area; for the remaining n-1 lines of images, find the left and right edges in the previous line within m pixels near the position of the left and right edges to find the line of this line. Left and right edges, get the white area of the image in this row, and then find the center of the white area corresponding to the image in this row;

(4.2.4)、将找到的n行图像的白色区域中心与理想二值化图中对应行的绝对中心作差,再累加求平均,将平均差值作为自主飞行四旋翼偏离白色区域的像素量;(4.2.4) Make the difference between the center of the white area of the n-line image and the absolute center of the corresponding line in the ideal binarized image, then accumulate and average, and use the average difference as the pixel of the autonomous flying quadrotor that deviates from the white area quantity;

(4.2.5)、根据自主飞行四旋翼偏离白色区域的像素量、自主飞行四旋翼当前实时高度信息,以及USB摄像头自带的焦距信息,利用针孔成像模型计算自主飞行四旋翼偏离黄色标识线的实际距离d;(4.2.5) According to the pixel amount of the autonomous flight quadrotor deviating from the white area, the current real-time altitude information of the autonomous flight quadrotor, and the focal length information of the USB camera, the pinhole imaging model is used to calculate the deviation of the autonomous flight quadrotor from the yellow marking line the actual distance d;

(4.2.6)、在n行图像中,选取第n/3行和第2n/3行提取得到的白色区域中心,并对这两行的白色区域中心的差值e,由arctan(e/(n/3))计算反正切值,得到自主飞行四旋翼飞行方向与隧道走向的偏角θ;(4.2.6) In the n-line image, select the center of the white area extracted from the n/3th line and the 2n/3th line, and calculate the difference e between the white area centers of these two lines by arctan(e/ (n/3)) Calculate the arctangent value to obtain the deflection angle θ between the flight direction of the autonomous flying quadrotor and the direction of the tunnel;

(4.3)、开启自主飞行四旋翼的光流线程,得到自主飞行四旋翼的实时飞行速度v;(4.3), open the optical flow thread of the autonomous flying quadrotor, and obtain the real-time flight speed v of the autonomous flying quadrotor;

(4.3.1)、在自主飞行四旋翼上电的初始时刻t0,对USB摄像头初始时刻采集的环境图像运行角点检测算法,得到k个Shi-Tomasi角点,再将这k个Shi-Tomasi角点作为本时刻环境图像的Shi-Tomasi角点集a(t0);(4.3.1) At the initial time t 0 when the autonomous flying quadrotor is powered on, run the corner detection algorithm on the environmental image collected by the USB camera at the initial time to obtain k Shi-Tomasi corners, and then use the k Shi-Tomasi corners The Tomasi corner point is used as the Shi-Tomasi corner point set a(t 0 ) of the environment image at this moment;

(4.3.2)、在随后的其余时刻,周期性采集环境图像,并利用LK光流运动估计法对当前时刻的环境图像进行处理,寻找前一时刻环境图像的Shi-Tomasi角点集a(t-1)中的Shi-Tomasi角点的匹配点,并组成点集b(t),再从点集a(t-1)中剔除未能寻找到匹配点的Shi-Tomasi角点,从而得到当前时刻的Shi-Tomasi角点集a(t);(4.3.2) In the rest of the following moments, periodically collect the environmental image, and use the LK optical flow motion estimation method to process the environmental image at the current moment to find the Shi-Tomasi corner point set a of the environmental image at the previous moment ( The matching points of the Shi-Tomasi corner points in t-1) are formed into a point set b(t), and then the Shi-Tomasi corner points that fail to find matching points are eliminated from the point set a(t-1), so that Get the Shi-Tomasi corner set a(t) at the current moment;

(4.3.3)、判断a(t)中Shi-Tomasi角点个数是否大于阈值M,如果大于则进入步骤(4.3.4),否则返回步骤(4.3.1);(4.3.3), determine whether the number of Shi-Tomasi corner points in a(t) is greater than the threshold M, if so, go to step (4.3.4), otherwise return to step (4.3.1);

(4.3.4)将a(t)与b(t)中对应点间的坐标偏差累加求平均,得到像素意义上的飞行速度信息v1(4.3.4) Accumulate and average the coordinate deviations between the corresponding points in a(t) and b(t) to obtain the flight speed information v 1 in the pixel sense;

(4.3.5)、利用自主飞行四旋翼当前飞行的实时高度及实时姿态角对v1进行补偿,得到飞行速度信息v2(4.3.5), use the real-time altitude and real-time attitude angle of the current flight of the autonomous flying quadrotor to compensate v1, and obtain the flight speed information v2 ;

(4.3.6)、利用加速度计及陀螺仪估计出惯导速度信息v3,再将v3与v2进行卡尔曼滤波融合,得到自主飞行四旋翼实时飞行速度v;(4.3.6), use the accelerometer and gyroscope to estimate the inertial navigation speed information v 3 , and then perform Kalman filter fusion with v 3 and v 2 to obtain the real-time flight speed v of the autonomous flying quadrotor;

(4.4)、根据自主飞行四旋翼偏离黄色标识线的实际距离d、实时飞行速度v以及自主飞行四旋翼飞行方向与隧道走向的偏角θ,通过飞行控制模块控制自主飞行四旋翼飞行,使d和θ趋近于0,并向前巡航飞行,当机头激光测距传感器测量到的距离为隧道半个宽度时到达隧道出口;(4.4) According to the actual distance d of the autonomous flight quadrotor deviating from the yellow marking line, the real-time flight speed v and the deviation angle θ between the flight direction of the autonomous flight quadrotor and the tunnel direction, the flight control module controls the autonomous flight quadrotor to fly, so that d and θ approach 0, and cruise forward, and reach the tunnel exit when the distance measured by the nose laser ranging sensor is half the width of the tunnel;

(5)、控制自主飞行四旋翼从隧道出口中心位置驶离隧道井口(5) Control the autonomous flying quadrotor to leave the tunnel wellhead from the center of the tunnel exit

利用机头及机身右侧的激光测距传感器定位自主飞行四旋翼的实时位置信息,飞行控制模块控制自主飞行四旋翼从隧道出口处垂直上升,当上升到隧道顶部平面时,改用机尾及机身右侧的激光测距传感器定位自主飞行四旋翼的实时位置信息,飞行控制模块控制自主飞行四旋翼继续从隧道出口中心位置垂直上升,当上升到高于隧道井口后驶离隧道井口。Using the laser ranging sensor on the nose and the right side of the fuselage to locate the real-time position information of the autonomous flying quadrotor, the flight control module controls the autonomous flying quadrotor to rise vertically from the tunnel exit. When it rises to the top plane of the tunnel, the tail is used instead. And the laser ranging sensor on the right side of the fuselage locates the real-time position information of the autonomous flying quadrotor. The flight control module controls the autonomous flying quadrotor to continue to rise vertically from the center of the tunnel exit. When it rises above the tunnel wellhead, it leaves the tunnel wellhead.

本发明的发明目的是这样实现的:The purpose of the invention of the present invention is achieved in this way:

本发明基于视觉巡视的自主飞行四旋翼穿行隧道方法,利用自主飞行四旋翼自主寻找入口并进入隧道,在隧道内部,再通过底部摄像头对自主飞行四旋翼实现自主导航飞行,在飞行的同时利用摄像头对隧道内部电缆及周边情况进行实时检测,最后自主飞出隧道,整个过程无须人工干预,具有良好的自主飞行及导航能力,还具有检测范围广、速度快、精确度高等特点。The present invention is based on a visual inspection-based method for autonomously flying quadrotors to travel through tunnels. The autonomous flight quadrotors are used to autonomously find the entrance and enter the tunnel. Inside the tunnel, the autonomous flight quadrotors are autonomously navigated through the bottom camera, and the camera is used while flying. It conducts real-time detection of the cables inside the tunnel and the surrounding conditions, and finally flies out of the tunnel autonomously. The whole process does not require manual intervention. It has good autonomous flight and navigation capabilities, and also has the characteristics of wide detection range, high speed and high accuracy.

同时,本发明基于视觉巡视的自主飞行四旋翼穿行隧道方法还具有以下有益效果:At the same time, the present invention based on the visual inspection of the autonomous flying quadrotor traveling through the tunnel method also has the following beneficial effects:

(1)、目前,隧道用检测机器人多以轨道式为主,而这种方法需要在隧道顶上或者地面铺设轨道,且机器人需提前部署在隧道中,即每条隧道都需配备一个机器人,花费成本高需巡视的且检测范围也有限,而本发明无需铺设轨道,具有低成本的优点;(1) At present, most of the inspection robots used in tunnels are mainly track type, and this method needs to lay tracks on the top of the tunnel or on the ground, and the robot needs to be deployed in the tunnel in advance, that is, each tunnel needs to be equipped with a robot. The cost is high, the inspection range is limited, and the present invention does not need to lay tracks, and has the advantage of low cost;

(2)、本发明可全自主地实现出入隧道,无需提前部署在目标隧道中,一台本机器人即可完成一个片区所有隧道的巡视任务,且机器人本身的维护工作也较需下井维护的有轨机器人方便很多;(2) The present invention can fully autonomously realize the entry and exit of the tunnel, without being deployed in the target tunnel in advance, one robot can complete the inspection task of all the tunnels in an area, and the maintenance work of the robot itself is also more difficult than the rails that need to go down for maintenance. Robots are much more convenient;

(3)、本发明在隧道内有更好地通过性,只要留有一定的空间,四旋翼便能在隧道内开展检测巡视任务,受隧道内环境的影响较小;(3), the present invention has better passability in the tunnel, as long as a certain space is left, the quadrotor can carry out detection and inspection tasks in the tunnel, and is less affected by the environment in the tunnel;

(4)、本发明能够替代人工下井完成检测巡视任务,使用及维护过程都无需工人下井,极大地缩短检测时间,保障人员安全,降低使用及维护成本,更加安全可靠。(4) The present invention can replace the manual downhole to complete the inspection and inspection task, and the use and maintenance process does not require workers to go downhole, greatly shortens the inspection time, ensures the safety of personnel, reduces the use and maintenance costs, and is safer and more reliable.

附图说明Description of drawings

图1是隧道的剖面结构示意图;Fig. 1 is the sectional structure schematic diagram of the tunnel;

图2是隧道的俯视结构示意图;Fig. 2 is the top view structure schematic diagram of tunnel;

图3是本发明基于视觉巡视的自主飞行四旋翼穿行隧道方法流程图;Fig. 3 is the flow chart of the present invention's self-flying quadrotor traveling through tunnel method based on visual inspection;

具体实施方式Detailed ways

下面结合附图对本发明的具体实施方式进行描述,以便本领域的技术人员更好地理解本发明。需要特别提醒注意的是,在以下的描述中,当已知功能和设计的详细描述也许会淡化本发明的主要内容时,这些描述在这里将被忽略。The specific embodiments of the present invention are described below with reference to the accompanying drawings, so that those skilled in the art can better understand the present invention. It should be noted that, in the following description, when the detailed description of known functions and designs may dilute the main content of the present invention, these descriptions will be omitted here.

实施例Example

在本实施例中,隧道可抽象为图1及图2所示结构,隧道出入口均为三面封闭型下面结合图1和图2对本发明进行详细说明。。In this embodiment, the tunnel can be abstracted into the structure shown in FIG. 1 and FIG. 2 , and the entrance and exit of the tunnel are closed on three sides. The present invention will be described in detail below with reference to FIGS. 1 and 2 . .

如图3所示,本发明一种基于视觉巡视的自主飞行四旋翼穿行隧道方法,具体包括以下步骤:As shown in FIG. 3 , a method of autonomously flying quadrotors traveling through tunnels based on visual inspection of the present invention specifically includes the following steps:

S1、沿着隧道中线添加黄色标识线,在自主飞行四旋翼的底部安装一个USB摄像头,并加装LED补光灯确保摄像头在隧道内部正常工作,在机头前后方及机身右侧处加装微型激光测距传感器;S1. Add a yellow marking line along the center line of the tunnel, install a USB camera at the bottom of the autonomous flying quadrotor, and install LED fill light to ensure that the camera works normally inside the tunnel. Install miniature laser ranging sensor;

S2、控制自主飞行四旋翼驶入隧道井口S2. Control the autonomous flying quadrotor to drive into the tunnel wellhead

S2.1、自主飞行四旋翼上电初始化;S2.1. Power-on initialization of the autonomous flight quadrotor;

S2.2、手动起飞自主飞行四旋翼,使自主飞行四旋翼离地面一定高度时,此高度无严格要求,再手动遥控自主飞行四旋翼驶入隧道井口附近,一般情况是自主飞行四旋翼飞到机头离对侧墙面1米左右距离处,通过遥控器切换自主飞行四旋翼为全自主飞行模式;S2.2. Manual take-off of the autonomous flying quadrotor, so that when the autonomous flying quadrotor is at a certain height from the ground, there is no strict requirement for this height, and then manually remote control the autonomous flying quadrotor and drive it into the vicinity of the tunnel wellhead. Generally, the autonomous flying quadrotor flies to When the nose is about 1 meter away from the opposite side wall, switch the autonomous flight quadrotor to fully autonomous flight mode through the remote control;

S2.3、飞行控制模块计算自主飞行四旋翼的实时姿态角;S2.3. The flight control module calculates the real-time attitude angle of the autonomous flying quadrotor;

S2.4、自主飞行四旋翼利用机头及机身右侧的激光测距传感器定位当前自主飞行四旋翼的实时位置信息,并与加速度计融合估计飞行速度,结合实时姿态角、位置信息和飞行速度,飞行控制模块控制自主飞行四旋翼飞向入隧道井口中心位置;S2.4. The autonomous flying quadrotor uses the laser ranging sensor on the nose and the right side of the fuselage to locate the real-time position information of the current autonomous flying quadrotor, and fuses it with the accelerometer to estimate the flight speed, combining the real-time attitude angle, position information and flight speed speed, the flight control module controls the autonomous flying quadrotor to fly to the center of the tunnel wellhead;

S3、控制自主飞行四旋翼从隧道井口中心位置驶入隧道入口S3. Control the autonomous flying quadrotor to drive into the tunnel entrance from the center of the tunnel wellhead

飞行控制模块控制自主飞行四旋翼从隧道井口中心位置处垂直下降,当下降到隧道顶部平面时,由于前向墙面的消失,前向激光测距传感器测得距离将发生大的跳变或直接失效,与此同时,后向的激光测距传感器开始能得到有效数据,因此,控制算法通过感测这个前向测距的跳变,立即启用机尾及机身右侧的激光测距传感器用于获取自主飞行四旋翼的实时位置信息,根据当前位置信息和实时姿态角继续垂直下降,当下降到设定的巡航高度时,达到隧道入口;The flight control module controls the autonomous flying quadrotor to descend vertically from the center of the tunnel wellhead. When descending to the top plane of the tunnel, due to the disappearance of the forward wall, the distance measured by the forward laser ranging sensor will jump greatly or directly. At the same time, the backward laser ranging sensor can start to obtain valid data. Therefore, the control algorithm immediately activates the laser ranging sensor on the tail and the right side of the fuselage by sensing the jump in the forward ranging. In order to obtain the real-time position information of the autonomous flying quadrotor, it continues to descend vertically according to the current position information and real-time attitude angle, and when it descends to the set cruising altitude, it reaches the tunnel entrance;

S4、飞行控制模块控制自主飞行四旋翼穿行隧道S4. The flight control module controls the autonomous flying quadrotor to travel through the tunnel

S4.1、USB摄像头在LED补光灯的配合下周期性采集隧道内的环境图像;S4.1, the USB camera periodically collects the environmental images in the tunnel with the cooperation of the LED fill light;

S4.2、开启自主飞行四旋翼的巡线线程,得到自主飞行四旋翼偏离黄色标识线的实际距离d与自主飞行四旋翼飞行方向与隧道走向的偏角θ;S4.2. Open the line inspection thread of the autonomous flight quadrotor, and obtain the actual distance d of the autonomous flight quadrotor deviating from the yellow marking line and the deviation angle θ between the flight direction of the autonomous flight quadrotor and the tunnel direction;

S4.2.1、将环境图像转换到HSV颜色空间,再根据黄色的HSV范围值对环境图像进行二值化处理,得到二值化图像,其中,黄色的HSV范围值为:h:26-34;s:43-255;v:46-255;S4.2.1. Convert the environmental image to the HSV color space, and then perform binarization processing on the environmental image according to the HSV range value of yellow to obtain a binarized image, where the HSV range value of yellow is: h:26-34; s: 43-255; v: 46-255;

S4.2.2、利用开运算滤除二值化图像中的白色噪点,再利用闭运算连接二值化图像中离散的白色区域,得到理想二值化图;S4.2.2. Use the open operation to filter out the white noise in the binarized image, and then use the closed operation to connect the discrete white areas in the binarized image to obtain an ideal binarized image;

S4.2.3、取理想二值化图像上方前100行图像,对于第一行,从理想二值化图最左侧开始向右寻找白色区域左边沿,以及从最右侧开始向左寻找白色区域右边沿,得到该行图像的白色区域,再找出该白色区域中心;对于剩下的99行图像,在其上一行寻找所得左右边沿位置附近30个像素范围内寻找本行的左右边沿,得到本行图像的白色区域,再找出该行图像对应的白色区域中心;S4.2.3. Take the first 100 lines of images above the ideal binarized image. For the first line, start from the leftmost edge of the ideal binarized image to the right to find the left edge of the white area, and start from the rightmost to the left to search for the white area. The right edge, get the white area of the line image, and then find the center of the white area; for the remaining 99 lines of images, search for the left and right edges of the current line within 30 pixels near the left and right edges of the previous line, and get The white area of the image in this line, and then find the center of the white area corresponding to the image in this line;

S4.2.4、将找到的100行图像的白色区域中心与理想二值化图中对应行的绝对中心作差,再累加求平均,将平均差值作为自主飞行四旋翼偏离白色区域的像素量;S4.2.4. Make the difference between the center of the white area of the 100-line image and the absolute center of the corresponding line in the ideal binarized image, and then accumulate and average, and use the average difference as the amount of pixels that the autonomous flying quadrotor deviates from the white area;

S4.2.5、根据自主飞行四旋翼偏离白色区域的像素量、自主飞行四旋翼当前实时高度信息,以及USB摄像头自带的焦距信息,利用针孔成像模型计算自主飞行四旋翼偏离黄色标识线的实际距离d;S4.2.5. According to the pixel amount of the autonomous flight quadrotor deviating from the white area, the current real-time altitude information of the autonomous flight quadrotor, and the focal length information of the USB camera, use the pinhole imaging model to calculate the actual deviation of the autonomous flight quadrotor from the yellow marking line. distance d;

S4.2.6、在100行图像中,选取第33行和第66行提取得到的白色区域中心,并对这两行的白色区域中心的差值e,由arctan(e/33)计算反正切值,得到自主飞行四旋翼飞行方向与隧道走向的偏角θ;S4.2.6. In the 100-line image, select the center of the white area extracted from the 33rd line and the 66th line, and calculate the arc tangent value by arctan(e/33) for the difference e between the white area centers of the two lines , get the deflection angle θ between the flight direction of the autonomous flying quadrotor and the direction of the tunnel;

S4.3、开启自主飞行四旋翼的光流线程,得到自主飞行四旋翼的实时飞行速度v;S4.3. Open the optical flow thread of the autonomous flying quadrotor to obtain the real-time flight speed v of the autonomous flying quadrotor;

S4.3.1、在自主飞行四旋翼上电的初始时刻t0,对USB摄像头初始时刻采集的环境图像运行角点检测算法,得到200个Shi-Tomasi角点,再将这200个Shi-Tomasi角点作为本时刻环境图像的Shi-Tomasi角点集a(t0);S4.3.1. At the initial time t 0 when the autonomous flying quadrotor is powered on, run the corner detection algorithm on the environmental image collected by the USB camera at the initial time to obtain 200 Shi-Tomasi corners, and then use these 200 Shi-Tomasi corners point as the Shi-Tomasi corner point set a(t 0 ) of the environment image at this moment;

S4.3.2、在随后的其余时刻,周期性采集环境图像,并利用LK光流运动估计法对当前时刻的环境图像进行处理,寻找前一时刻环境图像的Shi-Tomasi角点集a(t-1)中的Shi-Tomasi角点的匹配点,并组成点集b(t),再从点集a(t-1)中剔除未能寻找到匹配点的Shi-Tomasi角点,从而得到当前时刻的Shi-Tomasi角点集a(t);S4.3.2. At the rest of the following moments, periodically collect environmental images, and use the LK optical flow motion estimation method to process the environmental images at the current moment to find the Shi-Tomasi corner point set a(t- The matching points of the Shi-Tomasi corner points in 1) are formed into a point set b(t), and then the Shi-Tomasi corner points that fail to find a matching point are removed from the point set a(t-1), so as to obtain the current The Shi-Tomasi corner set a(t) at time;

S4.3.3、判断a(t)中Shi-Tomasi角点个数是否大于阈值30,如果大于则进入步骤S4.3.4,否则返回步骤S4.3.1;S4.3.3. Determine whether the number of Shi-Tomasi corner points in a(t) is greater than the threshold 30, if so, go to step S4.3.4, otherwise return to step S4.3.1;

S4.3.4、将a(t)与b(t)中对应点间的坐标偏差累加求平均,得到像素意义上的飞行速度信息v1S4.3.4. Accumulate and average the coordinate deviations between the corresponding points in a(t) and b(t) to obtain the flight speed information v 1 in the pixel sense;

S4.3.5、利用自主飞行四旋翼当前飞行的实时高度及实时姿态角对v1进行补偿,得到飞行速度信息v2S4.3.5. Compensate v1 by using the real-time altitude and real-time attitude angle of the current flight of the autonomous flying quadrotor to obtain the flight speed information v2 ;

S4.3.6、利用加速度计及陀螺仪估计出惯导速度信息v3,再将v3与v2进行卡尔曼滤波融合,得到自主飞行四旋翼实时飞行速度v;S4.3.6. Use the accelerometer and gyroscope to estimate the inertial navigation speed information v 3 , and then perform Kalman filter fusion with v 3 and v 2 to obtain the real-time flight speed v of the autonomous flying quadrotor;

S4.4、根据自主飞行四旋翼偏离黄色标识线的实际距离d、实时飞行速度v以及自主飞行四旋翼飞行方向与隧道走向的偏角θ,通过飞行控制模块控制自主飞行四旋翼飞行,使d和θ趋近于0,并向前巡航飞行,当机头激光测距传感器测量到的距离为隧道半个宽度时到达隧道出口中心位置;S4.4. According to the actual distance d of the autonomous flight quadrotor deviating from the yellow marking line, the real-time flight speed v, and the deviation angle θ between the flight direction of the autonomous flight quadrotor and the tunnel direction, the autonomous flight quadrotor is controlled by the flight control module to fly, so that d and θ approach 0, and cruise forward, and reach the center of the tunnel exit when the distance measured by the laser ranging sensor on the nose is half the width of the tunnel;

S5、控制自主飞行四旋翼从隧道出口中心位置驶离隧道井口S5. Control the autonomous flying quadrotor to leave the tunnel wellhead from the center of the tunnel exit

利用机头及机身右侧的激光测距传感器定位自主飞行四旋翼的实时位置信息,飞行控制模块控制自主飞行四旋翼从隧道出口中心位置处垂直上升,当上升到隧道顶部平面时,改用机尾及机身右侧的激光测距传感器定位自主飞行四旋翼的实时位置信息,飞行控制模块控制自主飞行四旋翼继续从隧道出口中心位置垂直上升,当上升到高于隧道井口后驶离隧道井口。Using the laser ranging sensor on the nose and the right side of the fuselage to locate the real-time position information of the autonomous flying quadrotor, the flight control module controls the autonomous flying quadrotor to rise vertically from the center of the tunnel exit. When it rises to the top plane of the tunnel, use The laser ranging sensors on the tail and the right side of the fuselage locate the real-time position information of the autonomous flying quadrotor. The flight control module controls the autonomous flying quadrotor to continue to rise vertically from the center of the tunnel exit. When it rises above the tunnel wellhead, it leaves the tunnel. wellhead.

尽管上面对本发明说明性的具体实施方式进行了描述,以便于本技术领域的技术人员理解本发明,但应该清楚,本发明不限于具体实施方式的范围,对本技术领域的普通技术人员来讲,只要各种变化在所附的权利要求限定和确定的本发明的精神和范围内,这些变化是显而易见的,一切利用本发明构思的发明创造均在保护之列。Although the illustrative specific embodiments of the present invention have been described above to facilitate the understanding of the present invention by those skilled in the art, it should be clear that the present invention is not limited to the scope of the specific embodiments. For those skilled in the art, As long as various changes are within the spirit and scope of the present invention as defined and determined by the appended claims, these changes are obvious, and all inventions and creations utilizing the inventive concept are included in the protection list.

Claims (1)

1. The method for the autonomous flight four-rotor-wing tunnel passing based on the visual inspection is characterized by comprising the following steps of:
(1) adding a yellow identification line along the central line of the tunnel, mounting a USB camera at the bottom of the four self-flying rotors, additionally mounting L ED light supplement lamps to ensure that the camera normally works in the tunnel, and additionally mounting miniature laser ranging sensors at the front and rear parts of the camera and the right side of the machine body;
(2) four rotors of controlling independent flight are sailed into tunnel well head
(2.1) carrying out power-on initialization on four autonomous flight rotors;
(2.2) manually taking off and autonomously flying four rotors to make autonomous flyingDistance h between four rotors and ground1Then manually remotely controlling the four self-flying rotors to drive into the vicinity of the tunnel wellhead, and switching the four self-flying rotors into a full self-flying mode through a remote controller;
(2.3) calculating the real-time attitude angle of the four autonomically flying rotors by the flight control module;
(2.4) positioning real-time position information of the current autonomous flight four-rotor by using laser ranging sensors on the right sides of the machine head and the machine body through the autonomous flight four-rotor, fusing the real-time position information and the current autonomous flight four-rotor with an accelerometer to estimate flight speed, and controlling the autonomous flight four-rotor to fly to the central position of a tunnel wellhead by a flight control module by combining a real-time attitude angle, the position information and the flight speed;
(3) the four rotors which are controlled to fly autonomously are driven into the tunnel entrance from the center of the tunnel wellhead
The flight control module controls the autonomous flight quadrotors to vertically descend from the center of a tunnel wellhead, when the autonomous flight quadrotors descend to a tunnel top plane, the laser ranging sensors on the tail and the right side of the machine body are used for positioning real-time position information of the autonomous flight quadrotors, the autonomous flight quadrotors continue to vertically descend according to the current position information and the real-time attitude angle, and when the autonomous flight quadrotors descend to a set cruising height, the autonomous flight quadrotors reach a tunnel entrance;
(4) four-rotor-wing passing tunnel controlled by flight control module to fly autonomously
(4.1) periodically collecting an environment image in the tunnel by the USB camera under the coordination of an L ED supplementary lighting lamp;
(4.2) starting a line patrol thread of the four self-flying rotors to obtain the actual distance d of the four self-flying rotors deviating from the yellow identification line and the deflection angle theta of the four self-flying rotors in the flying direction and the tunnel trend;
(4.2.1) converting the environment image into an HSV color space, and then carrying out binarization processing on the environment image according to the yellow HSV range value to obtain a binarization image;
(4.2.2) filtering white noise points in the binary image by utilizing open operation, and connecting discrete white areas in the binary image by utilizing closed operation to obtain an ideal binary image;
(4.2.3) taking the front n rows of images above the ideal binary image, searching the left edge of a white area from the leftmost side of the ideal binary image to the right and searching the right edge of the white area from the rightmost side to the left for the first row to obtain the white area of the row of images, and then finding out the center of the white area; for the rest n-1 lines of images, searching the left edge and the right edge of the line in the range of m pixels near the left edge and the right edge searched in the previous line of the line to obtain the white area of the line of images, and then finding out the center of the white area corresponding to the line of images;
(4.2.4) making a difference between the center of the white area of the found n rows of images and the absolute center of the corresponding row in the ideal binary image, accumulating and averaging, and taking the average difference value as the pixel quantity of the autonomous flight quadrotors deviating from the white area;
(4.2.5) calculating the actual distance d of the autonomous flight four rotors deviating from the yellow identification line by using a pinhole imaging model according to the pixel quantity of the autonomous flight four rotors deviating from the white area, the current real-time height information of the autonomous flight four rotors and the focal length information of the USB camera;
(4.2.6) selecting the white area centers extracted from the n/3 th line and the 2n/3 th line from the n-line images, and calculating the arctan value of the difference e of the white area centers of the two lines by arctan (e/(n/3)) to obtain the drift angle theta of the flight direction of the autonomous flight four rotors and the tunnel trend;
(4.3) starting an optical flow thread of the four independent flying rotors to obtain the real-time flying speed v of the four independent flying rotors;
(4.3.1) at the initial moment t of the powering-on of the four autonomy flight rotors0Operating a corner detection algorithm on an environment image acquired at the initial moment of the USB camera to obtain k Shi-Tomasi corners, and taking the k Shi-Tomasi corners as a Shi-Tomasi corner set a (t) of the environment image at the moment0);
(4.3.2) periodically acquiring the environment image at the rest of subsequent moments, processing the environment image at the current moment by using an L K optical flow motion estimation method, searching matching points of Shi-Tomasi angular points in a Shi-Tomasi angular point set a (t-1) of the environment image at the previous moment, forming a point set b (t), and removing Shi-Tomasi angular points, at which the matching points cannot be found, from the point set a (t-1) so as to obtain a Shi-Tomasi angular point set a (t) at the current moment;
(4.3.3), judging whether the number of the Shi-Tomasi angular points in a (t) is larger than a threshold value M, if so, entering a step (4.3.4), otherwise, returning to the step (4.3.1);
(4.3.4) accumulating and averaging coordinate deviations between corresponding points in a (t) and b (t) to obtain pixel-wise flight speed information v1
(4.3.5) utilizing the real-time altitude and the real-time attitude angle of the current flight of the four independent flight rotors to v1Compensating to obtain flight speed information v2
(4.3.6) estimating inertial navigation speed information v by using accelerometer and gyroscope3Then v is further determined3And v2Performing Kalman filtering fusion to obtain the real-time flight speed v of the autonomous flight quadrotors;
(4.4) according to the actual distance d of the autonomous flight four rotors deviating from the yellow identification line, the real-time flight speed v and the deflection angle theta between the flight direction of the autonomous flight four rotors and the tunnel trend, controlling the autonomous flight four rotors to fly through the flight control module, enabling d and theta to approach 0, cruising and flying forwards, and enabling the distance measured by the locomotive laser ranging sensor to reach the tunnel outlet when the distance is half the width of the tunnel;
(5) the four rotors of the autonomous flight are controlled to drive away from the well head of the tunnel from the center position of the tunnel outlet
Utilize the laser range finding sensor location on aircraft nose and fuselage right side to independently fly the real-time positional information of four rotors, flight control module control independently flies four rotors and rises perpendicularly from tunnel exit central point department, when rising to tunnel top plane, change into the laser range finding sensor location on aircraft tail and fuselage right side and independently flies four rotors's of real-time positional information, flight control module control independently flies four rotors and continues to rise perpendicularly from tunnel exit central point, when rising to being higher than tunnel well head after driving away from tunnel well head.
CN201710750773.5A 2017-08-28 2017-08-28 Autonomous flight four-rotor tunnel passing method based on visual inspection Active CN107632615B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN201710750773.5A CN107632615B (en) 2017-08-28 2017-08-28 Autonomous flight four-rotor tunnel passing method based on visual inspection

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN201710750773.5A CN107632615B (en) 2017-08-28 2017-08-28 Autonomous flight four-rotor tunnel passing method based on visual inspection

Publications (2)

Publication Number Publication Date
CN107632615A CN107632615A (en) 2018-01-26
CN107632615B true CN107632615B (en) 2020-07-17

Family

ID=61100590

Family Applications (1)

Application Number Title Priority Date Filing Date
CN201710750773.5A Active CN107632615B (en) 2017-08-28 2017-08-28 Autonomous flight four-rotor tunnel passing method based on visual inspection

Country Status (1)

Country Link
CN (1) CN107632615B (en)

Families Citing this family (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN108803652A (en) * 2018-04-26 2018-11-13 中国计量大学 A kind of autonomous tracking control method of rotor craft
CN112173104A (en) * 2020-09-03 2021-01-05 昆明理工大学 A line-following robot based on a quadrotor aircraft
CN112985296B (en) * 2021-02-06 2022-06-24 郑州地铁集团有限公司 Urban rail transit tunnel structure and control method of protection area
CN113655803A (en) * 2021-08-26 2021-11-16 国网江苏省电力有限公司无锡供电分公司 Vision-based system and method for heading calibration of rotary-wing UAV in tunnel environment

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US6879878B2 (en) * 2001-06-04 2005-04-12 Time Domain Corporation Method and system for controlling a robot
WO2012061137A2 (en) * 2010-10-25 2012-05-10 Lockheed Martin Corporation Building a three dimensional model of an underwater structure
CN102980510A (en) * 2012-08-07 2013-03-20 孟繁志 Laser optical ruler image tree measuring device and method thereof
WO2017091768A1 (en) * 2015-11-23 2017-06-01 Kespry, Inc. Autonomous mission action alteration
US9886845B2 (en) * 2008-08-19 2018-02-06 Digimarc Corporation Methods and systems for content processing

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US6879878B2 (en) * 2001-06-04 2005-04-12 Time Domain Corporation Method and system for controlling a robot
US9886845B2 (en) * 2008-08-19 2018-02-06 Digimarc Corporation Methods and systems for content processing
WO2012061137A2 (en) * 2010-10-25 2012-05-10 Lockheed Martin Corporation Building a three dimensional model of an underwater structure
CN102980510A (en) * 2012-08-07 2013-03-20 孟繁志 Laser optical ruler image tree measuring device and method thereof
WO2017091768A1 (en) * 2015-11-23 2017-06-01 Kespry, Inc. Autonomous mission action alteration

Non-Patent Citations (2)

* Cited by examiner, † Cited by third party
Title
"LED 定位技术在隧道无人驾驶导航中的应用";杜艳忠 等;《测绘通报》;20161231;第100-101页和第154页 *
"基于视觉的机器人在管道检测中的远程控制研究";陈应松 等;《制冷与空调》;20100831;第24卷(第4期);第133-137页 *

Also Published As

Publication number Publication date
CN107632615A (en) 2018-01-26

Similar Documents

Publication Publication Date Title
CN107632615B (en) Autonomous flight four-rotor tunnel passing method based on visual inspection
US10878709B2 (en) System, method, and computer readable medium for autonomous airport runway navigation
CN105389988B (en) A kind of express highway intelligent cruising inspection system of multiple no-manned plane collaboration
CN105739523B (en) A kind of police vehicle-mounted unmanned aerial vehicle monitoring system and control method
CN104699102B (en) A kind of unmanned plane and intelligent vehicle collaborative navigation and investigation monitoring system and method
JP5690539B2 (en) Automatic take-off and landing system
KR101933714B1 (en) System for guiding a drone during the approach phase to a platform, in particular a naval platform, with a view to landing same
CN106697322A (en) Automatic abutting system and method for boarding bridge
JP2022535351A (en) System and method for vehicle navigation
CN110262546A (en) A kind of tunnel intelligent unmanned plane cruising inspection system and method
CN114954525B (en) An unmanned transport vehicle system and operation method suitable for phosphate mining tunnels
Savva et al. ICARUS: Automatic autonomous power infrastructure inspection with UAVs
CN104843176A (en) Unmanned-gyroplane system used for automatic-inspection of bridges and tunnels and navigation method
CN105654773A (en) Intelligent guide system of vehicle-mounted flight-accompanying unmanned plane
WO2020000790A1 (en) Vertical mine shaft detection method and system
CN114162318B (en) An airport foreign body monitoring system
CN107992829A (en) A kind of traffic lights track level control planning extracting method and device
CN114489112A (en) A smart car-unmanned aerial vehicle (UAV) collaborative sensing system and method
CN104360688A (en) Guide device of line-cruising unmanned aerial vehicle and control method of guide device
KR20230115622A (en) boarding bridge autonomous docking apparatus and operation method thereof
CN115272953A (en) AGV navigation method and system based on machine vision and artificial intelligence
Cui et al. Coarse-to-fine visual autonomous unmanned aerial vehicle landing on a moving platform
CN114162317B (en) An airport foreign body monitoring system
CN116109675A (en) A method and device for capturing and sensing the reality of underground coal mine scenes
CN209795830U (en) Pipe gallery inspection device

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant