CN107632615B - Autonomous flight four-rotor tunnel passing method based on visual inspection - Google Patents
Autonomous flight four-rotor tunnel passing method based on visual inspection Download PDFInfo
- Publication number
- CN107632615B CN107632615B CN201710750773.5A CN201710750773A CN107632615B CN 107632615 B CN107632615 B CN 107632615B CN 201710750773 A CN201710750773 A CN 201710750773A CN 107632615 B CN107632615 B CN 107632615B
- Authority
- CN
- China
- Prior art keywords
- tunnel
- flight
- rotors
- autonomous
- flying
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Landscapes
- Control Of Position, Course, Altitude, Or Attitude Of Moving Bodies (AREA)
Abstract
本发明公开了一种基于视觉巡视的自主飞行四旋翼穿行隧道方法,利用自主飞行四旋翼自主寻找入口并进入隧道,在隧道内部,再通过底部摄像头对自主飞行四旋翼实现自主导航飞行,在飞行的同时利用摄像头对隧道内部电缆及周边情况进行实时检测,最后自主飞出隧道,整个过程无须人工干预,具有良好的自主飞行及导航能力,还具有检测范围广、速度快、精确度高等特点。
The invention discloses a method for traveling through a tunnel with an autonomous flying quadrotor based on visual inspection. The autonomous flying quadrotor is used to autonomously find an entrance and enter the tunnel. Inside the tunnel, a bottom camera is used to realize autonomous navigation and flight for the autonomous flying quadrotor. At the same time, the camera is used to detect the cables inside the tunnel and the surrounding conditions in real time, and finally fly out of the tunnel autonomously. The whole process does not require manual intervention. It has good autonomous flight and navigation capabilities, and also has a wide detection range, high speed and high accuracy.
Description
技术领域technical field
本发明属于飞行控制技术领域,更为具体地讲,涉及一种基于视觉巡视的自主飞行四旋翼穿行隧道方法。The invention belongs to the technical field of flight control, and more particularly relates to a method for traveling through a tunnel based on an autonomous flying quadrotor based on visual inspection.
背景技术Background technique
近年来,在大型城市的发展中,以地下电缆方式取代传统的架空线路已经成为主流。统计表明,在许多现代化都市中,地下输电线路的比例已经超过70%。电缆隧道作为容纳电缆数量较多、有供安装和巡视的通道的全封闭地下构筑物,是地下电缆的最佳承载方式,随着地下电缆的普及,如何定期对封闭隧道进行检测并预防突发情况成为了急需解决的问题。In recent years, in the development of large cities, it has become mainstream to replace traditional overhead lines with underground cables. Statistics show that in many modern cities, the proportion of underground transmission lines has exceeded 70%. As a fully enclosed underground structure that accommodates a large number of cables and has passages for installation and inspection, cable tunnels are the best way to carry underground cables. With the popularization of underground cables, how to regularly detect closed tunnels and prevent emergencies has become an urgent problem.
由此人们提出了利用有轨机器人或者人工进行定期检测。其中,有轨机器人主要是依托轨道进行运动的,轨道铺设在隧道顶部或底部,机器人通过自身电机的旋转或者轨道的传动,沿着轨道在隧道内穿行,并且通过机器人搭载的相关传感器对隧道进行实时检测,有轨机器人通常需提前部署在目标隧道中,即一条隧道就需配备一个有轨机器人及配套的轨道设施,且仍需要定期派人下到隧道中检查维护有轨机器人自身,因此使用及维护成本较高,而无需提前部署,能够自主出入隧道,对隧道进行全自主,全面检测的无人装置还没有出现。Therefore, it has been proposed to use orbital robots or humans to perform periodic inspections. Among them, the tracked robot mainly relies on the track to move. The track is laid on the top or bottom of the tunnel. The robot travels through the tunnel along the track through the rotation of its own motor or the transmission of the track, and carries out the tunnel through the relevant sensors carried by the robot. For real-time detection, the rail robot usually needs to be deployed in the target tunnel in advance, that is, a tunnel needs to be equipped with a rail robot and supporting rail facilities, and it is still necessary to regularly send people down to the tunnel to check and maintain the rail robot itself, so use And the maintenance cost is high, without the need for advance deployment, the unmanned device that can enter and exit the tunnel autonomously, conduct full autonomous and comprehensive inspection of the tunnel has not yet appeared.
发明内容SUMMARY OF THE INVENTION
本发明的目的在于克服现有技术的不足,提供一种基于视觉巡视的自主飞行四旋翼穿行隧道方法,利用自主飞行四旋翼自主寻找入口并进入隧道,在隧道内部实现自主导航飞行,并实时检测隧道内部电缆及周边情况,最后自主飞出隧道。The purpose of the present invention is to overcome the deficiencies of the prior art, and to provide a method for traveling through a tunnel based on an autonomous flight quadrotor based on visual inspection. The autonomous flight quadrotor is used to autonomously search for an entrance and enter the tunnel, realize autonomous navigation and flight inside the tunnel, and detect in real time. The cables inside the tunnel and the surrounding conditions finally flew out of the tunnel autonomously.
为实现上述发明目的,本发明基于视觉巡视的自主飞行四旋翼穿行隧道方法,其特征在于,包括以下步骤:In order to achieve the above-mentioned purpose of the invention, the present invention based on the visual inspection of the autonomous flying quadrotor traveling through the tunnel method, is characterized in that, comprises the following steps:
(1)、沿着隧道中线添加黄色标识线,在自主飞行四旋翼的底部安装一个USB摄像头,并加装LED补光灯确保摄像头在隧道内部正常工作,在机头前后方及机身右侧处加装微型激光测距传感器;(1) Add a yellow marking line along the tunnel center line, install a USB camera at the bottom of the autonomous flying quadrotor, and install LED fill light to ensure that the camera works normally inside the tunnel, on the front and rear of the nose and the right side of the fuselage A miniature laser ranging sensor is installed;
(2)、控制自主飞行四旋翼驶入隧道井口(2) Control the autonomous flying quadrotor to drive into the tunnel wellhead
(2.1)、自主飞行四旋翼上电初始化;(2.1) Power-on initialization of the autonomous flight quadrotor;
(2.2)、手动起飞自主飞行四旋翼,使自主飞行四旋翼离地面距离h1,再手动遥控自主飞行四旋翼驶入隧道井口附近,此时,通过遥控器切换自主飞行四旋翼为全自主飞行模式;(2.2) Manually take off the autonomous flight quadrotor, make the autonomous flight quadrotor a distance h 1 from the ground, and then manually control the autonomous flight quadrotor to drive into the tunnel near the wellhead. At this time, switch the autonomous flight quadrotor to fully autonomous flight through the remote control model;
(2.3)、飞行控制模块计算自主飞行四旋翼的实时姿态角;(2.3), the flight control module calculates the real-time attitude angle of the autonomous flying quadrotor;
(2.4)、自主飞行四旋翼利用机头及机身右侧的激光测距传感器定位当前自主飞行四旋翼的实时位置信息,并与加速度计融合估计飞行速度,结合实时姿态角、位置信息和飞行速度,飞行控制模块控制自主飞行四旋翼飞向隧道井口中心位置;(2.4) The autonomous flying quadrotor uses the laser ranging sensor on the nose and the right side of the fuselage to locate the real-time position information of the current autonomous flying quadrotor, and fuses it with the accelerometer to estimate the flight speed, combining the real-time attitude angle, position information and flight speed speed, the flight control module controls the autonomous flying quadrotor to fly to the center of the tunnel wellhead;
(3)、控制自主飞行四旋翼从隧道井口中心位置处驶入隧道入口(3) Control the autonomous flying quadrotor to drive into the tunnel entrance from the center of the tunnel wellhead
飞行控制模块控制自主飞行四旋翼从隧道井口中心位置处垂直下降,当下降到隧道顶部平面时,改用机尾及机身右侧的激光测距传感器定位自主飞行四旋翼的实时位置信息,根据当前位置信息和实时姿态角继续垂直下降,当下降到设定的巡航高度时,达到隧道入口;The flight control module controls the autonomous flight quadrotor to descend vertically from the center of the tunnel wellhead. When it descends to the top plane of the tunnel, the laser ranging sensors on the tail and the right side of the fuselage are used to locate the real-time position information of the autonomous flight quadrotor. The current position information and real-time attitude angle continue to descend vertically, and when it descends to the set cruising altitude, it will reach the tunnel entrance;
(4)、飞行控制模块控制自主飞行四旋翼穿行隧道(4) The flight control module controls the autonomous flying quadrotor to travel through the tunnel
(4.1)、USB摄像头在LED补光灯的配合下周期性采集隧道内的环境图像;(4.1) The USB camera periodically collects the environmental images in the tunnel with the cooperation of the LED fill light;
(4.2)、开启自主飞行四旋翼的巡线线程,得到自主飞行四旋翼偏离黄色标识线的实际距离d与自主飞行四旋翼飞行方向与隧道走向的偏角θ;(4.2), open the line inspection thread of the autonomous flight quadrotor, and obtain the actual distance d of the autonomous flight quadrotor deviating from the yellow marking line and the deviation angle θ between the flight direction of the autonomous flight quadrotor and the tunnel direction;
(4.2.1)、将环境图像转换到HSV颜色空间,再根据黄色的HSV范围值对环境图像进行二值化处理,得到二值化图像;(4.2.1) Convert the environmental image to the HSV color space, and then perform a binarization process on the environmental image according to the yellow HSV range value to obtain a binarized image;
(4.2.2)、利用开运算滤除二值化图像中的白色噪点,再利用闭运算连接二值化图像中离散的白色区域,得到理想二值化图;(4.2.2), use the open operation to filter out the white noise in the binarized image, and then use the closed operation to connect the discrete white areas in the binarized image to obtain an ideal binarized image;
(4.2.3)、取理想二值化图像上方前n行图像,对于第一行,从理想二值化图最左侧开始向右寻找白色区域左边沿,以及从最右侧开始向左寻找白色区域右边沿,得到该行图像的白色区域,再找出该白色区域中心;对于剩下的n-1行图像,在其上一行寻找所得左右边沿位置附近m个像素范围内寻找本行的左右边沿,得到本行图像的白色区域,再找出该行图像对应的白色区域中心;(4.2.3), take the first n rows of images above the ideal binarized image, for the first row, start from the leftmost left of the ideal binarized image to the right to find the left edge of the white area, and start from the rightmost to the left For the right edge of the white area, get the white area of the line of the image, and then find the center of the white area; for the remaining n-1 lines of images, find the left and right edges in the previous line within m pixels near the position of the left and right edges to find the line of this line. Left and right edges, get the white area of the image in this row, and then find the center of the white area corresponding to the image in this row;
(4.2.4)、将找到的n行图像的白色区域中心与理想二值化图中对应行的绝对中心作差,再累加求平均,将平均差值作为自主飞行四旋翼偏离白色区域的像素量;(4.2.4) Make the difference between the center of the white area of the n-line image and the absolute center of the corresponding line in the ideal binarized image, then accumulate and average, and use the average difference as the pixel of the autonomous flying quadrotor that deviates from the white area quantity;
(4.2.5)、根据自主飞行四旋翼偏离白色区域的像素量、自主飞行四旋翼当前实时高度信息,以及USB摄像头自带的焦距信息,利用针孔成像模型计算自主飞行四旋翼偏离黄色标识线的实际距离d;(4.2.5) According to the pixel amount of the autonomous flight quadrotor deviating from the white area, the current real-time altitude information of the autonomous flight quadrotor, and the focal length information of the USB camera, the pinhole imaging model is used to calculate the deviation of the autonomous flight quadrotor from the yellow marking line the actual distance d;
(4.2.6)、在n行图像中,选取第n/3行和第2n/3行提取得到的白色区域中心,并对这两行的白色区域中心的差值e,由arctan(e/(n/3))计算反正切值,得到自主飞行四旋翼飞行方向与隧道走向的偏角θ;(4.2.6) In the n-line image, select the center of the white area extracted from the n/3th line and the 2n/3th line, and calculate the difference e between the white area centers of these two lines by arctan(e/ (n/3)) Calculate the arctangent value to obtain the deflection angle θ between the flight direction of the autonomous flying quadrotor and the direction of the tunnel;
(4.3)、开启自主飞行四旋翼的光流线程,得到自主飞行四旋翼的实时飞行速度v;(4.3), open the optical flow thread of the autonomous flying quadrotor, and obtain the real-time flight speed v of the autonomous flying quadrotor;
(4.3.1)、在自主飞行四旋翼上电的初始时刻t0,对USB摄像头初始时刻采集的环境图像运行角点检测算法,得到k个Shi-Tomasi角点,再将这k个Shi-Tomasi角点作为本时刻环境图像的Shi-Tomasi角点集a(t0);(4.3.1) At the initial time t 0 when the autonomous flying quadrotor is powered on, run the corner detection algorithm on the environmental image collected by the USB camera at the initial time to obtain k Shi-Tomasi corners, and then use the k Shi-Tomasi corners The Tomasi corner point is used as the Shi-Tomasi corner point set a(t 0 ) of the environment image at this moment;
(4.3.2)、在随后的其余时刻,周期性采集环境图像,并利用LK光流运动估计法对当前时刻的环境图像进行处理,寻找前一时刻环境图像的Shi-Tomasi角点集a(t-1)中的Shi-Tomasi角点的匹配点,并组成点集b(t),再从点集a(t-1)中剔除未能寻找到匹配点的Shi-Tomasi角点,从而得到当前时刻的Shi-Tomasi角点集a(t);(4.3.2) In the rest of the following moments, periodically collect the environmental image, and use the LK optical flow motion estimation method to process the environmental image at the current moment to find the Shi-Tomasi corner point set a of the environmental image at the previous moment ( The matching points of the Shi-Tomasi corner points in t-1) are formed into a point set b(t), and then the Shi-Tomasi corner points that fail to find matching points are eliminated from the point set a(t-1), so that Get the Shi-Tomasi corner set a(t) at the current moment;
(4.3.3)、判断a(t)中Shi-Tomasi角点个数是否大于阈值M,如果大于则进入步骤(4.3.4),否则返回步骤(4.3.1);(4.3.3), determine whether the number of Shi-Tomasi corner points in a(t) is greater than the threshold M, if so, go to step (4.3.4), otherwise return to step (4.3.1);
(4.3.4)将a(t)与b(t)中对应点间的坐标偏差累加求平均,得到像素意义上的飞行速度信息v1;(4.3.4) Accumulate and average the coordinate deviations between the corresponding points in a(t) and b(t) to obtain the flight speed information v 1 in the pixel sense;
(4.3.5)、利用自主飞行四旋翼当前飞行的实时高度及实时姿态角对v1进行补偿,得到飞行速度信息v2;(4.3.5), use the real-time altitude and real-time attitude angle of the current flight of the autonomous flying quadrotor to compensate v1, and obtain the flight speed information v2 ;
(4.3.6)、利用加速度计及陀螺仪估计出惯导速度信息v3,再将v3与v2进行卡尔曼滤波融合,得到自主飞行四旋翼实时飞行速度v;(4.3.6), use the accelerometer and gyroscope to estimate the inertial navigation speed information v 3 , and then perform Kalman filter fusion with v 3 and v 2 to obtain the real-time flight speed v of the autonomous flying quadrotor;
(4.4)、根据自主飞行四旋翼偏离黄色标识线的实际距离d、实时飞行速度v以及自主飞行四旋翼飞行方向与隧道走向的偏角θ,通过飞行控制模块控制自主飞行四旋翼飞行,使d和θ趋近于0,并向前巡航飞行,当机头激光测距传感器测量到的距离为隧道半个宽度时到达隧道出口;(4.4) According to the actual distance d of the autonomous flight quadrotor deviating from the yellow marking line, the real-time flight speed v and the deviation angle θ between the flight direction of the autonomous flight quadrotor and the tunnel direction, the flight control module controls the autonomous flight quadrotor to fly, so that d and θ approach 0, and cruise forward, and reach the tunnel exit when the distance measured by the nose laser ranging sensor is half the width of the tunnel;
(5)、控制自主飞行四旋翼从隧道出口中心位置驶离隧道井口(5) Control the autonomous flying quadrotor to leave the tunnel wellhead from the center of the tunnel exit
利用机头及机身右侧的激光测距传感器定位自主飞行四旋翼的实时位置信息,飞行控制模块控制自主飞行四旋翼从隧道出口处垂直上升,当上升到隧道顶部平面时,改用机尾及机身右侧的激光测距传感器定位自主飞行四旋翼的实时位置信息,飞行控制模块控制自主飞行四旋翼继续从隧道出口中心位置垂直上升,当上升到高于隧道井口后驶离隧道井口。Using the laser ranging sensor on the nose and the right side of the fuselage to locate the real-time position information of the autonomous flying quadrotor, the flight control module controls the autonomous flying quadrotor to rise vertically from the tunnel exit. When it rises to the top plane of the tunnel, the tail is used instead. And the laser ranging sensor on the right side of the fuselage locates the real-time position information of the autonomous flying quadrotor. The flight control module controls the autonomous flying quadrotor to continue to rise vertically from the center of the tunnel exit. When it rises above the tunnel wellhead, it leaves the tunnel wellhead.
本发明的发明目的是这样实现的:The purpose of the invention of the present invention is achieved in this way:
本发明基于视觉巡视的自主飞行四旋翼穿行隧道方法,利用自主飞行四旋翼自主寻找入口并进入隧道,在隧道内部,再通过底部摄像头对自主飞行四旋翼实现自主导航飞行,在飞行的同时利用摄像头对隧道内部电缆及周边情况进行实时检测,最后自主飞出隧道,整个过程无须人工干预,具有良好的自主飞行及导航能力,还具有检测范围广、速度快、精确度高等特点。The present invention is based on a visual inspection-based method for autonomously flying quadrotors to travel through tunnels. The autonomous flight quadrotors are used to autonomously find the entrance and enter the tunnel. Inside the tunnel, the autonomous flight quadrotors are autonomously navigated through the bottom camera, and the camera is used while flying. It conducts real-time detection of the cables inside the tunnel and the surrounding conditions, and finally flies out of the tunnel autonomously. The whole process does not require manual intervention. It has good autonomous flight and navigation capabilities, and also has the characteristics of wide detection range, high speed and high accuracy.
同时,本发明基于视觉巡视的自主飞行四旋翼穿行隧道方法还具有以下有益效果:At the same time, the present invention based on the visual inspection of the autonomous flying quadrotor traveling through the tunnel method also has the following beneficial effects:
(1)、目前,隧道用检测机器人多以轨道式为主,而这种方法需要在隧道顶上或者地面铺设轨道,且机器人需提前部署在隧道中,即每条隧道都需配备一个机器人,花费成本高需巡视的且检测范围也有限,而本发明无需铺设轨道,具有低成本的优点;(1) At present, most of the inspection robots used in tunnels are mainly track type, and this method needs to lay tracks on the top of the tunnel or on the ground, and the robot needs to be deployed in the tunnel in advance, that is, each tunnel needs to be equipped with a robot. The cost is high, the inspection range is limited, and the present invention does not need to lay tracks, and has the advantage of low cost;
(2)、本发明可全自主地实现出入隧道,无需提前部署在目标隧道中,一台本机器人即可完成一个片区所有隧道的巡视任务,且机器人本身的维护工作也较需下井维护的有轨机器人方便很多;(2) The present invention can fully autonomously realize the entry and exit of the tunnel, without being deployed in the target tunnel in advance, one robot can complete the inspection task of all the tunnels in an area, and the maintenance work of the robot itself is also more difficult than the rails that need to go down for maintenance. Robots are much more convenient;
(3)、本发明在隧道内有更好地通过性,只要留有一定的空间,四旋翼便能在隧道内开展检测巡视任务,受隧道内环境的影响较小;(3), the present invention has better passability in the tunnel, as long as a certain space is left, the quadrotor can carry out detection and inspection tasks in the tunnel, and is less affected by the environment in the tunnel;
(4)、本发明能够替代人工下井完成检测巡视任务,使用及维护过程都无需工人下井,极大地缩短检测时间,保障人员安全,降低使用及维护成本,更加安全可靠。(4) The present invention can replace the manual downhole to complete the inspection and inspection task, and the use and maintenance process does not require workers to go downhole, greatly shortens the inspection time, ensures the safety of personnel, reduces the use and maintenance costs, and is safer and more reliable.
附图说明Description of drawings
图1是隧道的剖面结构示意图;Fig. 1 is the sectional structure schematic diagram of the tunnel;
图2是隧道的俯视结构示意图;Fig. 2 is the top view structure schematic diagram of tunnel;
图3是本发明基于视觉巡视的自主飞行四旋翼穿行隧道方法流程图;Fig. 3 is the flow chart of the present invention's self-flying quadrotor traveling through tunnel method based on visual inspection;
具体实施方式Detailed ways
下面结合附图对本发明的具体实施方式进行描述,以便本领域的技术人员更好地理解本发明。需要特别提醒注意的是,在以下的描述中,当已知功能和设计的详细描述也许会淡化本发明的主要内容时,这些描述在这里将被忽略。The specific embodiments of the present invention are described below with reference to the accompanying drawings, so that those skilled in the art can better understand the present invention. It should be noted that, in the following description, when the detailed description of known functions and designs may dilute the main content of the present invention, these descriptions will be omitted here.
实施例Example
在本实施例中,隧道可抽象为图1及图2所示结构,隧道出入口均为三面封闭型下面结合图1和图2对本发明进行详细说明。。In this embodiment, the tunnel can be abstracted into the structure shown in FIG. 1 and FIG. 2 , and the entrance and exit of the tunnel are closed on three sides. The present invention will be described in detail below with reference to FIGS. 1 and 2 . .
如图3所示,本发明一种基于视觉巡视的自主飞行四旋翼穿行隧道方法,具体包括以下步骤:As shown in FIG. 3 , a method of autonomously flying quadrotors traveling through tunnels based on visual inspection of the present invention specifically includes the following steps:
S1、沿着隧道中线添加黄色标识线,在自主飞行四旋翼的底部安装一个USB摄像头,并加装LED补光灯确保摄像头在隧道内部正常工作,在机头前后方及机身右侧处加装微型激光测距传感器;S1. Add a yellow marking line along the center line of the tunnel, install a USB camera at the bottom of the autonomous flying quadrotor, and install LED fill light to ensure that the camera works normally inside the tunnel. Install miniature laser ranging sensor;
S2、控制自主飞行四旋翼驶入隧道井口S2. Control the autonomous flying quadrotor to drive into the tunnel wellhead
S2.1、自主飞行四旋翼上电初始化;S2.1. Power-on initialization of the autonomous flight quadrotor;
S2.2、手动起飞自主飞行四旋翼,使自主飞行四旋翼离地面一定高度时,此高度无严格要求,再手动遥控自主飞行四旋翼驶入隧道井口附近,一般情况是自主飞行四旋翼飞到机头离对侧墙面1米左右距离处,通过遥控器切换自主飞行四旋翼为全自主飞行模式;S2.2. Manual take-off of the autonomous flying quadrotor, so that when the autonomous flying quadrotor is at a certain height from the ground, there is no strict requirement for this height, and then manually remote control the autonomous flying quadrotor and drive it into the vicinity of the tunnel wellhead. Generally, the autonomous flying quadrotor flies to When the nose is about 1 meter away from the opposite side wall, switch the autonomous flight quadrotor to fully autonomous flight mode through the remote control;
S2.3、飞行控制模块计算自主飞行四旋翼的实时姿态角;S2.3. The flight control module calculates the real-time attitude angle of the autonomous flying quadrotor;
S2.4、自主飞行四旋翼利用机头及机身右侧的激光测距传感器定位当前自主飞行四旋翼的实时位置信息,并与加速度计融合估计飞行速度,结合实时姿态角、位置信息和飞行速度,飞行控制模块控制自主飞行四旋翼飞向入隧道井口中心位置;S2.4. The autonomous flying quadrotor uses the laser ranging sensor on the nose and the right side of the fuselage to locate the real-time position information of the current autonomous flying quadrotor, and fuses it with the accelerometer to estimate the flight speed, combining the real-time attitude angle, position information and flight speed speed, the flight control module controls the autonomous flying quadrotor to fly to the center of the tunnel wellhead;
S3、控制自主飞行四旋翼从隧道井口中心位置驶入隧道入口S3. Control the autonomous flying quadrotor to drive into the tunnel entrance from the center of the tunnel wellhead
飞行控制模块控制自主飞行四旋翼从隧道井口中心位置处垂直下降,当下降到隧道顶部平面时,由于前向墙面的消失,前向激光测距传感器测得距离将发生大的跳变或直接失效,与此同时,后向的激光测距传感器开始能得到有效数据,因此,控制算法通过感测这个前向测距的跳变,立即启用机尾及机身右侧的激光测距传感器用于获取自主飞行四旋翼的实时位置信息,根据当前位置信息和实时姿态角继续垂直下降,当下降到设定的巡航高度时,达到隧道入口;The flight control module controls the autonomous flying quadrotor to descend vertically from the center of the tunnel wellhead. When descending to the top plane of the tunnel, due to the disappearance of the forward wall, the distance measured by the forward laser ranging sensor will jump greatly or directly. At the same time, the backward laser ranging sensor can start to obtain valid data. Therefore, the control algorithm immediately activates the laser ranging sensor on the tail and the right side of the fuselage by sensing the jump in the forward ranging. In order to obtain the real-time position information of the autonomous flying quadrotor, it continues to descend vertically according to the current position information and real-time attitude angle, and when it descends to the set cruising altitude, it reaches the tunnel entrance;
S4、飞行控制模块控制自主飞行四旋翼穿行隧道S4. The flight control module controls the autonomous flying quadrotor to travel through the tunnel
S4.1、USB摄像头在LED补光灯的配合下周期性采集隧道内的环境图像;S4.1, the USB camera periodically collects the environmental images in the tunnel with the cooperation of the LED fill light;
S4.2、开启自主飞行四旋翼的巡线线程,得到自主飞行四旋翼偏离黄色标识线的实际距离d与自主飞行四旋翼飞行方向与隧道走向的偏角θ;S4.2. Open the line inspection thread of the autonomous flight quadrotor, and obtain the actual distance d of the autonomous flight quadrotor deviating from the yellow marking line and the deviation angle θ between the flight direction of the autonomous flight quadrotor and the tunnel direction;
S4.2.1、将环境图像转换到HSV颜色空间,再根据黄色的HSV范围值对环境图像进行二值化处理,得到二值化图像,其中,黄色的HSV范围值为:h:26-34;s:43-255;v:46-255;S4.2.1. Convert the environmental image to the HSV color space, and then perform binarization processing on the environmental image according to the HSV range value of yellow to obtain a binarized image, where the HSV range value of yellow is: h:26-34; s: 43-255; v: 46-255;
S4.2.2、利用开运算滤除二值化图像中的白色噪点,再利用闭运算连接二值化图像中离散的白色区域,得到理想二值化图;S4.2.2. Use the open operation to filter out the white noise in the binarized image, and then use the closed operation to connect the discrete white areas in the binarized image to obtain an ideal binarized image;
S4.2.3、取理想二值化图像上方前100行图像,对于第一行,从理想二值化图最左侧开始向右寻找白色区域左边沿,以及从最右侧开始向左寻找白色区域右边沿,得到该行图像的白色区域,再找出该白色区域中心;对于剩下的99行图像,在其上一行寻找所得左右边沿位置附近30个像素范围内寻找本行的左右边沿,得到本行图像的白色区域,再找出该行图像对应的白色区域中心;S4.2.3. Take the first 100 lines of images above the ideal binarized image. For the first line, start from the leftmost edge of the ideal binarized image to the right to find the left edge of the white area, and start from the rightmost to the left to search for the white area. The right edge, get the white area of the line image, and then find the center of the white area; for the remaining 99 lines of images, search for the left and right edges of the current line within 30 pixels near the left and right edges of the previous line, and get The white area of the image in this line, and then find the center of the white area corresponding to the image in this line;
S4.2.4、将找到的100行图像的白色区域中心与理想二值化图中对应行的绝对中心作差,再累加求平均,将平均差值作为自主飞行四旋翼偏离白色区域的像素量;S4.2.4. Make the difference between the center of the white area of the 100-line image and the absolute center of the corresponding line in the ideal binarized image, and then accumulate and average, and use the average difference as the amount of pixels that the autonomous flying quadrotor deviates from the white area;
S4.2.5、根据自主飞行四旋翼偏离白色区域的像素量、自主飞行四旋翼当前实时高度信息,以及USB摄像头自带的焦距信息,利用针孔成像模型计算自主飞行四旋翼偏离黄色标识线的实际距离d;S4.2.5. According to the pixel amount of the autonomous flight quadrotor deviating from the white area, the current real-time altitude information of the autonomous flight quadrotor, and the focal length information of the USB camera, use the pinhole imaging model to calculate the actual deviation of the autonomous flight quadrotor from the yellow marking line. distance d;
S4.2.6、在100行图像中,选取第33行和第66行提取得到的白色区域中心,并对这两行的白色区域中心的差值e,由arctan(e/33)计算反正切值,得到自主飞行四旋翼飞行方向与隧道走向的偏角θ;S4.2.6. In the 100-line image, select the center of the white area extracted from the 33rd line and the 66th line, and calculate the arc tangent value by arctan(e/33) for the difference e between the white area centers of the two lines , get the deflection angle θ between the flight direction of the autonomous flying quadrotor and the direction of the tunnel;
S4.3、开启自主飞行四旋翼的光流线程,得到自主飞行四旋翼的实时飞行速度v;S4.3. Open the optical flow thread of the autonomous flying quadrotor to obtain the real-time flight speed v of the autonomous flying quadrotor;
S4.3.1、在自主飞行四旋翼上电的初始时刻t0,对USB摄像头初始时刻采集的环境图像运行角点检测算法,得到200个Shi-Tomasi角点,再将这200个Shi-Tomasi角点作为本时刻环境图像的Shi-Tomasi角点集a(t0);S4.3.1. At the initial time t 0 when the autonomous flying quadrotor is powered on, run the corner detection algorithm on the environmental image collected by the USB camera at the initial time to obtain 200 Shi-Tomasi corners, and then use these 200 Shi-Tomasi corners point as the Shi-Tomasi corner point set a(t 0 ) of the environment image at this moment;
S4.3.2、在随后的其余时刻,周期性采集环境图像,并利用LK光流运动估计法对当前时刻的环境图像进行处理,寻找前一时刻环境图像的Shi-Tomasi角点集a(t-1)中的Shi-Tomasi角点的匹配点,并组成点集b(t),再从点集a(t-1)中剔除未能寻找到匹配点的Shi-Tomasi角点,从而得到当前时刻的Shi-Tomasi角点集a(t);S4.3.2. At the rest of the following moments, periodically collect environmental images, and use the LK optical flow motion estimation method to process the environmental images at the current moment to find the Shi-Tomasi corner point set a(t- The matching points of the Shi-Tomasi corner points in 1) are formed into a point set b(t), and then the Shi-Tomasi corner points that fail to find a matching point are removed from the point set a(t-1), so as to obtain the current The Shi-Tomasi corner set a(t) at time;
S4.3.3、判断a(t)中Shi-Tomasi角点个数是否大于阈值30,如果大于则进入步骤S4.3.4,否则返回步骤S4.3.1;S4.3.3. Determine whether the number of Shi-Tomasi corner points in a(t) is greater than the threshold 30, if so, go to step S4.3.4, otherwise return to step S4.3.1;
S4.3.4、将a(t)与b(t)中对应点间的坐标偏差累加求平均,得到像素意义上的飞行速度信息v1;S4.3.4. Accumulate and average the coordinate deviations between the corresponding points in a(t) and b(t) to obtain the flight speed information v 1 in the pixel sense;
S4.3.5、利用自主飞行四旋翼当前飞行的实时高度及实时姿态角对v1进行补偿,得到飞行速度信息v2;S4.3.5. Compensate v1 by using the real-time altitude and real-time attitude angle of the current flight of the autonomous flying quadrotor to obtain the flight speed information v2 ;
S4.3.6、利用加速度计及陀螺仪估计出惯导速度信息v3,再将v3与v2进行卡尔曼滤波融合,得到自主飞行四旋翼实时飞行速度v;S4.3.6. Use the accelerometer and gyroscope to estimate the inertial navigation speed information v 3 , and then perform Kalman filter fusion with v 3 and v 2 to obtain the real-time flight speed v of the autonomous flying quadrotor;
S4.4、根据自主飞行四旋翼偏离黄色标识线的实际距离d、实时飞行速度v以及自主飞行四旋翼飞行方向与隧道走向的偏角θ,通过飞行控制模块控制自主飞行四旋翼飞行,使d和θ趋近于0,并向前巡航飞行,当机头激光测距传感器测量到的距离为隧道半个宽度时到达隧道出口中心位置;S4.4. According to the actual distance d of the autonomous flight quadrotor deviating from the yellow marking line, the real-time flight speed v, and the deviation angle θ between the flight direction of the autonomous flight quadrotor and the tunnel direction, the autonomous flight quadrotor is controlled by the flight control module to fly, so that d and θ approach 0, and cruise forward, and reach the center of the tunnel exit when the distance measured by the laser ranging sensor on the nose is half the width of the tunnel;
S5、控制自主飞行四旋翼从隧道出口中心位置驶离隧道井口S5. Control the autonomous flying quadrotor to leave the tunnel wellhead from the center of the tunnel exit
利用机头及机身右侧的激光测距传感器定位自主飞行四旋翼的实时位置信息,飞行控制模块控制自主飞行四旋翼从隧道出口中心位置处垂直上升,当上升到隧道顶部平面时,改用机尾及机身右侧的激光测距传感器定位自主飞行四旋翼的实时位置信息,飞行控制模块控制自主飞行四旋翼继续从隧道出口中心位置垂直上升,当上升到高于隧道井口后驶离隧道井口。Using the laser ranging sensor on the nose and the right side of the fuselage to locate the real-time position information of the autonomous flying quadrotor, the flight control module controls the autonomous flying quadrotor to rise vertically from the center of the tunnel exit. When it rises to the top plane of the tunnel, use The laser ranging sensors on the tail and the right side of the fuselage locate the real-time position information of the autonomous flying quadrotor. The flight control module controls the autonomous flying quadrotor to continue to rise vertically from the center of the tunnel exit. When it rises above the tunnel wellhead, it leaves the tunnel. wellhead.
尽管上面对本发明说明性的具体实施方式进行了描述,以便于本技术领域的技术人员理解本发明,但应该清楚,本发明不限于具体实施方式的范围,对本技术领域的普通技术人员来讲,只要各种变化在所附的权利要求限定和确定的本发明的精神和范围内,这些变化是显而易见的,一切利用本发明构思的发明创造均在保护之列。Although the illustrative specific embodiments of the present invention have been described above to facilitate the understanding of the present invention by those skilled in the art, it should be clear that the present invention is not limited to the scope of the specific embodiments. For those skilled in the art, As long as various changes are within the spirit and scope of the present invention as defined and determined by the appended claims, these changes are obvious, and all inventions and creations utilizing the inventive concept are included in the protection list.
Claims (1)
Priority Applications (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
CN201710750773.5A CN107632615B (en) | 2017-08-28 | 2017-08-28 | Autonomous flight four-rotor tunnel passing method based on visual inspection |
Applications Claiming Priority (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
CN201710750773.5A CN107632615B (en) | 2017-08-28 | 2017-08-28 | Autonomous flight four-rotor tunnel passing method based on visual inspection |
Publications (2)
Publication Number | Publication Date |
---|---|
CN107632615A CN107632615A (en) | 2018-01-26 |
CN107632615B true CN107632615B (en) | 2020-07-17 |
Family
ID=61100590
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
CN201710750773.5A Active CN107632615B (en) | 2017-08-28 | 2017-08-28 | Autonomous flight four-rotor tunnel passing method based on visual inspection |
Country Status (1)
Country | Link |
---|---|
CN (1) | CN107632615B (en) |
Families Citing this family (4)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN108803652A (en) * | 2018-04-26 | 2018-11-13 | 中国计量大学 | A kind of autonomous tracking control method of rotor craft |
CN112173104A (en) * | 2020-09-03 | 2021-01-05 | 昆明理工大学 | A line-following robot based on a quadrotor aircraft |
CN112985296B (en) * | 2021-02-06 | 2022-06-24 | 郑州地铁集团有限公司 | Urban rail transit tunnel structure and control method of protection area |
CN113655803A (en) * | 2021-08-26 | 2021-11-16 | 国网江苏省电力有限公司无锡供电分公司 | Vision-based system and method for heading calibration of rotary-wing UAV in tunnel environment |
Citations (5)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US6879878B2 (en) * | 2001-06-04 | 2005-04-12 | Time Domain Corporation | Method and system for controlling a robot |
WO2012061137A2 (en) * | 2010-10-25 | 2012-05-10 | Lockheed Martin Corporation | Building a three dimensional model of an underwater structure |
CN102980510A (en) * | 2012-08-07 | 2013-03-20 | 孟繁志 | Laser optical ruler image tree measuring device and method thereof |
WO2017091768A1 (en) * | 2015-11-23 | 2017-06-01 | Kespry, Inc. | Autonomous mission action alteration |
US9886845B2 (en) * | 2008-08-19 | 2018-02-06 | Digimarc Corporation | Methods and systems for content processing |
-
2017
- 2017-08-28 CN CN201710750773.5A patent/CN107632615B/en active Active
Patent Citations (5)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US6879878B2 (en) * | 2001-06-04 | 2005-04-12 | Time Domain Corporation | Method and system for controlling a robot |
US9886845B2 (en) * | 2008-08-19 | 2018-02-06 | Digimarc Corporation | Methods and systems for content processing |
WO2012061137A2 (en) * | 2010-10-25 | 2012-05-10 | Lockheed Martin Corporation | Building a three dimensional model of an underwater structure |
CN102980510A (en) * | 2012-08-07 | 2013-03-20 | 孟繁志 | Laser optical ruler image tree measuring device and method thereof |
WO2017091768A1 (en) * | 2015-11-23 | 2017-06-01 | Kespry, Inc. | Autonomous mission action alteration |
Non-Patent Citations (2)
Title |
---|
"LED 定位技术在隧道无人驾驶导航中的应用";杜艳忠 等;《测绘通报》;20161231;第100-101页和第154页 * |
"基于视觉的机器人在管道检测中的远程控制研究";陈应松 等;《制冷与空调》;20100831;第24卷(第4期);第133-137页 * |
Also Published As
Publication number | Publication date |
---|---|
CN107632615A (en) | 2018-01-26 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
CN107632615B (en) | Autonomous flight four-rotor tunnel passing method based on visual inspection | |
US10878709B2 (en) | System, method, and computer readable medium for autonomous airport runway navigation | |
CN105389988B (en) | A kind of express highway intelligent cruising inspection system of multiple no-manned plane collaboration | |
CN105739523B (en) | A kind of police vehicle-mounted unmanned aerial vehicle monitoring system and control method | |
CN104699102B (en) | A kind of unmanned plane and intelligent vehicle collaborative navigation and investigation monitoring system and method | |
JP5690539B2 (en) | Automatic take-off and landing system | |
KR101933714B1 (en) | System for guiding a drone during the approach phase to a platform, in particular a naval platform, with a view to landing same | |
CN106697322A (en) | Automatic abutting system and method for boarding bridge | |
JP2022535351A (en) | System and method for vehicle navigation | |
CN110262546A (en) | A kind of tunnel intelligent unmanned plane cruising inspection system and method | |
CN114954525B (en) | An unmanned transport vehicle system and operation method suitable for phosphate mining tunnels | |
Savva et al. | ICARUS: Automatic autonomous power infrastructure inspection with UAVs | |
CN104843176A (en) | Unmanned-gyroplane system used for automatic-inspection of bridges and tunnels and navigation method | |
CN105654773A (en) | Intelligent guide system of vehicle-mounted flight-accompanying unmanned plane | |
WO2020000790A1 (en) | Vertical mine shaft detection method and system | |
CN114162318B (en) | An airport foreign body monitoring system | |
CN107992829A (en) | A kind of traffic lights track level control planning extracting method and device | |
CN114489112A (en) | A smart car-unmanned aerial vehicle (UAV) collaborative sensing system and method | |
CN104360688A (en) | Guide device of line-cruising unmanned aerial vehicle and control method of guide device | |
KR20230115622A (en) | boarding bridge autonomous docking apparatus and operation method thereof | |
CN115272953A (en) | AGV navigation method and system based on machine vision and artificial intelligence | |
Cui et al. | Coarse-to-fine visual autonomous unmanned aerial vehicle landing on a moving platform | |
CN114162317B (en) | An airport foreign body monitoring system | |
CN116109675A (en) | A method and device for capturing and sensing the reality of underground coal mine scenes | |
CN209795830U (en) | Pipe gallery inspection device |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
PB01 | Publication | ||
PB01 | Publication | ||
SE01 | Entry into force of request for substantive examination | ||
SE01 | Entry into force of request for substantive examination | ||
GR01 | Patent grant | ||
GR01 | Patent grant |