Background
The existing vehicle speed measurement is generally based on bayonet speed measurement mostly, the speed measurement is carried out by combining a speed measuring radar basically, the equipment can only shoot and obtain evidence for the vehicle running at an overspeed aiming at a specific position or a specific interval, and the equipment has great limitation.
Because the traditional method for measuring the vehicle speed based on the fixed speed measuring camera and the speed measuring instrument has the defects of limited snapshot area, complex equipment deployment, high equipment manufacturing cost, difficulty in timely repair after damage and the like, and the positions of most speed measuring equipment are known, a vehicle driver can change the vehicle speed in a targeted manner so as to avoid monitoring of the speed measuring equipment. Therefore, the vehicle speed measurement by using the vision of the aerial robot has great practical value and commercial value.
Vehicle speed measurement based on vision of an aerial robot is a challenging and significant task in current intelligent traffic management. When the aerial robot operates, real-time monitoring pictures can be provided for ground personnel such as traffic police and the like, so that the ground personnel can find out drivers violating regulations in a large area. Although the aerial robots can help traffic police to find out dangerous drivers and give warnings and punishments, the aerial robots are less applied to traffic monitoring and management scenes for vehicle speed measurement at present, and the snapshot and evidence collection of illegal violation phenomena such as overspeed and the like still mainly depend on a fixed speed measurement camera and a speed measurement instrument.
The patent [ a method and a monitoring system for monitoring overspeed driving by using an unmanned aerial vehicle, and an unmanned aerial vehicle, CN 111583670 a ] proposes a method and a monitoring system for monitoring overspeed driving by using an unmanned aerial vehicle, and an unmanned aerial vehicle. The displacement of the unmanned aerial vehicle is obtained through the ground speed of the unmanned aerial vehicle within a period of time, meanwhile, the displacement of the target vehicle in a picture within the period of time is calculated, the relative displacement of the vehicle and the unmanned aerial vehicle is calculated according to the ground angle of a pod on the unmanned aerial vehicle and the height of the unmanned aerial vehicle, so that the ground displacement of the target vehicle is calculated, and finally the speed of the vehicle is calculated. This method requires that the control of the flight crew of the drone is required to ensure that the drone has a fixed flying speed, flying height and pod-to-ground angle while the drone is in operation, which puts high demands on the control level of the flight crew. Therefore, the method has great limitation and cannot adapt to complicated and variable environments.
Disclosure of Invention
In order to overcome the defects of the prior art, the invention provides a vehicle speed measuring method based on aerial robot mobile vision, which can acquire the motion trail of a vehicle by acquiring the video image picture shot by the aerial robot; meanwhile, the distance on the graph is calculated by detecting the lane line on the road where the vehicle is located, the ratio of the actual distance to the distance on the graph is obtained, and the actual movement distance of the vehicle is calculated; so as to finally calculate the instantaneous speed of all vehicles in the picture. The method overcomes the defects of low flexibility, high manufacturing cost, difficult installation and the like of the traditional fixed speed measuring equipment, overcomes the high requirement of the existing speed measuring scheme of the aerial vehicle on the flight attitude of the aerial robot, has strong maneuverability and low requirements on the flight attitude and the flying hand, can monitor the speed of the ground vehicle in real time, and is convenient to use.
The technical scheme adopted by the invention for solving the technical problem comprises the following steps:
step 1: collecting data;
acquiring images of different vehicles and roads by using an aerial robot under mobile vision, and labeling the vehicles and the roads in the acquired images;
performing data augmentation on the marked image;
then, a training set is formed by the image with augmented data;
step 2: model training
Setting training parameters by adopting a neural network model, and training the neural network model by utilizing a training set;
the trained neural network model can detect vehicles and roads in the input image;
and step 3: measuring the vehicle speed;
step 3-1: intercepting an image every N frames of videos collected by the aerial robot, inputting the images into the neural network model trained in the step 2, and detecting vehicles and roads in the images;
step 3-2: calculating the distance between two adjacent lane lines in the image, and calculating the average value d of the distances between the lane lines in the imagespic;
Step 3-3: obtaining the actual distance d between two adjacent lane linesactual;
Step 3-4: calculating the ratio of the average distance of the lane line in the image to the actual distance of the lane line
Step 3-5: calculating the interval time of every N frames of video
f is the video frame rate;
step 3-6: calculating the moving distance s of the vehicle on the adjacent N frames of images
picAnd calculating the actual distance of the vehicle moving in the interval time delta t
Step 3-7: calculating a speed of movement of a vehicle
And 4, step 4: and 3, measuring the speed of the vehicle in the video by adopting the step 3, and marking the vehicle as red to warn an operator when the speed of the vehicle exceeds a set value.
Preferably, the data augmentation method includes translation, rotation, flipping, cropping, scaling, adjusting hue, adjusting lightness, adjusting saturation, perspective transformation, image fusion, and image occlusion.
Preferably, the neural network model is YOLOv 5.
Preferably, N ═ 10.
The invention has the following beneficial effects:
1. the method overcomes the defects of low flexibility, high manufacturing cost, difficult installation and the like of the traditional fixed speed measuring equipment, and gets rid of the high requirement of the existing speed measuring scheme of the aerial vehicle on the flight attitude of the aerial robot.
2. The method only depends on the vision of the aerial robot, has strong maneuverability and low requirements on flight attitude and flyers, can monitor the speed of the ground vehicle in real time, and is convenient to use.
Detailed Description
The invention is further illustrated by the following examples.
The invention provides a method for measuring the speed of a ground vehicle by using an aerial robot to visually measure the speed of the ground vehicle, which can detect the vehicle and a lane line through a neural network, calculate the distance on a lane line graph and calculate the ratio of the distance on the graph to the actual distance to solve the actual movement distance of the ground vehicle, and finally calculate the speed of the ground vehicle and track and monitor the speed of the ground vehicle.
A vehicle speed measuring method based on aerial robot mobile vision comprises the following steps:
step 1: collecting data;
acquiring images of different vehicles and roads by using an aerial robot under mobile vision, and labeling the vehicles and the roads in the acquired images;
performing data augmentation on the marked image;
then, a training set is formed by the image with augmented data;
step 2: model training
Setting training parameters by adopting a neural network model, and training the neural network model by utilizing a training set; the method comprises the following specific steps:
(1) setting a neural network; initializing model weight, an optimization function and a loss function, and setting initial training parameters of a neural network;
(2) pre-training a neural network; and the classification performance of the neural network on the images is improved by utilizing an open-source image data set pre-training model.
(3) Importing a training set; importing the data subjected to data preprocessing into a neural network model;
(4) training a neural network; optimizing and updating the network weight through a loss function and an optimization function by using the marked training data, updating the model parameters, and continuously iterating;
(5) modifying the hyper-parameters and optimizing the model; and (3) modifying the hyper-parameters, adjusting the threshold value, continuously comparing the experimental results and selecting the neural network model with the best effect.
The trained neural network model can detect vehicles and roads in the input image;
and step 3: measuring the vehicle speed;
step 3-1: intercepting an image every N frames of videos collected by the aerial robot, inputting the images into the neural network model trained in the step 2, and detecting vehicles and roads in the images;
step 3-2: calculating the distance between two adjacent lane lines in the image, and calculating the average value d of the distances between the lane lines in the imagespic;
Step 3-3: obtaining the actual distance d between two adjacent lane linesactual;
Step 3-4: calculating the ratio of the average distance of the lane line in the image to the actual distance of the lane line
Step 3-5: calculating the interval time of every N frames of video
f is the video frame rate;
step 3-6: calculating the moving distance s of the vehicle on the adjacent N frames of images
picAnd calculating the actual distance of the vehicle moving in the interval time delta t
Step 3-7: calculating a speed of movement of a vehicle
And 4, step 4: and 3, measuring the speed of the vehicle in the video by adopting the step 3, and marking the vehicle as red to warn an operator when the speed of the vehicle exceeds a set value.
The specific embodiment is as follows:
1. collecting data
The aerial robot is used for shooting road surface images of expressways, provincial roads and the like, and a large amount of image data of various roads based on the moving vision of the aerial robot is obtained. And the labelme software is used for carrying out frame selection and marking on the vehicles and the lane lines in the image.
2. Data pre-processing
Because vehicle occlusion, vehicle lane changing and lane line pressing, inconsistency of flight direction and road direction of the aerial robot, illumination on a shot picture and the like can occur frequently, data amplification needs to be carried out on data before the neural network training is started in order to ensure the accuracy and precision of model training. The method adopts the means of translation, rotation, turnover, cutting, scaling, adjustment of hue, brightness, saturation, perspective transformation, image fusion and image shielding, and combines the means to process the image so as to simulate the road conditions under different conditions and increase data. And forming a training set after data augmentation.
3. Model training
The method of YOLOv5 was used to train and detect vehicles and lane markings. And setting the detection threshold value to be 0.5, namely, the detected frame and the real frame IOU are greater than 0.5, namely, the vehicle and the lane line can be detected. During training, the sizes of all training image data are adjusted to 448 multiplied by 448 to pre-train the features to extract the neural network, then the size of the input training image data is increased to 640 multiplied by 640, the marked training data is utilized, network weights are optimized and updated through a loss function and an optimization function, model parameters are updated, iteration is carried out continuously, and then the neural network is subjected to fine tuning by using a detection data set. Continuously modifying the hyper-parameters, optimizing the model, adjusting the threshold value by modifying the hyper-parameters, continuously comparing the experimental results and selecting the neural network model with the best effect.
4. Vehicle speed measurement
(1) The trained neural network model can detect the lane lines and vehicles in the road image shot by the aerial robot.
(2) Intercepting one image frame every 10 frames to obtain a video imageCalculating the distance on the image of the adjacent lane lines and calculating the average value d of the distance on the imagepic。
(3) Looking up the national standard or provincial standard requirements of the roads of the flying ground of the aerial robot to obtain the actual distance d between the adjacent lane linesactual。
(4) Calculating the ratio of the distance on the lane line graph to the actual distance
(5) Reading the video frame rate information to obtain the frame rate f of video shooting, and calculating the interval time of every 10 frames of the video
(6) Calculating the distance s of the vehicle moving on the adjacent 10 frames of images
picDetermining the actual distance of the vehicle moving within Δ t
(7) Calculating a speed of movement of a vehicle
5. By using the method, the position of the vehicle is tracked in the video frame, the motion track of the vehicle in the picture within a certain period of time is obtained, the vehicle is tracked and monitored, and when the speed of the vehicle in the picture exceeds a set value, the vehicle is marked red to warn an operator.