[go: up one dir, main page]

CN111160224B - A high-speed rail catenary foreign object detection system and method based on FPGA and horizon segmentation - Google Patents

A high-speed rail catenary foreign object detection system and method based on FPGA and horizon segmentation Download PDF

Info

Publication number
CN111160224B
CN111160224B CN201911369862.0A CN201911369862A CN111160224B CN 111160224 B CN111160224 B CN 111160224B CN 201911369862 A CN201911369862 A CN 201911369862A CN 111160224 B CN111160224 B CN 111160224B
Authority
CN
China
Prior art keywords
horizon
image
segmentation
target detection
module
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
CN201911369862.0A
Other languages
Chinese (zh)
Other versions
CN111160224A (en
Inventor
陈积明
吕嘉宜
贺诗波
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Zhejiang University ZJU
Original Assignee
Zhejiang University ZJU
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Zhejiang University ZJU filed Critical Zhejiang University ZJU
Priority to CN201911369862.0A priority Critical patent/CN111160224B/en
Publication of CN111160224A publication Critical patent/CN111160224A/en
Application granted granted Critical
Publication of CN111160224B publication Critical patent/CN111160224B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V20/00—Scenes; Scene-specific elements
    • G06V20/40—Scenes; Scene-specific elements in video content
    • G06V20/41—Higher-level, semantic clustering, classification or understanding of video scenes, e.g. detection, labelling or Markovian modelling of sport events or news items
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
    • G06F18/214—Generating training patterns; Bootstrap methods, e.g. bagging or boosting
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/24—Classification techniques
    • G06F18/241—Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/20—Image preprocessing
    • G06V10/26—Segmentation of patterns in the image field; Cutting or merging of image elements to establish the pattern region, e.g. clustering-based techniques; Detection of occlusion
    • G06V10/267—Segmentation of patterns in the image field; Cutting or merging of image elements to establish the pattern region, e.g. clustering-based techniques; Detection of occlusion by performing operations on regions, e.g. growing, shrinking or watersheds

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • Data Mining & Analysis (AREA)
  • General Physics & Mathematics (AREA)
  • Bioinformatics & Cheminformatics (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Evolutionary Biology (AREA)
  • Evolutionary Computation (AREA)
  • Bioinformatics & Computational Biology (AREA)
  • General Engineering & Computer Science (AREA)
  • Artificial Intelligence (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Multimedia (AREA)
  • Computational Linguistics (AREA)
  • Software Systems (AREA)
  • Image Analysis (AREA)

Abstract

本发明公开了一种基于FPGA和地平线分割的高铁接触网异物检测系统及方法，该系统包括视频采集模块、图像存储模块、地平线分割模块和目标检测模块。视频采集模块用于随车实时采集高速铁路接触网监控视频；地平线分割模块使用改进的最大类间方差法和智能迭代算法实现快速自适应地平线分割，将地平线以上部分作为检测区域输入到目标检测模块；目标检测模块利用目标检测神经网络在地平线分割模块传入的图片中检测出可能存在的异物。这些模块运行于随车FPGA终端上以实现加速计算。本发明能够快速准确地检测出接触网上的异物，符合铁路部门实际需要，能够有效保障高速铁路运行安全性。

Figure 201911369862

The invention discloses a high-speed rail catenary foreign object detection system and method based on FPGA and horizon segmentation. The system includes a video acquisition module, an image storage module, a horizon segmentation module and a target detection module. The video acquisition module is used to collect real-time monitoring video of the high-speed railway catenary on the vehicle; the horizon segmentation module uses the improved maximum inter-class variance method and intelligent iterative algorithm to achieve fast adaptive horizon segmentation, and the part above the horizon is input to the target detection module as the detection area ; The target detection module uses the target detection neural network to detect possible foreign objects in the pictures passed in by the horizon segmentation module. These modules run on the on-board FPGA terminal for accelerated computing. The invention can quickly and accurately detect the foreign objects on the contact wire, meets the actual needs of the railway department, and can effectively guarantee the operation safety of the high-speed railway.

Figure 201911369862

Description

High-speed rail contact net foreign matter detection system and method based on FPGA and horizon line segmentation
Technical Field
The invention relates to the field of high-speed railway contact net foreign body hanging detection, in particular to an image processing method for FPGA (field programmable gate array) accelerated operation, deep learning and the like.
Background
The contact net of the high-speed railway is used as a device for supplying power to the electric locomotive, and if foreign matters (such as plastic bags, kites and other light drifts) are attached to the contact net, the power supply to the train can be influenced, so that the normal operation of the high-speed railway is threatened. At present, China mainly collects and stores contact network video images along a railway through a vehicle-mounted camera such as a motor train unit driver control information analysis system, but the contact network video images do not have the function of detecting and identifying foreign matters, so that detection personnel are required to manually observe video image data to check whether the foreign matters are attached to the contact network, the detection mode is time-consuming and labor-consuming, and the real-time performance is low.
Nowadays, image processing technology based on neural network is developed at a high speed, and has been well applied in various fields such as road video monitoring and analysis, automatic driving vehicles and the like. If can utilize advanced image processing technique to carry out the automatic analysis of video content to the contact net surveillance video to whether detect the foreign matter invasion contact net, just so can discover the foreign matter that exists on the contact net fast, ensure high-speed railway's safe operation.
Compared with target detection methods in other scenes, the method for detecting the foreign matters in the high-speed rail contact network is characterized in that the ground images are complex and have excessive interference information and are not detection target areas. Therefore, the method is characterized in that a horizon segmentation step is added before the target detection algorithm is applied, the video is automatically segmented into a sky part and a ground part according to the horizon after a monitoring video is trained by an improved maximum inter-class variance method (OTSU algorithm), and the sky part is extracted to carry out foreign matter detection by using a target detection neural network based on deep learning. Meanwhile, in consideration of the requirement of detection on real-time performance, the method makes full use of the characteristics of FPGA high-speed data processing capacity, hardware programming design and the like, and the algorithm is deployed on the FPGA, so that the foreign matter net hanging condition of the high-speed railway contact net is effectively monitored in real time.
Disclosure of Invention
In order to overcome the defect that time and labor are consumed for manually observing and monitoring videos to inspect foreign matters, the invention provides a high-speed rail contact net foreign matter detection system and method based on FPGA and horizon segmentation. The system uses an improved maximum inter-class variance method and a neural network algorithm to automatically analyze video image data, automatically divides a video into a sky part and a ground part according to a horizon line, quickly identifies foreign matters entering and exiting a contact net by using a target detection neural network in the sky part, and deploys the foreign matters on a vehicle-mounted FPGA terminal to increase real-time performance, improve the detection efficiency of the foreign matters invading the contact net and ensure the safe operation of a high-speed railway.
The technical scheme adopted by the invention for solving the technical problems is as follows: a high-speed rail contact net foreign matter detection system based on FPGA and horizon segmentation, this system includes following part:
(1) the video acquisition module: using FPGA as core, adopting SAA7113H video decoding chip, passing through I2The C bus protocol carries out initialization configuration on a CCD camera, the CCD camera is externally connected with a camera simulating a PAL/NTAL system, the camera is deployed in a train cab of the motor train unit to shoot towards the front, a running monitoring video containing a contact net is collected in real time, and format compression coding is carried out on a video signal;
(2) an image storage module: one path of the collected video data is written into DDR2 SDRAM through an FIFO buffer for storage, and the video data is read out through the FIFO buffer; the other path is encoded by an SAA7121 video encoding chip and then is output to a monitor to display a real-time image;
(3) horizon segmentation module: video data are obtained from DDR2 SDRAM, self-adaptive horizon segmentation is realized through an improved maximum inter-class variance method (OTSU algorithm) and an intelligent iterative algorithm, and a sky partial image above the horizon is taken and transmitted to a target detection module for further processing;
the improved maximum inter-class variance method specifically comprises the following steps: calculating a pixel point when the inter-class variance of a blue component (B value) is maximum in the range of each column of pixels (0.4N,0.8N) of an RGB image with the input size of M multiplied by N (M, N is the number of pixels in the horizontal and column directions of the RGB image respectively) as a horizon position point; obtaining a complete horizon position by traversing the whole image, taking a horizontal line so that 85% of horizon points are below the horizontal line, and taking the horizontal line as a final horizon estimation position; cutting partial images above a reserved horizon line as sky partial images, and inputting the sky partial images into a target detection neural network for training;
in the intelligent iterative algorithm, the updating frequency of the horizon position is controlled, and the image frame number interval of the horizon position updating is determined through an exponential function related to the change rate of the horizon position, so that the computing resources are saved, and the method specifically comprises the following steps:
initially, the horizon position is calculated every 1 second, i.e. v for recording frequency (in fps)The k frame picture calculates the horizon position h1Then, the k + v frame picture calculates the horizon position h2(ii) a Then, the rate of change of the position of the horizon is calculated twice
Figure BDA0002339385400000021
Figure BDA0002339385400000022
According to the rate of change
Figure BDA0002339385400000023
Image frame number interval u for automatic change horizon position update:
Figure BDA0002339385400000024
i.e. the local horizon change rate is
Figure BDA0002339385400000025
Then, the k + v frame calculates the horizon position h2Then, continuously calculating the position of the horizon in the k + v + u frame; iteratively updating the horizon position by the algorithm;
(4) a target detection module: and inputting the sky partial image transmitted into the module by the horizon segmentation module into a pre-trained target detection neural network for target detection, detecting possible foreign matters such as plastic bags and the like in the image, marking and early warning according to a detection result.
Further, in the adaptive horizon segmentation, the original image is compressed to 1/8 in length and width and then input to an algorithm for calculation, and the calculated corresponding estimated horizon position is mapped to the original image for segmentation, thereby speeding up calculation.
Further, the target detection neural network takes VGG-16 pre-training neural network weight as initial weight, and trains the high-speed rail contact network foreign matter data set marked in the VOC2012 format, so as to train out network weight suitable for the scene; the process specifically comprises the following steps: presetting 4 prior frames with different length-width ratios, wherein the specific size of the prior frame is determined by the size of the characteristic diagram; during training, matching each real target (ground route) in the picture with a prior frame with the largest IOU formed by the real targets; during classification, calculating the background as one of classes, namely for the detection targets of c classes, predicting c +1 confidence degrees by an algorithm, wherein the class with the highest confidence degree is a prediction class; during prediction, for each prediction frame, determining a target category according to the maximum category confidence, and filtering out the prediction frames of the background and the prediction frames with too low confidence (less than 0.5); and decoding the remaining prediction frames, obtaining the real position parameters of the prediction frames according to the prior frames, and reserving a plurality of prediction frames with the maximum confidence coefficient.
The invention also provides a high-speed rail contact net foreign matter detection method based on FPGA and horizon segmentation, which comprises the following steps:
(1) the FPGA equipment and the CCD image sensor are arranged in the locomotive of the motor train unit, and the visual angle is shot towards the front so as to be shot into a contact net structure, and the real-time acquisition is carried out on the train;
(2) the collected video data is continuously written into DDR2 SDRAM through FIFO buffer for storage, and then read out through FIFO buffer, and displayed in monitor through decoder;
(3) estimating the horizon position by traversing each row of pixels in the image acquired in the step (2) by adopting an improved maximum inter-class variance method, thereby performing self-adaptive horizon segmentation on the input image and segmenting a sky partial image; meanwhile, the updating frequency of the horizon position is controlled through an intelligent iterative algorithm, so that computing resources are saved;
(4) and inputting the segmented sky partial image into a pre-trained target detection neural network for target detection, detecting foreign matters such as plastic bags and the like possibly existing in the image, marking and early warning according to a detection result.
Further, in the step (4), the specific steps of detecting and identifying the foreign object by the target detection neural network are as follows:
(1) constructing a positive and negative sample data set according to the existing catenary monitoring video;
(2) marking a bounding box of a positive sample (foreign body part) in the data set according to a VOC2012 target detection task format;
(3) inputting the data set into a target detection neural network for training, adjusting the data set according to a training result, and training until a network weight with excellent detection performance indexes is obtained;
(4) and according to the actual operation detection result, continuously updating and training the data set, and optimizing the network weight, thereby improving the detection performance.
The invention has the beneficial effects that:
(1) the invention divides the horizon in advance through a fast self-adaptive algorithm, and only occupies small computing resources to extract the key monitoring area of the sky part, thereby reducing the detection range, eliminating the ground information interference and simultaneously increasing the foreign matter detection accuracy.
(2) The invention adopts the image processing neural network based on deep learning to accurately and efficiently detect the foreign matters in the contact network.
(3) The invention fully utilizes the high-speed and large data processing capacity of the FPGA to transfer the computational power of image acquisition and processing from the remote end of the server to the on-site edge end, thereby enhancing the effectiveness of railway monitoring.
(4) By applying the image processing technology based on the neural network, the invention can enable a computer to replace part of manual work, improve the detection efficiency of the contact net invading foreign matter, ensure the safe operation of the high-speed railway, simultaneously liberate manpower and improve the office automation level.
Drawings
FIG. 1 is a schematic diagram of the system of the present invention;
FIG. 2 is a schematic flow chart of the method of the present invention;
FIG. 3 is a flow chart of an adaptive horizon segmentation algorithm;
FIG. 4 is a schematic diagram of the effect of dividing the horizon;
fig. 5 is a schematic diagram of the target detection effect.
Detailed Description
The invention is described in further detail below with reference to the figures and specific examples.
As shown in fig. 1, the high-speed rail catenary foreign matter detection system based on FPGA and horizon segmentation provided by the invention comprises the following parts:
(1) the video acquisition module: the method comprises the steps of taking an FPGA as a core, adopting an SAA7113H video decoding chip, carrying out initialization configuration on a CCD camera through an I2C bus protocol, externally connecting a camera simulating a PAL/NTAL system, arranging the camera in a train cab of the motor train unit to shoot towards the front, collecting a driving monitoring video containing a contact net in real time, and carrying out format compression coding on a video signal.
Further, the video capture module comprises the following parts: the adopted FPGA chip is EP4CE617C8 of Cyclone IV series developed by Alter company; the adopted video decoding chip is SAA7113H, the video decoding chip completes configuration and initialization processes through an I2C bus under the control of FPGA, and 8-bit video data compatible with CCIR656 is output.
(2) An image storage module: one path of the collected video data is written into DDR2 SDRAM through an FIFO buffer for storage, and the video data is read out through the FIFO buffer; and the other path is encoded in an SAA7121 video encoding chip and then output to a monitor to display a real-time image.
Further, the image storage module includes the following parts: DDR 2800 series SDRAM of 8GB memory is used as memory; a FIFO is used as a data buffer.
(3) Horizon segmentation module: video data are obtained from DDR2 SDRAM, self-adaptive horizon segmentation is realized through an improved maximum inter-class variance method and an intelligent iterative algorithm, and a sky partial image above the horizon is taken and transmitted to a target detection module for further processing.
(4) A target detection module: and inputting the sky partial image transmitted into the module by the horizon segmentation module into a pre-trained target detection neural network for target detection calculation, detecting possible foreign matters such as plastic bags and the like in the image, and marking and early warning according to a detection result.
FIG. 2 is a schematic flow chart of the method of the present invention. The method comprises the steps of estimating the position of a horizon through a self-adaptive horizon segmentation algorithm based on an improved maximum inter-class variance method and an intelligent iterative algorithm for a frame of image obtained from a CCD monitoring video, and cutting the position to obtain a partial image above the horizon (namely a sky partial image). After the horizontal line positions are initially calculated twice, whether the horizontal line positions of the current frame image need to be calculated is determined through an intelligent iterative algorithm, and the image which does not need to be calculated is directly cut after being segmented at the calculated positions of the previous time. And then, performing Beijing target detection calculation in the sky partial image through a target detection neural network algorithm of a trained model: if the foreign body can be detected, marking a foreign body boundary frame, carrying out corresponding early warning prompt, and then continuously detecting the next image; otherwise, the next image is directly detected.
FIG. 3 is a flow chart of an adaptive horizon segmentation algorithm. The self-adaptive horizon calculation based on the improved maximum inter-class variance method is specifically calculated as follows:
for an input image I (x, y) expressed by an RGB format with the size of M multiplied by N (pixels), the vertex at the upper left corner of the image is I (x,0), the ith row of pixels is a matrix of 1 multiplied by N, wherein the sky ground segmentation threshold is marked as T, and the proportion of pixel points belonging to the sky in the whole matrix is marked as omega0The B value (blue component value) in RGB is averaged to μ0(ii) a The proportion of the pixels belonging to the ground occupying the whole matrix is recorded as omega1The average B value is mu1(ii) a The total average B value for an entire column of pixels is denoted as μ and the inter-class variance is denoted as g. In a 1 XN matrix formed by the ith row of pixels, the number of pixels with B value smaller than the threshold value T is counted as N0The number of pixels with B value greater than threshold T is recorded as N1Then, there are:
Figure BDA0002339385400000051
Figure BDA0002339385400000052
μ＝ω0μ0+ω1μ1 (3)
g＝ω0(μ0-μ)2+ω1(μ1-μ)2 (4)
substituting formula (3) for formula (4) to obtain the equivalent formula:
g＝ω0ω1(μ0-μ1)2 (5)
g obtained by the formula (5) is the inter-class variance.
According to experience, the horizon is generally positioned at the middle lower part of a picture, so that the g is maximum only by traversing the (0.4N,0.8N) interval of a 1 xN matrix formed by the ith row of pixels, namely the maximum between-class variance; the pixel position I (I, j) at this time is recorded, which is the pixel position of the horizon in the row of pixels. Traversing the whole image, recording the pixel position of the horizon in each row of pixels, recording j into a matrix, and finally forming an M multiplied by 1 matrix which is recorded as TH.
Because the input image of the neural network of the next module is rectangular, according to experience, a value H is taken as a final horizon position in the TH matrix, so that the H is smaller than 85% of the value in the TH matrix (namely, is more than 85% of horizon points), the horizon position can be well expressed, and noise interference is reduced. And (3) cutting and reserving pixels with y < H in the input image I (x, y) of M multiplied by N, namely acquiring an image above the horizon, namely a sky partial image.
Meanwhile, in order to accelerate the calculation, in the actual operation, the length and width dimensions of the original image of M multiplied by N are compressed to 1/8, then the input module performs calculation, and the calculated corresponding estimated position of the horizon line is mapped to the original image for segmentation, so that the calculation is accelerated; from experimental comparisons, such compression calculations result in negligible error.
The method for controlling the updating frequency of the horizon estimation position through the intelligent iterative algorithm specifically comprises the following steps:
initially, the horizon position is calculated every 1 second, namely for a video with a recording frequency v (the unit is fps), the horizon position h is calculated for the k frame picture1Then, the k + v frame picture calculates the horizon position h2(ii) a Then, the rate of change of the position of the horizon is calculated twice
Figure BDA0002339385400000061
Figure BDA0002339385400000062
According to the rate of change
Figure BDA0002339385400000063
Image frame number interval u for automatic change horizon position update:
Figure BDA0002339385400000064
i.e. the local horizon change rate is
Figure BDA0002339385400000065
Then, the k + v frame calculates the horizon position h2Then, continuously calculating the position of the horizon in the k + v + u frame; iteratively updating the horizon position by the algorithm; and the other frame images are directly segmented and cut by using the previously calculated horizon position.
FIG. 4 is a diagram illustrating the effect of dividing the horizon. Fig. 5 is a schematic diagram of the target detection effect. The detected target is marked out by a bounding box, and characters represent the identification type and the confidence of the target. The invention can quickly and accurately detect the foreign matters on the contact net, meets the actual requirements of railway departments, and can effectively ensure the operation safety of the high-speed railway.
The foregoing is only a preferred embodiment of the present invention, and although the present invention has been disclosed in the preferred embodiments, it is not intended to limit the present invention. Those skilled in the art can make numerous possible variations and modifications to the present teachings, or modify equivalent embodiments to equivalent variations, without departing from the scope of the present teachings, using the methods and techniques disclosed above. Therefore, any simple modification, equivalent change and modification made to the above embodiments according to the technical essence of the present invention are still within the scope of the protection of the technical solution of the present invention, unless the contents of the technical solution of the present invention are departed.

Claims (5)

1.一种基于FPGA和地平线分割的高铁接触网异物检测系统，其特征在于，该系统包括以下部分：1. a high-speed rail catenary foreign body detection system based on FPGA and horizon segmentation, is characterized in that, this system comprises the following parts: (1)视频采集模块：以FPGA为核心，采用SAA7113H视频解码芯片，通过I2C总线协议对CCD摄像头进行初始化配置，外接模拟PAL/NTAL制式的摄像头，摄像头部署于动车组列车驾驶室朝前方拍摄，实时采集包含接触网的行车监控视频，并将视频信号进行格式压缩编码；(1) Video acquisition module: With FPGA as the core, SAA7113H video decoding chip is used, the CCD camera is initialized and configured through the I 2 C bus protocol, and an analog PAL/NTAL camera is connected externally. The camera is deployed in the EMU train cab facing forward Shooting, real-time collection of driving surveillance video including catenary, and format compression encoding of video signal; (2)图像存储模块：采集的视频数据一路通过FIFO缓存器写入到DDR2 SDRAM中存储，再经过FIFO缓存器读出视频数据；另一路通过SAA7121视频编码芯片进行编码处理后输出到监视器上显示实时图像；(2) Image storage module: the collected video data is written into DDR2 SDRAM through the FIFO buffer all the way for storage, and then the video data is read out through the FIFO buffer; the other way is encoded through the SAA7121 video encoding chip and then output to the monitor. display live images; (3)地平线分割模块：从DDR2 SDRAM中获取视频数据，通过改进的最大类间方差法和智能迭代算法对其实现自适应地平线分割，取地平线以上的天空部分图像传送至目标检测模块；(3) Horizon segmentation module: Obtain video data from DDR2 SDRAM, realize adaptive horizon segmentation through the improved maximum inter-class variance method and intelligent iterative algorithm, and transmit the image of the sky part above the horizon to the target detection module; 所述改进的最大类间方差法具体为：通过计算输入尺寸为M×N的RGB图像的每列像素(0.4N,0.8N)范围中，蓝色分量(B值)类间方差最大时的像素点为地平线位置点；通过遍历整个图像得到完整的地平线位置，取一条水平线使得85％的地平线位置点在该水平线以下，将该水平线作为最终地平线估计位置；裁剪保留地平线以上部分图像作为天空部分图像，输入到目标检测神经网络中进行训练；The improved maximum inter-class variance method is specifically: by calculating the maximum inter-class variance of the blue component (B value) in the range of each column of pixels (0.4N, 0.8N) of an RGB image with an input size of M×N. The pixel point is the horizon position point; the complete horizon position is obtained by traversing the entire image, and a horizontal line is taken so that 85% of the horizon position points are below the horizontal line, and the horizontal line is used as the final horizon estimate position; crop and retain the part of the image above the horizon as the sky part The image is input into the target detection neural network for training; 所述智能迭代算法具体为：初始设定每1秒计算一次地平线位置，即对录制频率为v的视频，第k帧图片计算了地平线位置h1后，第k+v帧图片再计算地平线位置h2；然后，计算两次地平线位置变化率
Figure FDA0003508625100000011
The intelligent iterative algorithm is specifically as follows: the initial setting is to calculate the horizon position every 1 second, that is, for a video with a recording frequency of v, after the kth frame picture has calculated the horizon position h1 , the k+vth frame picture is calculated again the horizon position. h 2 ; then, calculate the rate of change of the horizon position twice
Figure FDA0003508625100000011
Figure FDA0003508625100000012
Figure FDA0003508625100000012
 根据变化率
Figure FDA0003508625100000013
自动变化地平线位置更新的图像帧数间隔u：According to the rate of change
Figure FDA0003508625100000013
The interval u of the image frame number for automatic change of horizon position update:
Figure FDA0003508625100000014
Figure FDA0003508625100000014
 即当地平线变化率为
Figure FDA0003508625100000015
时，k+v帧计算出地平线位置h2后，第k+v+u帧继续计算地平线位置；并以此算法迭代更新地平线位置；That is, when the horizon change rate is
Figure FDA0003508625100000015
When the horizon position h 2 is calculated in the k+v frame, the k+v+uth frame continues to calculate the horizon position; and this algorithm is used to iteratively update the horizon position; (4)目标检测模块：将地平线分割模块输出的天空部分图像输入至预先训练的目标检测神经网络中进行目标检测，检测出图像中可能存在的异物，根据检测结果作出标注并预警。(4) Target detection module: Input the image of the sky part output by the horizon segmentation module into the pre-trained target detection neural network for target detection, detect possible foreign objects in the image, and make annotations and warnings according to the detection results. 2.根据权利要求1所述的一种基于FPGA和地平线分割的高铁接触网异物检测系统，其特征在于，所述地平线分割模块中，将原图的长宽尺寸均压缩至1/8大小后再输入到改进的最大类间方差法中进行计算，计算得到的相应的地平线估计位置映射至原图进行分割，由此加速计算。2. a kind of high-speed rail catenary foreign body detection system based on FPGA and horizon segmentation according to claim 1, is characterized in that, in described horizon segmentation module, after the length and width dimensions of the original image are all compressed to 1/8 size Then input it into the improved maximum inter-class variance method for calculation, and the corresponding estimated horizon position obtained by calculation is mapped to the original image for segmentation, thereby speeding up the calculation. 3.根据权利要求1所述的一种基于FPGA和地平线分割的高铁接触网异物检测系统，其特征在于，所述的目标检测神经网络以VGG-16预训练神经网络权重作为初始权重，对以VOC2012格式标注好的高铁接触网异物数据集进行训练，从而训练出适合本场景的网络权重；具体为：预先设置4种长宽比不同的先验框，先验框具体尺寸由特征图尺寸确定；训练时，对于图片中每个真实目标，与其形成的IOU最大的先验框进行匹配；分类时，将背景算作类别的一种，即对c个类别的检测目标，预测c+1个置信度，置信度最高的类别为预测类别；预测时，对于每个预测框，根据最大的类别置信度确定目标类别，并过滤掉背景的预测框和置信度过低的预测框；对于留下的预测框进行解码，根据先验框得到其真实的位置参数，并保留置信度最大的若干预测框。3. a kind of high-speed rail catenary foreign body detection system based on FPGA and horizon segmentation according to claim 1, is characterized in that, described target detection neural network takes VGG-16 pre-training neural network weight as initial weight, to The high-speed rail catenary foreign body data set marked in VOC2012 format is used for training, so as to train the network weights suitable for this scene. Specifically, 4 kinds of a priori frames with different aspect ratios are preset, and the specific size of the a priori frame is determined by the size of the feature map. ; During training, for each real target in the picture, match the a priori frame with the largest IOU formed by it; when classifying, the background is counted as one of the categories, that is, for c categories of detection targets, predict c + 1 Confidence, the category with the highest confidence is the prediction category; during prediction, for each prediction box, the target category is determined according to the maximum category confidence, and the background prediction box and the prediction box with too low confidence are filtered out; Decoding the prediction frame, obtain its real position parameters according to the prior frame, and retain several prediction frames with the highest confidence. 4.一种基于权利要求1-3任一项所述的FPGA和地平线分割的高铁接触网异物系统的检测方法，其特征在于，该方法包括以下步骤：4. A detection method based on the high-speed rail catenary foreign body system of the FPGA and the horizon segmentation described in any one of claims 1-3, is characterized in that, this method comprises the following steps: (1)FPGA设备与CCD图像传感器部署于动车组列车头中，视角朝前方拍摄以摄入接触网结构，随车实时采集；(1) FPGA equipment and CCD image sensor are deployed in the locomotive of the EMU, and the viewing angle is taken forward to capture the catenary structure, and real-time collection is carried out with the vehicle; (2)采集的视频数据通过FIFO缓存器不断写入到DDR2 SDRAM中存储，再经过FIFO缓存器读出视频数据，同时通过解码器显示在监视器中；(2) The collected video data is continuously written into the DDR2 SDRAM through the FIFO buffer for storage, and then reads out the video data through the FIFO buffer, and is displayed in the monitor through the decoder at the same time; (3)采用改进的最大类间方差法，通过遍历步骤(2)采集的图像中每一列像素估计地平线位置，从而对输入图像进行自适应地平线分割，分割出天空部分图像；同时通过智能迭代算法控制地平线位置更新频率，节省计算资源；(3) Using the improved maximum inter-class variance method, the horizon position is estimated by traversing each column of pixels in the image collected in step (2), so as to perform adaptive horizon segmentation on the input image and segment the sky part of the image; at the same time, through the intelligent iterative algorithm Control the update frequency of the horizon position to save computing resources; (4)将分割出的天空部分图像输入至预先训练的目标检测神经网络中进行目标检测，检测出图像中可能存在的异物，根据检测结果作出标注并预警。(4) Input the segmented sky image into the pre-trained target detection neural network for target detection, detect possible foreign objects in the image, and make labels and warnings according to the detection results. 5.根据权利要求4所述的一种基于FPGA和地平线分割的高铁接触网异物系统的检测方法，其特征在于，所述步骤(4)中，所述目标检测神经网络对异物进行检测并识别的具体步骤为：5. a kind of detection method of high-speed rail catenary foreign body system based on FPGA and horizon segmentation according to claim 4, is characterized in that, in described step (4), described target detection neural network detects and identifies foreign body The specific steps are: (1)根据现有接触网监控视频，构建正负样本数据集；(1) According to the existing catenary surveillance video, construct a positive and negative sample data set; (2)对数据集中正样本(异物部分)根据VOC2012目标检测任务格式进行边界框标注；(2) Label the positive samples (foreign objects) in the dataset according to the VOC2012 target detection task format; (3)将数据集输入目标检测神经网络中进行训练，根据训练结果调整数据集，训练直至获得检测性能指标优秀的网络权重；(3) Input the data set into the target detection neural network for training, adjust the data set according to the training results, and train until the network weights with excellent detection performance indicators are obtained; (4)根据实际运行检测结果，不断更新数据集并训练，优化网络权重，从而提升检测性能。(4) According to the actual operation detection results, the data set is continuously updated and trained, and the network weight is optimized to improve the detection performance.
CN201911369862.0A 2019-12-26 2019-12-26 A high-speed rail catenary foreign object detection system and method based on FPGA and horizon segmentation Active CN111160224B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN201911369862.0A CN111160224B (en) 2019-12-26 2019-12-26 A high-speed rail catenary foreign object detection system and method based on FPGA and horizon segmentation

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN201911369862.0A CN111160224B (en) 2019-12-26 2019-12-26 A high-speed rail catenary foreign object detection system and method based on FPGA and horizon segmentation

Publications (2)

Publication Number Publication Date
CN111160224A CN111160224A (en) 2020-05-15
CN111160224B true CN111160224B (en) 2022-04-05

Family

ID=70558316

Family Applications (1)

Application Number Title Priority Date Filing Date
CN201911369862.0A Active CN111160224B (en) 2019-12-26 2019-12-26 A high-speed rail catenary foreign object detection system and method based on FPGA and horizon segmentation

Country Status (1)

Country Link
CN (1) CN111160224B (en)

Families Citing this family (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN112995505B (en) * 2021-02-09 2022-06-17 西南科技大学 Image processing method, device and storage medium
CN113917446B (en) * 2021-08-31 2025-11-21 际络科技(上海)有限公司 Road guardrail prediction method, device and storage medium
CN115546738A (en) * 2022-08-31 2022-12-30 成都众乐泰科技有限公司 A method for detecting foreign objects on railway tracks
CN115690730B (en) * 2022-09-22 2025-09-26 浙江大学 High-speed railway contact network foreign body detection method and system based on single classification and anomaly generation

Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN109697424A (en) * 2018-12-19 2019-04-30 浙江大学 A kind of high-speed railway impurity intrusion detection device and method based on FPGA and deep learning
CN109753893A (en) * 2018-12-20 2019-05-14 广州航天海特系统工程有限公司 Video detecting method, system, computer equipment and storage medium along track

Family Cites Families (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US10282622B2 (en) * 2016-12-09 2019-05-07 Hitachi Kokusai Electric Inc. Marine intrusion detection system and method

Patent Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN109697424A (en) * 2018-12-19 2019-04-30 浙江大学 A kind of high-speed railway impurity intrusion detection device and method based on FPGA and deep learning
CN109753893A (en) * 2018-12-20 2019-05-14 广州航天海特系统工程有限公司 Video detecting method, system, computer equipment and storage medium along track

Non-Patent Citations (3)

* Cited by examiner, † Cited by third party
Title
一种基于类间方差的地平线检测算法;程序等;《航空学报》;20101025(第10期);全文 *
区域协方差与中值校正融合的天际线检测算法研究;涂兵等;《计算机科学》;20170315(第03期);全文 *
基于残差回归网络的复杂背景下海界线检测;邱艺铭等;《舰船电子工程》;20180820(第08期);全文 *

Also Published As

Publication number Publication date
CN111160224A (en) 2020-05-15

Similar Documents

Publication Publication Date Title
CN111160224B (en) A high-speed rail catenary foreign object detection system and method based on FPGA and horizon segmentation
CN112800860B (en) High-speed object scattering detection method and system with coordination of event camera and visual camera
CN108446630B (en) Intelligent monitoring method for airport runway, application server and computer storage medium
CN104112370B (en) Parking lot based on monitoring image intelligent car position recognition methods and system
CN110298300B (en) Method for detecting vehicle illegal line pressing
CN110766333B (en) Intelligent weather phenomenon information processing method and system
CN101656023A (en) Management method of indoor car park in video monitor mode
CN109271872A (en) High-voltage isolating switch split state judgment and fault diagnosis device and method
CN115600124A (en) Subway tunnel inspection system and inspection method
CN104092988A (en) Method, device and system for managing passenger flow in public place
CN107133610B (en) Visual detection and counting method for traffic flow under complex road conditions
CN111967396A (en) Processing method, device and equipment for obstacle detection and storage medium
CN115346143A (en) Behavior detection method, electronic device, and computer-readable medium
CN111626170A (en) Image identification method for railway slope rockfall invasion limit detection
CN110255318B (en) Method for detecting idle articles in elevator car based on image semantic segmentation
CN113936257A (en) Detection method and detection device for vehicle violation behaviors and vehicle-mounted electronic equipment
CN116461571B (en) Obstacle sensing system based on vision, radio frequency positioning and multi-line laser fusion
CN112836619A (en) Embedded vehicle-mounted far infrared pedestrian detection method, system, equipment and storage medium
CN113065478A (en) Complex weather road vehicle target detection method
Su et al. A new local-main-gradient-orientation HOG and contour differences based algorithm for object classification
CN110316630B (en) Deviation early warning method and system for installation angle of elevator camera
CN115546738A (en) A method for detecting foreign objects on railway tracks
CN107992799A (en) Towards the preprocess method of Smoke Detection application
CN112769877A (en) Group fog early warning method, cloud server, vehicle and medium
CN116630904B (en) Small target vehicle detection method integrating non-adjacent skip connections and multi-scale residual structure

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant