CN111079819A

CN111079819A - Method for judging state of coupler knuckle pin of railway wagon based on image recognition and deep learning

Info

Publication number: CN111079819A
Application number: CN201911272249.7A
Authority: CN
Inventors: 王斐
Original assignee: Harbin Kejia General Mechanical and Electrical Co Ltd
Current assignee: Harbin Kejia General Mechanical and Electrical Co Ltd
Priority date: 2019-12-12
Filing date: 2019-12-12
Publication date: 2020-04-28
Anticipated expiration: 2039-12-12
Also published as: CN111079819B

Abstract

A method for judging the state of a coupler knuckle pin of a railway wagon based on image recognition and deep learning relates to the field of fault judgment of railway wagons. The invention aims to solve the problems that when the fault detection is carried out on a truck manually in the prior art, the detection result is easy to be inaccurate, and further the fault can not be found in time easily. According to the invention, the image automatic identification mode is used for replacing manual detection, so that the fault identification detection efficiency and accuracy are improved, and the labor cost is reduced. Meanwhile, the invention also applies deep learning to component positioning and fault detection, and can effectively improve the robustness and accuracy of the algorithm. The small targets in the images can be effectively positioned and identified by adopting a mode of firstly carrying out coarse positioning on the target detection network and then carrying out identification in the positioned screenshot, so that the accuracy and precision of detection are improved.

Description

Method for judging state of coupler knuckle pin of railway wagon based on image recognition and deep learning

Technical Field

The invention belongs to the field of fault judgment of rail wagons, and particularly relates to fault judgment of a coupler knuckle pin of a rail wagon.

Background

The coupler of the railway wagon is a vehicle part which is used for realizing coupling between a locomotive and a vehicle or between the vehicle and the vehicle, transmitting traction force and impact force and keeping a certain distance between the vehicles. The coupler comprises a coupler knuckle and a coupler body, and the coupler is assembled as follows: two side surfaces of the coupler knuckle are respectively provided with a pin hole which is connected with a coupler knuckle pin and assembled at the mounting hole of the coupler body, and the coupler knuckle can rotate around the coupler knuckle pin. In the running process of a train, the coupler knuckle pin is frequently subjected to the action of pulling force, compression force and impact force, so that the coupler knuckle pin has the faults of cracking, falling, breaking and the like in a long time. Once the faults occur in the running process of the train, the train is easily separated, the emergency braking of the train is forced, wheel set grooves are scratched and the like, and serious accidents such as the derailment and overturn of the train can be caused in serious cases. Therefore, inspection for coupler knuckle failure is enhanced when the vehicle is inspected.

At present, the fault detection of the truck generally adopts a manual troubleshooting mode. The investigation process is greatly influenced by factors such as the business quality, the responsibility and the labor intensity of operators, so that the conditions of missing inspection, operation simplification and the like are easy to occur. When the coupler knuckle pin is cracked, the coupler knuckle pin is often difficult to find manually in time, further deterioration, even loss or breakage of the coupler knuckle pin are easily caused, and at the moment, faults cannot be found in time, and further serious vehicle faults are caused.

Disclosure of Invention

The invention provides a method for judging the state of a coupler knuckle pin of a railway wagon based on image recognition and deep learning, aiming at solving the problems that when the conventional manual fault detection is carried out on the wagon, the detection result is easy to be inaccurate, and further the fault can not be found in time easily.

The method for judging the state of the coupler knuckle pin of the railway wagon based on image recognition and deep learning comprises the following steps of:

a data set establishing step:

acquiring coupler pictures of different types of railway wagon couplers in different time, places and environments to establish a sample library, wherein the coupler pictures comprise coupler pictures in a normal state and coupler pictures in a fault state;

marking the position of a coupler knuckle pin in a coupler picture, generating a corresponding label file, and taking the marked coupler picture and the corresponding label file as a training data set of a front target detection network;

intercepting the position of the coupler knuckle pin in a marked coupler picture, respectively delineating the outline of a hose, the outline of the coupler knuckle pin in a normal state, the outline of the coupler knuckle pin when the coupler knuckle pin is broken and the outline of a pin hole when the coupler knuckle pin is lost in different screenshots, respectively marking the outlines in 4 states, and taking all screenshots and corresponding state marks as training data sets of a segmentation network;

weight training:

training a target detection network by using data in a training data set of the preposed target detection network, wherein the target detection network is an SSD deep learning network;

training a segmentation network by using data in a training data set of the segmentation network, wherein the training segmentation network is a Mask-rcnn deep learning network;

a picture acquisition step:

acquiring a coupler picture of a railway wagon to be detected, and adjusting pixels of the picture into a picture to be detected of 512 multiplied by 512;

and a fault identification step:

inputting the picture to be detected obtained in the picture acquisition step into a trained target detection network, and intercepting the picture to be detected to obtain a screenshot of the position of the coupler knuckle pin in the picture to be detected;

inputting the screenshot into a segmentation network, respectively matching the screenshot with the 4 state marks, matching the state marks corresponding to the screenshot, and taking the state corresponding to the state marks as a coupler knuckle pin state result of the railway wagon to be detected.

The sample library not only comprises the acquired pictures, but also comprises the pictures after the acquired pictures are stretched, rotated and mirrored.

The SSD deep learning network comprises a front-end feature extraction network and a rear-end multi-scale feature detection network, wherein the front-end feature extraction network is a VVG-16 network.

The target detection network can obtain a plurality of position frames after intercepting the picture to be detected, and the image intercepted by the position frame with the highest score is used as the screenshot of the position of the coupler knuckle pin in the picture to be detected.

In the picture collection step, high-definition image collection equipment is arranged around a rail of the railway wagon and is used for collecting a car coupler picture of the railway wagon to be detected.

According to the invention, the image automatic identification mode is used for replacing manual detection, so that the fault identification detection efficiency and accuracy are improved, and the labor cost is reduced. Meanwhile, the invention also applies deep learning to component positioning and fault detection, and can effectively improve the robustness and accuracy of the algorithm. The small targets in the images can be effectively positioned and identified by adopting a mode of firstly carrying out coarse positioning on the target detection network and then carrying out identification in the positioned screenshot, so that the accuracy and precision of detection are improved.

Drawings

Fig. 1 is a flowchart of a method for determining a state of a coupler knuckle pin of a railway wagon based on image recognition and deep learning according to the present invention.

Detailed Description

With the great improvement of the processing performance of chip hardware, a foundation is provided for the complex computation of a deep network. The deep learning network can be widely applied to the field of image processing, and compared with the traditional mode, the deep learning method integrates feature learning into the process of establishing the model, and can effectively improve the accuracy and efficiency of fault detection.

The deep learning method is integrated into the detection field of the wagon, so that the condition of the coupler knuckle pin of the wagon is accurately judged. The method comprises the following specific steps:

the first embodiment is as follows: specifically describing the present embodiment with reference to fig. 1, the method for determining the state of a coupler knuckle pin of a railway wagon based on image recognition and deep learning according to the present embodiment includes the following steps:

firstly, establishing a data set:

the method comprises the steps of collecting coupler pictures of different types of railway freight car couplers in different time, places and environments from a network big database or an actual application environment, then respectively carrying out stretching, rotation and mirror image conversion on each collected picture, and establishing a sample library by utilizing all pictures before and after the conversion, wherein the coupler pictures comprise coupler pictures in a normal state and coupler pictures in a fault state. For example: the method comprises the steps of collecting a car coupler picture, copying three identical pictures, respectively stretching, rotating and carrying out mirror image transformation on the three copied pictures to obtain a stretched picture, a rotated picture and a mirrored picture, and then storing an original picture and the three transformed pictures into a sample library together. The operation aims at amplifying the sample, collecting the car coupler images under different conditions is beneficial to enriching the sample data, and the robustness and the adaptability of the subsequent training result are improved.

And marking the positions of coupler knuckle pins in all coupler pictures of the sample library, generating corresponding label files, and taking the marked coupler pictures and the corresponding label files as a training data set of the front target detection network.

Intercepting the position of the coupler knuckle pin in the marked coupler picture, respectively delineating the outline of the hose, the outline of the coupler knuckle pin in a normal state, the outline of the coupler knuckle pin when the coupler knuckle pin is broken and the outline of the pin hole when the coupler knuckle pin is lost in different screenshots, respectively marking the outlines in 4 states, and taking all screenshots and corresponding state marks as training data sets of a segmentation network. The hose is used as a category in the embodiment, so that the false alarm problem caused by the shielding state can be effectively avoided.

Secondly, weight training:

training a target detection network by using data in a training data set of the preposed target detection network, wherein the target detection network is an SSD deep learning network; the SSD deep learning network comprises a front-end feature extraction network and a rear-end multi-scale feature detection network, wherein the front-end feature extraction network is a VVG-16 network.

And the size of the feature map generated by the front-end feature extraction network is reduced layer by layer through pooling operation, then the object classification and the deviation of a target boundary frame are predicted by using a plurality of feature maps of different convolutional layers, and finally a final detection result is generated by using a maximum suppression method, so that the detection of the plurality of scale feature maps is realized. The image size of the coupler in the image is large, the height direction ratio exceeds 0.7, and the width direction ratio exceeds 0.25. In the multi-scale detection process, a low layer predicts a small target, and a high layer predicts a large target. In the embodiment, the SSD deep learning network does not need small target detection, so that a low-level prediction network in the SSD deep learning network is removed to improve the operation efficiency.

And training the segmentation network by using the data in the training data set of the segmentation network, wherein the training segmentation network is a Mask-rcnn deep learning network.

Specifically, image pixels in the sample library are uniformly adjusted to 512 × 512, then the image pixels are sent to an SSD deep learning network for training, the learning rate is set to 0.0001 by tuning back parameters, and the front-end feature extraction network weight is obtained after training.

As the hose can partially shield the coupler knuckle pin in the advancing process of the truck, the Mask-rcnn deep learning network is mainly adopted for example segmentation for judging loss and breakage faults of the coupler knuckle pin, and the resnet-101 is adopted for feature extraction of the specific Mask-rcnn deep learning network. Since the coupler pin is small in the image, the anchor point is set to (8,16,32,64,128), and the image is sent to the network for training to obtain the example segmentation network weight.

Through the steps, the weight in the network is obtained, and the training of the network is realized.

Thirdly, picture acquisition:

firstly, arranging high-definition image acquisition equipment around a rail of a railway wagon for acquiring pictures at a coupler of the railway wagon to be detected, and then adjusting pixels of the acquired pictures into 512 multiplied by 512 pictures to be detected;

fourthly, fault identification:

Specifically, the target detection network can obtain a plurality of position frames after intercepting the picture to be detected, and an image intercepted by the position frame with the highest score is used as a screenshot of the position of the coupler knuckle pin in the picture to be detected.

In practical application, due to the fact that the positions of the car couplers in different car model images are different, the car coupler image with a large range needs to be intercepted, and the faults of the coupler knuckle pins cannot be directly positioned and classified in the car coupler image with the large range. Therefore, the present embodiment employs a policy of positioning before classification. And after the car coupler image is acquired by high-definition image acquisition equipment arranged around the track, the car coupler image is input to a target detection network. In the obtained result, a plurality of detection results may appear in the same image, and the scores of different results are compared, and the position frame with the largest score is taken as the final result. And acquiring the position of the coupler knuckle pin, intercepting a corresponding image, and closing the target detection network.

The coupler knuckle pin is shielded by the hose in the shooting process. Thus, the example split network is used to classify and locate the failure of the coupler knuckle pin. And inputting the screenshot acquired by the target detection network into an example network, judging whether the coupler knuckle pin is lost or broken, and marking the position of the coupler knuckle pin. And judging the fault according to the length, the width, the area and the corresponding fraction in the output result. Loss and breakage of the knuckle pin with a size and fraction less than a threshold value is not treated as a fault.

The accuracy of small target identification detection in the deep learning field is not high all the time. Therefore, the two networks are continuously used for identification in the process of identifying the loss and break faults of the coupler knuckle pin, so that the accuracy of small target identification can be effectively improved, and the missing report and the false report are avoided.

In summary, in the embodiment, the high-definition imaging devices are installed at the two sides and the center of the rail of the truck, and the truck obtains images after passing through the device installation position. Firstly, a target detection network in deep learning is used for accurately positioning the wagon coupler, then an example segmentation network in the deep learning is used for processing a coupler image, and a normal coupler knuckle pin, a fault coupler knuckle pin and other parts are positioned and classified, so that a state result is finally obtained. And the staff performs corresponding processing according to the image recognition result to ensure the safe operation of the locomotive.

Claims

1. the state judging method of the railroad wagon coupler pin based on image recognition and deep learning, is characterized in that, comprises the following steps:

Data set creation steps:

Collecting coupler pictures of different models of railway freight cars at different times, places and environments to establish a sample library, the coupler pictures include coupler pictures in normal state and coupler pictures in fault state;

Mark the position of the hook pin in the coupler picture, and generate the corresponding label file, and use the marked coupler picture and the corresponding label file as the training data set of the pre-target detection network;

In the marked coupler picture, the position of the knuckle pin is taken, and in different screenshots, the outline of the hose, the contour of the knuckle pin in the normal state, the contour of the knuckle pin when it is broken, and the missing knuckle pin are drawn respectively. The outline of the pin hole is marked, and the outlines in the four states are marked respectively, and all the screenshots and their corresponding state markings are used as the training data set of the segmentation network;

Weight training steps:

Use the data in the training data set of the pre-target detection network to train the target detection network, which is an SSD deep learning network;

Using the data in the training dataset of the segmentation network to train the segmentation network, the training segmentation network is a Mask-rcnn deep learning network;

Image collection steps:

Collect the coupler picture of the railway freight car to be tested, and adjust the pixels of the picture to a 512×512 picture to be tested;

Fault identification steps:

Input the picture to be detected obtained in the picture collection step into the target detection network after training, intercept the picture to be detected, and obtain a screenshot of the position of the hook pin in the picture to be detected;

Input the above screenshots into the segmentation network, and match them with the four state flags respectively to match the state flags corresponding to the screenshots.

2. The method for judging the state of the coupler pin of a railway freight car based on image recognition and deep learning according to claim 1, wherein the sample library not only includes the collected pictures, but also includes stretching the collected pictures. , rotated and mirrored images.

3. The method for judging the state of a railway wagon coupler pin based on image recognition and deep learning according to claim 1, wherein the SSD deep learning network comprises a front-end feature extraction network and a back-end multi-scale feature detection network, the described The front-end feature extraction network is VVG-16 network.

4. The method for judging the state of the coupler pin of a railway freight car based on image recognition and deep learning according to claim 1 or 3, wherein the target detection network can obtain a plurality of position frames after the image to be detected is intercepted, and the score The image captured by the highest position frame is used as a screenshot of the position of the hook pin in the picture to be detected.

5. The method for judging the state of a railroad wagon yoke pin based on image recognition and deep learning according to claim 1, wherein, in the picture collection step, firstly, a high-definition image collection device is set around the railroad wagon track, for collecting A picture of the coupler of the railway wagon under test.