Disclosure of Invention
The invention aims to provide a blanking port blocking detection method and system based on an optical flow, which have high automation degree and can improve the real-time performance and accuracy of blanking port blocking detection.
In order to achieve the purpose, the technical scheme adopted by the invention is that the blanking port blocking detection method based on optical flow comprises the following steps:
S1, continuously collecting video images of a blanking port area through camera equipment so as to monitor the material movement of the blanking port area in real time;
s2, comparing continuous image frames in the obtained video images, and analyzing and extracting materials in the images by an optical flow method;
s3, analyzing the area of the material flow by utilizing the optical flow analysis result;
S4, analyzing the movement direction of the material flow by utilizing an optical flow analysis result;
S5, combining analysis results of the material flow area and the material flow movement direction, and judging the material flow state, wherein if the material flow area is increased to exceed a material flow area threshold value and the dispersion degree of the material flow movement direction exceeds a dispersion degree threshold value, the current blanking port is judged to be blocked, otherwise, the current blanking port is judged to be normal;
S6, if the blanking port is judged to be blocked, blocking alarm is carried out.
Further, in step S2, the specific method for analyzing and extracting the material in the image by the optical flow method is as follows:
Global optical flow images L (x, y) are extracted through front and back frame images, pixel motion in the direction of a Cartesian coordinate system (x, y) is converted into the direction and amplitude of motion of a polar coordinate system (sigma, r), and an optical flow image L (sigma, r) expressed by the polar coordinate system is obtained:
The material is the part with larger motion amplitude and the image ratio is small, so the pixels with smaller motion amplitude in the image are regarded as background, the histogram of the amplitude component of the light flow graph L (sigma, r) is calculated, the amplitude is divided into N intervals, the interval is defined as A 1,A2,…,AN, wherein M is the amplitude, the accumulated number in each interval is counted, the optical flow pixels in the first interval are removed, because the motion amplitude is small and the number is large, namely the motion amplitude is smaller than the background of the first interval, the rest is foreground material, a threshold value is set
The optical flow pixels in the remaining intervals are moving materials because they have large motion amplitudes and a small number.
Further, the number of intervals n=20.
Further, in step S3, for all materials extracted from the image, the materials are combined to obtain a flow area, then an envelope curve of the flow area is made, and the area of the flow envelope area is calculated, namely, the flow area a.
Further, if the flow area A is not larger than the set flow area threshold T a, the state of the blanking port is judged to be normal, and if the flow area A is larger than the flow area threshold T a, namely A > T a, the risk of blocking is considered, but the blocking is not directly judged, and the blocking condition is analyzed together with the flow movement direction.
Further, in step S4, the dispersion degree of the stream movement direction is described by using the stream movement standard deviation epsilon, which specifically is:
Wherein sigma i is the pixel direction sample, Q is the sample number, Is the direction average value.
Further, in step S5, based on the set flow area threshold T a and the dispersion threshold T σ, it is determined whether the flow area a > T a and the flow movement standard deviation epsilon > T σ are simultaneously satisfied, if yes, it is determined that the current blanking port is about to be blocked, otherwise, it is determined that the current blanking port is in a normal state.
The invention also provides a blanking port blocking detection system based on the optical flow, which is used for realizing the method and comprises the following steps:
The camera equipment is arranged above the blanking port of the conveyor belt, and is used for collecting video images of the blanking port area and sending the collected video images to the upper computer;
the upper computer is connected with the camera equipment and used for receiving the collected video image, analyzing and extracting materials in the image through an optical flow method, analyzing the flow area and the flow moving direction, and judging the flow state by combining the analysis results of the flow area and the flow moving direction.
Further, the blanking device comprises an alarm device, and the upper computer is connected with the alarm device so as to perform blanking alarm through the alarm device when the blanking port is judged to be blocked.
Compared with the prior art, the blanking port blocking detection method and system based on the optical flow have the advantages that the imaging equipment is arranged on the conveyor belt, video images of blanking port areas are shot in real time, the optical flow state between front and rear frame images is analyzed by the optical flow analysis technology, the material flow area and the material flow moving direction are obtained through analysis, and then the blanking port blocking state is judged by combining the material flow area and the material flow moving direction. Therefore, the invention has strong practicability and wide application prospect.
Detailed Description
The invention will be further described with reference to the accompanying drawings and examples.
It should be noted that the following detailed description is exemplary and is intended to provide further explanation of the application. Unless defined otherwise, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application belongs.
It is noted that the terminology used herein is for the purpose of describing particular embodiments only and is not intended to be limiting of exemplary embodiments according to the present application. As used herein, the singular is also intended to include the plural unless the context clearly indicates otherwise, and furthermore, it is to be understood that the terms "comprises" and/or "comprising" when used in this specification are taken to specify the presence of stated features, steps, operations, devices, components, and/or combinations thereof.
As shown in fig. 1, the embodiment provides a blanking port blocking detection method based on optical flow, which specifically includes the following implementation steps.
S1, collecting real-time images, namely continuously collecting video images of the blanking port area through the camera equipment so as to monitor the material movement of the blanking port area in real time.
S2, analyzing the optical flow of the frames before and after the image, namely comparing the continuous image frames in the obtained video image, and analyzing and extracting materials in the image by an optical flow method. Optical flow is a vector field describing the motion of pixels in an image, and can reflect the motion trail of an object in the image.
S3, analyzing the flow area by utilizing the optical flow analysis result. When the material movement is normal, the material flow area is relatively small, and when the material is about to be blocked, the material flow area is obviously increased.
S4, analyzing the movement direction of the material flow by utilizing the optical flow analysis result. The movement direction of the material in the image is analyzed, and when the material flow directions are consistent, the material flow is smooth, otherwise, if the material flow directions are dispersed, the situation that the material blockage is about to occur is possibly indicated.
S5, combining the analysis results of the flow area and the flow movement direction, and judging the flow state. If the flow area is increased to exceed the flow area threshold and the dispersion degree of the flow movement direction exceeds the dispersion threshold, judging that the current blanking port is about to be blocked, otherwise, judging that the current blanking port is normal.
S6, if the blanking port is judged to be blocked, blocking alarming or information feedback is carried out so as to carry out timely maintenance and treatment.
By the detection method, the state of the blanking port can be accurately analyzed according to the real-time video, the impending blocking situation can be early warned in time, and a reliable monitoring means is provided for the production process.
In order to achieve the above-mentioned method, as shown in fig. 7, the present embodiment further provides a blanking port blocking detection system based on optical flow, which includes an image capturing device and an upper computer.
The camera equipment is arranged above the blanking port of the conveyor belt and used for collecting video images of the blanking port area and sending the collected video images to the upper computer.
The upper computer is connected with the camera equipment and is used for receiving the collected video image, analyzing and extracting materials in the image through an optical flow method, analyzing the flow area and the flow moving direction, and judging the flow state by combining the analysis results of the flow area and the flow moving direction.
Preferably, the system further comprises an alarm device, wherein the upper computer is connected with the alarm device, so that when the blanking port is judged to be blocked, the alarm device can be used for blocking the material to give an alarm.
The related art to which the present method relates is described in further detail below.
(1) Optical flow extraction and material segmentation of frame materials before and after images
The stream is the object in the video that is located at the blanking port and is moving. The invention uses the light flow method to extract the materials in the image. Optical flow is the movement of the same object in an image between the previous and subsequent frames.
Global optical flow images L (x, y) are extracted through front and back frame images, pixel motion in the direction of a Cartesian coordinate system (x, y) is converted into the direction and amplitude of motion of a polar coordinate system (sigma, r), and an optical flow image L (sigma, r) expressed by the polar coordinate system is obtained:
Since the material is the part of the image with larger motion amplitude and occupies a small image ratio, the pixels of the image with smaller motion amplitude are considered as the background. Calculating a histogram of amplitude components of a light flow graph L (sigma, r), dividing the amplitude into N bins, the bins being defined as A 1,A2,…,AN, wherein M is the amplitude, and the accumulated number in each interval is counted. The histogram of optical flow amplitude is shown in fig. 2. In the present embodiment, the number of intervals n=20 is taken. Removing optical flow pixels in the first interval because of small and numerous motion amplitudes, namely taking the background with the motion amplitude smaller than that of the first interval and the rest foreground materials, setting a threshold value
The optical flow pixels in the remaining intervals are moving materials because they have large motion amplitudes and a small number.
The implementation process of the material extraction is shown in fig. 3. In fig. 3, from left to right, an original image, an optical flow visualization image and a material mask image are respectively shown.
(2) Flow surface area analysis
And combining all materials extracted from the image to obtain a material flow area, then, taking an envelope curve of the material flow area, and calculating the area of the material flow envelope area, namely the material flow area.
If the material flow area A is not larger than the set material flow area threshold T a, the state of the blanking port is judged to be normal, and if the material flow area A is larger than the material flow area threshold T a, namely A > T a, the risk of material blockage is considered, but the material blockage cannot be directly judged, and the material blockage condition needs to be analyzed together with the material flow moving direction. The flow envelope area when no plugging occurred is shown in fig. 4. In fig. 4, the left side is a diagram of the stream envelope region when no blockage occurs, and the right side is a corresponding monitoring video image. The flow envelope area when plugging occurs is shown in fig. 5. In fig. 5, the left side is a diagram of a flow envelope area when a blockage occurs, the flow envelope area increases and exceeds a set threshold, and the right side is a corresponding monitoring video image.
(3) Stream direction consistency analysis
When the blanking port is not blocked, the material flow is smooth, and when the material is blocked, the material flow moves towards all directions. Therefore, the invention adopts the consistency of the material flow direction as the condition of blanking mouth blocking analysis. As shown in the radar chart of fig. 6, the left side is the optical flow intensity and pattern without blockage, and the right side is the optical flow intensity and pattern with blockage. In contrast, the optical flow direction without the plugging material is more concentrated, the strength is larger in a certain direction, and the optical flow direction of the plugging material is dispersed.
In this embodiment, the dispersion degree of the stream movement direction is described by using the stream movement standard deviation epsilon, which specifically is:
Wherein sigma i is the pixel direction sample, Q is the sample number, Is the direction average value.
(4) Material flow state discrimination
Based on the set material flow area threshold T a and the dispersion threshold T σ, judging whether the material flow area A is more than T a and the material flow standard deviation epsilon is more than T σ at the same time, if yes, judging that the current blanking port is about to be blocked, otherwise, judging that the current blanking port is normal.
The above description is only a preferred embodiment of the present invention, and is not intended to limit the invention in any way, and any person skilled in the art may make modifications or alterations to the disclosed technical content to the equivalent embodiments. However, any simple modification, equivalent variation and variation of the above embodiments according to the technical substance of the present invention still fall within the protection scope of the technical solution of the present invention.