CN117058036A - Road crack image preprocessing method based on LSD and FLD fusion - Google Patents

Abstract

The invention belongs to the field of image processing, and specifically relates to a road crack image preprocessing method based on the fusion of LSD (Line Segment Detector) and FLD (Fast Line Detector). The specific technical solution includes: using the LSD algorithm and the FLD algorithm to detect the crack images in the crack image. Detect straight line interference objects (road signs, curbs, etc.) and obtain the line segment coordinates of these interference objects; disconnect and reconnect based on the line segment coordinate values returned by the straight line detection algorithm, solving the problem of discontinuous line segment extraction by the straight line detection algorithm; Based on the line segment coordinates of the distractors obtained previously, a mask image with only the distractors is generated; using the mask image and the original image through the FMM (Fast Marching Method) algorithm, the distractors in the original image are replaced with surrounding pixels to eliminate interference. The purpose of this invention is to eliminate interference items in road crack images that are most likely to be misidentified as cracks, greatly improving the accuracy of crack identification, and at the same time, the implementation method and platform are simple and the execution efficiency is high.

Description

A road crack image preprocessing method based on LSD and FLD fusion

Technical field

The invention belongs to the field of image processing and relates to a method for eliminating interference objects in road crack images.

Background technique

Crack damage is one of the most common road diseases. It will cause damage to the pavement structure, thereby reducing the overall performance and life of the pavement. As the demand for pavement maintenance increases, the automatic crack detection system is extremely important for pavement maintenance. The detection of pavement cracks Parameter information includes: type, location, damage extent, length, width, depth and other information. Since the condition of the road surface is complex and changeable, there are still the following challenges in detecting pavement crack information: (1) It is difficult to identify pavement cracks quickly, correctly, completely and smoothly; (2) The evaluation mechanism of the crack identification algorithm is not sound enough; (3) ) lacks a mature system for automatically identifying the types and severity of pavement cracks. Among them, it is difficult to quickly, correctly, completely and smoothly identify pavement cracks, which is the top priority in road disease management. Reducing the interference of non-crack targets in crack images can effectively improve the accuracy of crack identification.

Contents of the invention

3.1 Purpose of invention

In order to solve the problem of interference objects affecting crack recognition in crack detection, the present invention proposes a road crack image preprocessing method based on the fusion of LSD (Line Segment Detector) and FLD (Fast Line Detector), which is used to eliminate straight lines in road crack images. interference objects to improve the accuracy of road crack detection. The specific technical solution includes the following four parts:

(1) Straight line detection: Use the LSD algorithm and FLD algorithm to detect straight line interference objects (road signs, curbs, etc.) in the crack image, and obtain their line segment coordinates;

(2) Disconnection and reconnection: Disconnection and reconnection are performed based on the obtained line segment coordinates to solve the problem of discontinuous line segment extraction;

(3) Obtain the mask image: Based on the line segment coordinates of the interference object obtained previously, generate a mask image with only the interference object;

(4) Eliminate interference objects: Use the mask image and the original image to use the FMM (Fast Marching Method) algorithm to replace the interference objects in the original image with surrounding pixels to achieve the purpose of eliminating interference objects;

3.2 Technical solutions

The present invention proposes a road crack image preprocessing method based on the fusion of LSD and FLD. The specific technical solution is described in detail below. The main process is shown in Figure 1:

Straight line detection stage: LSD is a straight line detection algorithm, which is based on the operation of target gradient and can quickly detect straight line features with sub-pixel accuracy on the image; FLD straight line detection algorithm is a straight line detection algorithm based on edge detection. , its principle is to find the straight line features in the image edges by analyzing them.

The present invention is about the straight line detection of crack images, so in the detection process, it is also necessary to consider that the cracks cannot be detected as straight lines. To address this problem, combined with traditional preprocessing methods such as filtering and dilation, an LSD algorithm and FLD algorithm based on grayscale histogram adaptation and combined with different thresholds are proposed to identify linear interference objects in crack images. The specific implementation method is: combined with the gray level histogram of the image, statistically obtains the specific range of gray levels occupied by the straight lines in the crack image and the total number of pixels occupied by the gray level range, and adaptively uses LSD according to the different input images. algorithm and the FLD algorithm. First, use labelme to mark straight lines of thickness to generate a mask image; secondly, based on the combination of the mask image and the original image, a histogram is generated, and the gray level range of the straight line is 0 to 60 (as shown in Figure 2); then, All images are divided into two groups: thin straight lines and thick straight lines. The pixel values of the gray level of each image between 0 and 60 are respectively counted to obtain the minimum pixel value min_pixel of the thick line and the maximum pixel value max_pixel of the thin line; finally, according to min_pixel and max_pixel use LSD algorithm and FLD algorithm to detect straight lines in crack images. When preprocessing road crack images, a fast and efficient method is needed, so the present invention uses an adaptive method to perform straight line detection. If the preprocessing work of the present invention is performed without using adaptive technology (the flow chart is shown in Figure 3), the processing time will become longer because each image has to undergo three straight line detection and three straight line elimination tasks; using adaptive After the method, the pixel values of the image gray level between 0 and 60 that are higher than 130,000 do not undergo FLD straight line processing, and those that are lower than 10,000 do not undergo LSD straight line processing (as shown in Figure 4), without reducing the straight line detection rate. , which greatly saves preprocessing time. Different straight line detection algorithms are used according to the different gray values of the image. The FLD algorithm detects long and thin straight lines, and the LSD algorithm detects long and thick straight lines and short straight lines.

Disconnection and reconnection stage: The algorithm used and the threshold of the algorithm are different depending on the straight line, but some straight lines will be recognized intermittently, as shown in Figure 5. Therefore, the present invention uses the straight line coordinate list returned by the straight line detection to perform disconnection. The main process of reconnection is as follows:

(1) Create empty arrays X1[], Y1[], X2[], Y2[] to store the first and last coordinate values of the line segment;

(2) Traverse the list of line segment coordinates returned by the straight line detection algorithm of the current image, return the first and last coordinates (x1, y1), (x2, y2) of the line segments whose length is within the threshold range according to requirements, and store them in pre-created arrays respectively. middle;

(3) Calculate the slope k1 and intercept d1 of the current line segment by double traversing X1[], Y1[], ) Calculate its slope k2 and intercept d2; then calculate the distance dis between the two line segments;

(4) Since the straight lines in the crack image are basically horizontal and vertical, it is necessary to connect the new straight lines according to the situation: if k1=k2=∞, then calculate the absolute value of the difference between x1 and x3. If it is less than 3 and dis If it is less than 500, connect the tail coordinate of the current line segment and the first coordinate of the line segment to be compared to form a new straight line; if k1=∞, the absolute value of the difference between x3 and x4 is less than 5, or k2=∞, the difference between x1 and x2 The absolute value of the value is less than 5, and dis is less than 500, then connect the tail coordinate of the current line segment and the first coordinate of the line segment to be compared to form a new straight line; if k1=k2=0, calculate the difference between y1 and y3 Absolute value, if it is less than 3 and dis is less than 500, connect the tail coordinate of the current line segment and the first coordinate of the line segment to be compared to form a new straight line; if k1 and k2 are both less than -1, or both are greater than 1, then if d1 If the absolute value of the difference from d2 is less than 10 and dis is less than 500, connect the tail coordinate of the current line segment and the first coordinate of the line segment to be compared to form a new straight line; if k1 and k2 are both between -1 and 1 (including -1, 1, but not including 0), if the absolute value of the difference between k1 and k2 is less than 0.1, the absolute value of the difference between d1 and d2 is 5, and dis is less than 500, then the tail coordinate of the current line segment and the line segment to be compared are The first coordinates are connected to form a new straight line.

Slope formula:

Intercept formula:

Distance formula:

(5) Repeat steps (3) (4) until every straight line in the image is compared with each other.

The reconnection mechanism of discontinuous line segments solves the problem of discontinuous straight lines, making the elimination of straight lines more thorough and reducing the false recognition rate of crack detection.

Interference elimination stage: The FMM algorithm is based on a fast marching method. Considering the area to be repaired in the image, the algorithm starts from the boundary of the area and enters the interior of the area; first, it gradually fills in all the content in the boundary, and then gradually fills in all the content in the boundary to be repaired. A small neighborhood is mined around the domain pixel; that pixel is then replaced by the normalized weighted sum of all nearby known pixels, so the choice of weights is important, those pixels located near the point, near the boundary normal and those located near Pixels on the boundary outline will be given greater weight; finally, once the pixel is fixed, it will be moved to the next closest pixel using the fast march method, and so on. The present invention first detects straight-line interference objects in the image through a straight-line detection algorithm, then generates a mask diagram with only straight-line interference objects, and then uses the FMM algorithm to process the original image based on the mask diagram. The result is shown in Figure 6 shown.

3.3 Beneficial effects

According to the complexity of road crack detection scenarios, in order to verify the applicability of the algorithm of the present invention in different scenarios, roads are divided into three different scenarios: seam category, zebra crossing, arrow and other road sign categories, and roadside line category. The algorithm of the present invention is used to process crack images of different scenes respectively.

(1) Preprocessing experimental results in joint scenarios

The seam is a very thin straight line with a darker color, while the crack is also a thinner line with a darker color. The difference between the two is that the crack has irregular branching lines, while the seam is a relatively straight line without branches. , so the present invention combined with the straight line detection algorithm can effectively remove this type of interference. The crack detection experimental results are shown in Figure 7.

(2) Preprocessing experimental results in road marking scenarios

Ground road signs are painted with paint, and they will crack over time. However, these are not cracks in the asphalt pavement. This kind of line is the line closest to the crack, so it is particularly easy for the machine to identify it as a crack. cracks, resulting in an increase in misrecognition rate. Eliminated by the present invention, the crack detection experimental results are shown in Figure 8.

(3) Preprocessing experimental results in roadside scenarios

When collecting crack image data, there will be roads paved with floor tiles on both sides, and such roads will have regular brick lines. When crack recognition is performed later, such lines can easily be mistakenly recognized as cracks, so they can also be removed by straight line inspection. The results of the crack detection experiment are shown in Figure 9.

At the same time, in order to verify that the straight line detection algorithm of the present invention has a better detection effect than other straight line detection algorithms, the straight line detection results of the algorithm of the present invention and other algorithms are compared. The effect diagram is shown in Figure 10.

In summary, the two comparative experimental results show that the advantages of the present invention are reflected in the following points: 1. This method eliminates interference items that are most easily mistakenly identified as cracks in road crack images, greatly improving the accuracy of crack identification. ; 2. Use the disconnection and reconnection mechanism to improve the integrity of linear detection and eliminate interference lines more thoroughly; 3. The implementation method and platform are simple and the execution efficiency is high.

Description of the drawings

Figure 1 is the overall flow chart of a road crack image preprocessing method based on the fusion of LSD and FLD involved in the present invention.

Figure 2 shows the grayscale histograms of thick lines and thin lines respectively.

(a) is a thick line grayscale histogram

(b) is a thin line grayscale histogram

Figure 3 shows the LSD and FLD series system

Figure 4 shows the parallel system of LSD and FLD

Figure 5 shows the comparison before and after disconnection and reconnection.

(a) Before reconnection

(b) After reconnection

Figure 6 shows the comparison before and after the FMM algorithm eliminates interference objects.

(a) is the original picture

(b) is the mask image

(c) is the image after elimination

Figure 7 is a comparison chart of crack identification after traditional pretreatment and treatment by the method of the present invention on joint interference objects.

(a) is the original picture

(b) Identification of cracks after traditional pretreatment

(c) Crack identification after processing by this algorithm

Figure 8 is a comparison chart of crack identification of road sign interference objects after traditional preprocessing and processing by the method of the present invention.

(a) is the original picture

(b) Identification of cracks after traditional pretreatment

(c) Crack identification after processing by this algorithm

Figure 9 is a comparison chart of crack recognition of roadside interference objects after traditional pretreatment and treatment by the method of the present invention.

(a) is the original picture

(b) Identification of cracks after traditional pretreatment

(c) Crack identification after processing by this algorithm

Figure 10 is a comparison chart of the effects of common straight line detection algorithms and the straight line detection algorithm of the present invention.

(a) is the Hough_line algorithm detection map

(b) Detection diagram for HoughP_line algorithm

(c) is the detection diagram of FLD algorithm

(d) is the detection diagram of LSD algorithm combined with Canny operator

(e) is the detection diagram of LSD algorithm

(f) Detection diagram for the algorithm of this article

Detailed ways

The present invention proposes a road crack image preprocessing method based on the fusion of LSD and FLD. In order to make the technical solution and process of the present invention clearer, the specific implementation of the present invention will be described in detail below with reference to Figure 1.

(1) Grayscale the image: Compared with color images, grayscale images occupy less memory and are faster in operation. After grayscale, visual contrast can be increased and the target area can be highlighted.

(2) Obtain the image gray level histogram: An image is composed of pixels with different gray values. The distribution of gray levels in the image is an important feature of the image. First, obtain the grayscale histogram of the image, and you can get the number of pixels with this grayscale in the image: where the abscissa is the grayscale, and the ordinate is the frequency of occurrence of the grayscale. Through data statistics in the early stage, the gray level range of the linear joints obtained is 0 to 60, the minimum pixel value min_pixel of the thick straight line is 10000, and the maximum pixel value max_pixel of the thin line is 130000.

(3) Perform FLD straight line detection: Obtain the total number of pixels with a gray level between 0 and 60 in each image. Images with a total number of pixels in this interval lower than 130,000 will be subjected to FLD detection. Images higher than this value will not be passed. FLD inspection, this step mainly eliminates thin and straight straight line interference objects, mostly seams. The main process of FLD straight line detection in the present invention is: the first step is to perform normalization, histogram equalization, Gaussian filtering and other operations on the image obtained in the previous round to make the thin and straight straight line features more obvious, which is beneficial to FLD detection. The second step generates an image threshold map and quickly produces a line map. The third step is to perform FLD linear detection. The fourth step is to draw the straight lines detected by FLD on the 3846*1000 mask image, and then perform image repair processing by combining the mask image with the image obtained in the previous round to eliminate thin and straight straight line interference.

(4) Carry out the first round of LSD detection: Although the LSD algorithm has the advantage of no parameter input, for the straight line detection in the crack image of the present invention, the threshold settings are different, and the recognition effects are also different. Therefore, this invention divides linear interference objects into seams and non-seams, and then the LSD algorithm automatically sets different thresholds according to the differences in interference objects in the input image, ultimately eliminating various linear interference objects without mistakenly eliminating cracks. the goal of. Similarly, based on the total number of pixels from 0 to 60 obtained previously, images with pixels higher than 10,000 in this interval are subjected to this round of LSD detection: the first step is to reduce the size of the image from 3846*1000 to 1000*500, and the LSD detection can be increased The recognition rate of long and thick straight lines while avoiding misidentification of cracks (LSD algorithm reduces the input image source to 80% of the original image, thus weakening or eliminating the aliasing effect in the image, and uses Gaussian downsampling to operate the input image . However, since the data used in this invention is high-resolution data, the original image is then scaled to increase the recognition rate of long and thick straight lines). The second step is to perform Bilateral filtering on the image (Bilateral filtering is a non-linear filtering method, which is a compromise process that combines the spatial proximity and pixel value similarity of the image, taking into account spatial domain information and grayscale. similarity to achieve the purpose of edge preservation and denoising, effectively avoiding the misidentification of some cracks as straight lines). The third step is to perform LSD straight line detection on the image. The tilt angle tolerance is 70 degrees. It has wider coverage of straight lines and can also detect straight lines with less smooth edges. Therefore, when drawing straight lines, set the straight line length threshold to 500, short straight lines are excluded to avoid identifying cracks as straight lines. The last step is to draw the straight lines detected by LSD on a 1000*500 mask image, and then perform image repair processing by combining the mask image and the resized image to eliminate long and thick straight line interference, and then restore the image size.

(5) Conduct the second round of LSD inspection: This round of inspection is mainly to detect other types of linear interference objects other than seams, such as paint lines, etc. These interference objects are characterized by being short, and at the same time, it is necessary to avoid mistaking cracks similar to them. It is recognized as a straight line, so its threshold setting is different from that of identifying straight lines such as seams, so it needs to be identified separately. The first step is to perform mean filtering. The second step is to perform LSD straight line detection with an inclination angle tolerance of 22 degrees and a quantization error limited to the gradient norm of 2. This parameter requires higher edge smoothness and straightness of the straight line. The third step is to disconnect and reconnect; the last step is to draw the straight lines detected by LSD on the 3846*1000 mask map, and then combine the mask map and the resized image for image repair processing to eliminate short straight lines Disturbance.

(6) Finally, the image after eliminating straight line interference is output and saved.

Claims (4)

1. A road crack image preprocessing method based on the fusion of LSD (Line Segment Detector) and FLD (Fast Line Detector). Its features include: Straight line detection: Use the LSD algorithm and FLD algorithm to detect straight line interference objects (road signs, curbs, etc.) in the crack image and obtain their line segment coordinates; Disconnection and reconnection: Disconnection and reconnection are performed based on the obtained line segment coordinates to solve the problem of discontinuous line segment extraction; Obtain the mask image: Based on the line segment coordinates obtained previously, generate a mask image with only interference objects; Eliminate interference objects: Use the mask image and the original image to use the FMM (Fast Marching Method) algorithm to replace the interference objects in the original image with surrounding pixels to achieve the purpose of eliminating interference objects. 2. The straight line detection according to claim 1 includes: In order to avoid detecting cracks as straight lines, an adaptive LSD algorithm and FLD algorithm based on gray-level histograms that combine different thresholds are proposed. The implementation method is: combined with the gray-level histogram of the image, statistically obtain the gray proportion of straight lines in the crack image. The specific range of the gray level and the total number of pixels occupied by the gray level range are used to distinguish different forms of straight line interference objects according to the gray level value; different straight line detection algorithms are used according to the different gray level values of the image, and the FLD algorithm detects long and thin types. Straight lines, LSD algorithm detects long and thick straight lines and short straight lines. 3. Disconnection and reconnection according to claim 2: The straight line detected by the straight line detection algorithm has a discontinuity problem. It is proposed to use the straight line coordinate list returned by the straight line detection to reconnect the line. The main operations are as follows: (1) Create empty arrays X1[], Y1[], X2[], Y2[] to store the first and last coordinate values of the line segment; (2) Traverse the list of line segment coordinates returned by the straight line detection algorithm of the current image, return the first and last coordinates (x1, y1), (x2, y2) of the line segments whose length is within the threshold range according to requirements, and store them in pre-created arrays respectively. middle; (3) Calculate the slope k1 and intercept d1 of the current line segment by double traversing X1[], Y1[], ) Calculate its slope k2 and intercept d2; then calculate the distance dis between the two line segments; (4) The straight lines in the crack image are basically horizontal and vertical. It is necessary to connect the new straight lines according to the situation. If k1=k2=∞, then calculate the absolute value of the difference between x1 and x3. If it is less than 3 and dis is less than 500 , then connect the tail coordinate of the current line segment and the first coordinate of the line segment to be compared to form a new straight line; if k1=∞, the absolute value of the difference between x3 and x4 is less than 5, or k2=∞, the absolute value of the difference between x1 and x2 If the absolute value is less than 5 and dis is less than 500, connect the tail coordinate of the current line segment and the first coordinate of the line segment to be compared to form a new straight line; if k1=k2=0, calculate the absolute value of the difference between y1 and y3 , if it is less than 3 and dis is less than 500, connect the tail coordinate of the current line segment and the first coordinate of the line segment to be compared to form a new straight line; if k1 and k2 are both less than -1, or both are greater than 1, then if d1 and d2 If the absolute value of the difference is less than 10 and dis is less than 500, connect the tail coordinate of the current line segment and the first coordinate of the line segment to be compared to form a new straight line; if k1 and k2 are both between -1 and 1 (including -1 , 1, but not including 0), if the absolute value of the difference between k1 and k2 is less than 0.1, the absolute value of the difference between d1 and d2 is 5, and dis is less than 500, then the tail coordinate of the current line segment and the first coordinate of the line segment to be compared Connect into a new straight line; Slope formula: Intercept formula: Distance formula: (5) Repeat steps (3) (4) until every straight line in the image is compared with each other. 4. Interference elimination according to claim 3 includes the following steps: The FMM algorithm is based on a fast marching method. Considering the area to be repaired in the image, the algorithm starts from the boundary of the area and enters the interior of the area; first gradually fills in everything in the boundary and digs an area around the neighborhood pixels to be repaired. small neighborhood; the pixel is then replaced by the normalized weighted sum of all nearby known pixels, so the choice of weights is important, those pixels located near the point, near the boundary normal and those located on the boundary contour The pixel will get a greater weight; finally, after the pixel is repaired, it will be moved to the next closest pixel using the fast march method, and so on, so that the FMM algorithm can be passed based on the mask map with only straight line distractors. Eliminate interference in the original image.