CN113743523B - Building rubbish fine classification method guided by visual multi-feature - Google Patents

Abstract

A visual multi-feature guided construction waste fine classification algorithm comprises the following steps: step 1) collecting samples through visual sensors and preprocessing; step 2): extracting and encoding the color features of the samples based on the global information of the image; step 3): extracting and encoding the texture features of the samples based on the local information of the image; step 4): constructing the color-texture statistical features of the construction materials; step 5): inputting the color features and the color-texture statistical features into the classifier respectively, training the classification model, and obtaining two evidence matrices from different spaces; step 6): designing an effective decision model according to the evidence matrix, and outputting the decision matrix; step 7): determining the label of the material to be sorted according to the category with the highest probability in the decision matrix. The present invention can effectively make a decision classification model, determine the category label of the material, and realize accurate recognition of the target.

Description

A method for fine classification of construction waste guided by visual multi-features

Technical Field

The invention belongs to the technical field of pattern recognition and machine vision, and in particular relates to a method for finely classifying construction waste guided by visual multi-features.

Background technique

Most of the existing recycling equipment has gone through processes such as iron removal, crushing, screening, and magnetic separation, which are inefficient and have complicated operating procedures. At the same time, the existence of manual sorting increases the cost of resource recycling. In addition, the harsh working environment will affect the health of the staff. Therefore, establishing efficient and fast sorting equipment has become an important way to improve the utilization rate of construction waste.

With the development of machine vision technology in the past decade, pattern recognition theory has been applied to various industries with its unique advantages. In the field of construction waste identification, some scholars have proposed using color features to separate bricks and stones with similar densities. Some scholars have also proposed using volume, weight and other features to classify building materials. However, the above methods are all proposed for the classification of two types of building materials, with poor scalability and unable to meet the actual needs of industry for the classification of multiple substances.

Summary of the invention

In order to overcome the shortcomings of the above-mentioned prior art, the purpose of the present invention is to provide a method for fine classification of construction waste guided by visual multi-features. The method is based on pattern recognition and machine vision theory, extracts information of building materials and constructs new significant features, thereby realizing accurate classification of multiple types of materials.

In order to achieve the above object, the technical solution adopted by the present invention is:

A method for fine classification of construction waste guided by visual multi-features, comprising the following steps;

Step 1: Collect the target sample image of construction waste through the visual sensor and binarize it;

Step 2: Based on the global information of the image, extract the color features of the sample and encode them into

Step 3: Extract the texture features of the sample and encode them based on the local information of the image

Step 4: Construct color-texture statistical features of building materials

Step 5: and/> The features are input into the classifier to train the classification model. After the test image is classified by the model, two evidence matrices are output.

Step 6: Based on the evidence in step 5 Design decision model and output decision matrix/>

Step 7: According to the decision matrix The category with the highest probability determines the label of the material to be sorted.

The method for extracting sample color features and encoding in step 2 is:

First, the RGB image collected by the visual sensor is converted to the HSV color space, where H represents hue, S represents saturation, and V represents brightness. The image H channel information is extracted to represent the color characteristics of the object. The formula is as follows;

Wherein, max and min satisfy: max = max(R, G, B), min = min(R, G, B);

Then, the H channel information is histogram counted to obtain the color feature

The method for extracting and encoding the texture features of the sample image in step 3 is:

Step 3.1, extract local texture features:

First, the binary image in step 1 is divided into several sub-regions containing 32×32 pixels, the cell size is determined to be 16×16 pixels, the block size is 2×2 cells, and the grayscale values of the four neighborhood pixels of the pixel (x, y) are extracted: N(x, y+1), N(x, y-1), N(x+1, y), and N(x-1, y);

Then, the gradient magnitude and gradient direction of each pixel are calculated by the following formula;

Finally, the directional gradient histograms of the cells in the cascade block are concatenated, and a 1×36-dimensional vector is used to represent the distribution of the unsigned gradient direction of each sub-region;

Where _my and _mx are the vertical and horizontal gradients of pixel (x, y), respectively.

Step 3.2: Construct visual feature words:

Extract local texture features of each type of image according to step 3.1, then cluster the vectors of each type of image into K clusters using the K-means clustering algorithm, and regard the cluster centers as visual words;

Step 3.3, texture feature encoding:

The visual words of all categories in step 3.2 are used to form a visual word bag. For the vector calculated in step 3.1, the visual word is searched according to the nearest neighbor principle to encode the texture feature, and the result is

The fourth step is specifically to construct the color-texture statistical features of building materials. The formula is as follows:

In the formula, They are color features/> Texture features/> The fusion weights, [·] ^T is the transposed matrix, where />

In step 6, the classification probability matrix output by the classifier in step 5 is Design a classification decision model and output a decision matrix/> The method is:

Building materials are divided into color-significant categories (including red bricks and wood blocks) and others (including foam, hard plastic, and concrete) by color characteristics. Is a two-dimensional matrix, introducing the allocation matrix/> For the matrix/> Redistribute the classification probabilities in the input and introduce a linear fusion decision model to distribute the classification probabilities/> With/> The specific formula for fusion is as follows:

Where ω ₁ and ω ₂ are matrices and/> The weight of represents the confidence of the classification result from the two feature classification spaces;/> It is a fusion classification probability matrix, which is used to determine the final category of the material. The category of construction waste corresponding to the maximum probability is the final result.

When ω ₁ ∈(0,1), the algorithm is called Visual multi-feature guided construction waste classification algorithmbased on joint decision model (VMF-J), and ω ₁ = 0.5 is taken as the default value. When ω ₁ = 0, the final classification result is completely based on color-texture statistical features. The classification probability of Based on this, the algorithm is called Visual multi-feature guided construction waste classification algorithm (VMF). When ω ₁ = 1, only color features are used to classify the target. The average distribution principle is satisfied, but accurate category labels cannot be obtained at this time.

According to the decision matrix The category with the highest probability determines the final category of the material to be sorted.

Beneficial effects of the present invention:

The present invention uses machine vision technology to accurately classify various types of materials, providing solutions to the diversified needs of the market; by extracting features from local and global multi-angles of the target, a new color-texture statistical feature is constructed, which promotes the comprehensive description of material information; by establishing an efficient classification decision model, the recognition accuracy of materials is effectively improved; through in-depth analysis and intelligent extraction of significant features of the target, the rapid recognition of materials is promoted, the automated and intelligent sorting of building materials is realized, and the cost of recycling building materials is reduced.

BRIEF DESCRIPTION OF THE DRAWINGS

Figure 1 is a schematic diagram of some sample sets.

Figure 2 shows the construction of color-texture statistical features flow chart.

Figure 3 is a schematic diagram showing the effect of _ω1 value on classification accuracy.

FIG. 4 is a schematic diagram of the process of the present invention.

Detailed ways

The present invention is further described in detail below in conjunction with the embodiments.

As shown in Figures 1 to 4: The present invention discloses a method for fine classification of construction waste guided by visual multi-features, which specifically includes the following steps in conjunction with Figure 2:

Step 1: Use visual sensors to collect the five materials that account for the largest proportion of construction waste in my country, as shown in Figure 1, and pre-process the collected data. Then, divide each type of image into 5 parts, select 4 parts as training sets, and the remaining images are regarded as test sets.

Step 2: Based on the global information of the image, the specific method of extracting the color features of the sample and encoding them is:

First, the image is converted from RGB color space to HSV color space and the H channel information is extracted. Then, the H channel is histogram counted to obtain the color feature

Step 3: Based on the local information of the image, the specific method of extracting the texture features of the sample and encoding them is:

Step 3.1, extract image texture features. First, binarize the RGB image. Then, divide the binary image into several sub-regions containing 32×32 pixels. Determine the cell size of 16×16 pixels and the block size of 2×2 cells, and extract the grayscale values of the four neighborhood pixels of the pixel (x, y): N(x, y+1), N(x, y-1), N(x+1, y), and N(x-1, y). Then, calculate the gradient amplitude and gradient direction of each pixel using the following formula. Finally, concatenate the directional gradient histograms of the cells in the block, and use a 1×36-dimensional vector to represent the distribution of the unsigned gradient direction of each sub-region.

Where _my and _mx are the vertical and horizontal gradients of pixel (x, y), respectively.

Step 3.2: Construct a visual feature word bag. Extract local texture features of each type of image according to step 3.1. Then, cluster the vectors of each type of image into K categories using the K-means clustering algorithm, and regard the cluster centers as visual words. Finally, all categories of visual words form a visual word bag.

Step 3.3: Texture feature encoding. For the vector calculated in step 3.1, search for visual words in the bag of words according to the nearest neighbor principle and encode the texture features to form

Step 4: Color Features and texture features/> Fusion, construction of building material color-texture statistical features The specific method is:

For any image in the sample space S The color feature and texture feature are combined to construct a new color-texture statistical feature/> The formula is as follows:

In the formula They are sample images/> Color characteristics/> Texture features/> The fusion weights, [·] ^T is the transposed matrix, where/>

Step 5: and/> The features are input into the classifier respectively, the classification model is trained, and two classification probability matrices from different feature spaces are obtained.

Step 6: According to the matrix obtained in step 5 Introduce the linear fusion decision model and transform the classification probability matrix/> With/> The result of fusion is / > The specific method is as follows:

Step 6.1, design a classification decision model. Since the color feature alone can effectively classify building materials into color-significant categories (including red bricks and wood blocks) and others (including foam, hard plastic, and concrete), Is a two-dimensional matrix. In order to improve the classification accuracy of the algorithm, the distribution matrix is introduced. For the matrix/> Redistribute the classification probabilities in the input and introduce a linear fusion decision model to distribute the classification probabilities/> With/> The specific formula for fusion is as follows:

Among them, ω ₁ and ω ₂ are matrices and/> The weight of represents the confidence of the decision model in the evidence from the two spaces; /> is the fusion classification probability matrix, which is used to determine the final category of the material. The category of construction waste corresponding to the maximum probability is the final result. When ω ₁ = 0, the final classification result is completely based on color-texture statistical features/> The classification probability As a basis, the algorithm is called Visual multi-feature guided construction waste classification algorithm (VMF). When ω ₁ = 1, the color feature is used to classify the target. The average distribution principle is satisfied, so it is impossible to obtain accurate category labels.

Step 6.2, parameter tuning of the classification decision model. The effect of ω ₁ on the classification accuracy is shown in Figure 4. As ω ₁ increases, the recognition rate shows two trends: first, it increases first and then decreases, as shown in Figure 3 (3); second, it remains unchanged first and then decreases, as shown in Figure 3 (1)-(2). When ω ₁ ∈ (0,1), the algorithm is called a visual multi-feature guided construction waste classification algorithm based on joint decision model (VMF-J). Experiments have shown that when ω ₁ = 0.5, the classification accuracy of the model reaches the optimal value. Therefore, the present invention takes ω ₁ = 0.5 as the default value.

Step 7: According to the decision matrix The category with the highest probability determines the final category of the material to be sorted.

In view of the effectiveness of the fine classification algorithm of construction waste guided by visual multi-features, this paper tests the VMF and VMF-J algorithms in three different commonly used classifiers (SVM, RF, KNN), and the experimental results are shown in Table 1, Table 2, and Table 3. In addition, this paper compares the VMF and VMF-J algorithms with the Lab, RGB, and HSV threshold segmentation algorithms, and the experimental results are shown in Table 4.

Table 1 Confusion matrix of SVM in identifying construction waste

Table 2 Confusion matrix of KNN recognition of construction waste

Table 3 Confusion matrix of RF recognition of construction waste

Table 4 Comparison of algorithm recognition effects

Claims (5)

1. A visual multi-feature guided construction waste fine classification method is characterized by comprising the following steps of;

step one, acquiring a construction waste target sample image through a visual sensor, and binarizing the construction waste target sample image;

step two, extracting color characteristics of the sample based on global information of the image and encoding and constructing

Step three, extracting texture features of the sample based on local information of the image and encoding the texture features to form

Fourth, building color-texture statistical characteristics of building materials

Step five, willAnd/>The characteristics are respectively input into a classifier, a classification model is trained, and two evidence matrixes/>' are output after the test image passes through the classification model

Step six, according to the evidence in the step fiveDesigning a decision model, and outputting a decision matrix/>

Step seven, according to the decision matrixThe category of the highest probability determines the label of the material to be sorted.

2. The method for finely classifying construction waste guided by visual multi-feature according to claim 1, wherein the method for extracting and encoding the color features of the sample in the second step comprises the following steps:

Firstly, converting an RGB image acquired by a vision sensor into an HSV color space, wherein H represents hue, S represents saturation, V represents brightness, extracting image H channel information to represent color characteristics of an object, and the formula is as follows;

wherein max and min satisfy: max=max (R, G, B), min=min (R, G, B);

Then, carrying out histogram statistics on the H channel information to obtain color characteristics

3. The method for finely classifying construction waste guided by visual multi-feature according to claim 1, wherein the method for extracting and encoding texture features of the sample image in the third step comprises the following steps:

step 3.1, extracting local texture features:

firstly, dividing the binary image in the first step into a plurality of subregions containing 32 multiplied by 32 pixels, determining the size of the cells to be 16 multiplied by 16 pixels, extracting the gray values N (x, y+1), N (x, y-1), N (x+1, y) and N (x-1, y) of the pixels in the four adjacent domains of the pixels (x, y) of the cells with the size of 2 multiplied by 2 blocks;

then, the gradient magnitude and gradient direction of each pixel are calculated by the following formula;

Finally, the directional gradient histograms of the cells in the cascade blocks are used for representing the distribution of the unsigned gradient directions of each sub-region by using a vector of 1×36 dimensions;

where m _y and m _x are the vertical gradient and the horizontal gradient of pixel (x, y), respectively;

step 3.2, constructing visual characteristic words:

extracting local texture features of each type of image according to the step 3.1, then gathering vectors of each type of image into K types through a K-means clustering algorithm, and regarding a clustering center as a visual word;

Step 3.3, texture feature coding:

forming visual word bags by all the visual words in the step 3.2, searching the visual words according to the nearest neighbor principle for the vector calculated in the step 3.1 to encode texture features to obtain

4. The method for finely classifying construction waste guided by visual multi-feature according to claim 1, wherein the fourth step is specifically: construction of building material color-texture statisticsThe formula of (2) is as follows:

Wherein n ₁、n₂ is a color feature respectively Texture features/>[ ] ^T Is the transposed matrix, where n ₁＝n₂ = 1.

5. The fine classification method of building rubbish guided by visual multi-feature according to claim 1, wherein in the sixth step, the classification probability matrix is output according to the classifier in the fifth stepDesigning a classification decision model, and outputting a decision matrix/>The method of (1) is as follows:

building materials are classified into color conspicuous categories by color characteristics, thus Is a two-dimensional matrix, introduces a distribution matrix/>For matrix/>The classification probability in the model is redistributed, a linear fusion decision model is introduced, and the classification probability/>And/>The fusion is carried out, and the specific formula is as follows:

wherein omega ₁ and omega ₂ are matrices And/>Representing the degree of trust of the classification result on the classification space from the two features; /(I)The method is a fusion classification probability matrix and is used for judging the final class of the materials, and the building rubbish class corresponding to the maximum probability is the final result;

When ω ₁ is epsilon (0, 1), the construction waste combined decision classification algorithm called visual multi-feature guidance takes ω ₁ =0.5 as a default value, and when ω ₁ =0, the final classification result is completely characterized by color-texture statistics Classification probability/>Based on this, the construction waste fine classification algorithm, now called visual multi-feature guided, classifies the target with only color features when ω ₁ =1, due to/>The average allocation principle is satisfied, and at this time, an accurate class label cannot be obtained.