CN111539306A - A method for building recognition in remote sensing images based on the replaceability of activation expressions - Google Patents

Abstract

The invention discloses a remote sensing image building identification method based on activation expression replaceability, which comprises the following steps: acquiring a remote sensing image building data set; training a common deep neural network model; computing an independent maximum response map identifying each convolution kernel in the model; calculating the activation expression replaceability of each convolution kernel; pruning the model convolution kernels according to the activation expression replaceability of each convolution kernel, and keeping small activation expression replaceability; and carrying out remote sensing image building identification by using the trimmed deep neural network model. The method provides an activated expression replaceability index method for a building identification model based on deep learning, and can quantify the replaceability of each convolution kernel on the same layer in the expression of the feature space, the lower the activated expression replaceability value of the convolution kernel is, the less the convolution kernel is replaceable in the feature space, and further selective pruning is carried out on the convolution kernel, so that the identification precision of the building identification model of the remote sensing image is effectively improved.

Description

A method for building recognition in remote sensing images based on the replaceability of activation expressions

technical field

The invention belongs to the technical field of remote sensing image recognition, and relates to a remote sensing image building recognition method based on the replaceability of activation expression.

Background technique

In recent years, a large number of remote sensing satellites have also brought a large number of remote sensing images. Remote sensing imagery data is proliferating day by day, including multiple remote sensing imagery with different spectral and spatial resolutions. These remote sensing images bring enormous economic value. The ability to quickly extract buildings and other objects from remote sensing images can effectively help urban planning, infrastructure construction, and illegal building detection. At present, a large number of building recognition algorithms based on deep learning have emerged, but due to the lack of understanding of the generalization of the remote sensing image building recognition model, the overall recognition accuracy of the model is difficult to meet practical applications.

At present, there are two main methods for measuring the generalization ability of deep learning models. The first scheme mainly relies on traditional statistical learning theory methods, such as VC dimension, Rademacher complexity and other algorithms to explore the relationship between model robustness, complexity and generalization. These theories suggest that models with a large number of parameters are prone to overfitting on the data, but at the same time reduce the generalization ability on the test data. However, this conclusion is contrary to the performance of current deep learning models, and traditional statistical learning theories cannot reasonably explain the generalization ability of deep learning models. The second scheme mainly interprets and evaluates the generalization ability of the model from the changes in the parameter space during the optimization of the deep learning model. Schmidhuber believes that the generalization ability of the model is related to the flatness of the minima and the flatness of the Bayesian bound. However, Dinh pointed out that non-smooth minima deep learning models can actually generalize well. Wang linked the smoothness of the solution to the generalization ability of the model under the Bayesian framework, and theoretically proved that the generalization ability of the model is not only related to the Hessian spectrum, but also to the smoothness of the solution and the scale of the parameters. And the number of training samples is also related. In addition to this, the stochastic gradient descent algorithm for training deep learning models can also improve the generalization ability of the model. Many of the conclusions in the second scenario are also contradictory.

In practical applications, a rich and well-distributed test set of remote sensing image buildings is usually used to evaluate the generalization ability of the model. However, for this method, Recht proposed that a model with good performance on a specific test set may not reflect the generalization ability of the model itself, and the accuracy based on the test set is fragile and easily changed by small changes in the data distribution. The evaluation on the test set also suffers from inaccuracy. Therefore, there is currently no reasonable algorithm that can correctly measure the generalization of remote sensing image recognition models.

SUMMARY OF THE INVENTION

In view of this, the purpose of the present invention is to provide a remote sensing image building recognition method based on the replaceability of activation expression, and the present invention proposes an effective index for evaluating the generalization of a remote sensing image recognition model. The recognition model is pruned to improve the accuracy of the model, thereby improving the recognition ability of the model.

The purpose of the present invention is to realize in this way, the remote sensing image building identification method based on activation expression replaceability, comprises the following steps:

Step 1, obtaining a remote sensing image building data set;

Step 2, train a common deep neural network model;

Step 3, calculate and identify the independent maximum response map of each convolution kernel in the model;

Step 4, calculate the replaceability of the activation expression of each convolution kernel;

Step 5, trim the model convolution kernel according to the replaceability of activation expression of each convolution kernel, and retain the replaceability of small activation expression;

Step 6, use the pruned deep neural network model to identify buildings in remote sensing images.

Specifically, in the training process of the deep neural network model described in step 2, the training set is represented as D={X, y}, where X represents the nth remote sensing image, and y represents the building corresponding to the nth remote sensing image. Label, Θ={W, b} is the weight of the deep neural network to be trained, W represents the lth convolutional layer, b represents the bias on the lth layer, by defining the loss function of the recognition task and using the BP algorithm , and Θ ^★ is obtained, so that the model can obtain a high recognition accuracy rate on the dataset D while maintaining a small error value statically.

Further, in the calculation process of the independent maximum response graph described in step 3, its objective function is

Among them, the initial input X is random noise, ^Θ is the trained weight, J represents the number of all convolution kernels in the l layer, and the output of each convolution kernel is h _{l, i} (X, Θ ^* ), which is Represents the output of the ith convolution kernel on the lth layer, h _{l, -i} (X, Θ ^* ) represents other feature maps that remove the output activation values of the target i convolution kernels, and arg max(*) represents the output Maximum response map, X ^* represents the independent maximum response map of the final output;

Fix the corresponding weights, use the gradient ascent algorithm to iteratively update X ^* , so that it can maximize the output activation value of the ith convolution kernel of the lth layer, and the X ^* obtained by the objective function can make the output of the target convolution kernel as far as possible At the same time, it also ensures that the overall output of other convolution kernels is small, and the final X ^* is the independent maximum response map of the corresponding convolution kernel in the feature space.

Further, the calculation of described activation expression replaceability comprises the following steps:

In step 401, the expression replaceability is calculated, and the formula for expressing the replaceability is as follows

RS(l, i) represents the feature that the disentangled feature of the ith convolution kernel in the lth layer can be replaced by other convolution kernels in the same layer, where |{x _l }| represents the total number of convolution kernels in the lth layer , IAM(l, i) represents the feature representation of the independent maximum response map generated by the ith convolution kernel of the lth layer, f(IAM(l, i)) represents forwarding the generated independent maximum response map feature representation The activation value of the i-th convolution kernel in the l-th layer obtained by propagation; |{x _{l, j} : x _{l, j} > x _{l, i} }| indicates that in the l-th layer, the activation value is greater than the j-th convolution kernel activation value The number of convolution kernels; expression replaceability quantifies the replaceability of convolution kernels expressed on the same layer, and its metric value ranges from [0, 1];

In step 402, the activated expression replaceability is calculated, and the activated expression replaceability is defined as:

AR(l, i) represents the proportion of the activation value of the corresponding convolution kernel that is not 0, and the replacement of the activation expression represents the replacement of the expression of the effective activation value output by the target convolution kernel in the feature space sex.

For the current remote sensing image building recognition model, the generalization upper bound obtained by traditional theoretical methods has limited evaluation of the model. They cannot specifically quantify the generalization ability of deep learning models. The method of setting the test set requires more skills to ensure the balance of the data distribution in the test set, and it is difficult to truly reflect the performance of the model on unseen data. Therefore, there is a need for an indicator to quantify the generalization of the model, which can directly compare the generalization ability of the model without using the test set. To improve the identification of buildings. In this regard, the main structure convolution kernel of the building recognition model is used to directly quantify the generalization ability of the model. Each convolution kernel has the ability to extract features. The final recognizer completes the reasoning of unseen data by identifying the extracted features, which is the embodiment of generalization. Numerous analyses in parameter space also verify the importance of convolution kernels for model generalization. The performance of the convolution kernel in the feature space also belongs to the performance of the generalization ability of the model. Naturally, when the convolution kernel can present richer feature expressions in the feature space, these rich features will be more conducive to predicting new unseen data. A measure of the richness of features acquired by a model has important implications for evaluating generalization. The method of the invention can obtain the richness of features by measuring the convolution kernel, so as to further quantify the generalization ability of the model, so as to prune the model, and effectively improve the recognition accuracy of the model for remote sensing image buildings.

Description of drawings

Fig. 1 is the schematic flow chart of the method of the present invention;

Fig. 2 is a schematic diagram of the alternative expression of activation in the method embodiment of the present invention.

Detailed ways

The present invention will be further described below in conjunction with the embodiments and the accompanying drawings, but the present invention is not limited in any way, and any transformation or replacement based on the teachings of the present invention belongs to the protection scope of the present invention.

As shown in Fig. 1, the method for identifying buildings in remote sensing images based on the replaceability of activation expression includes the following steps:

Step 1, obtaining a remote sensing image building data set;

Step 2, train a common deep neural network model;

Step 3, calculate and identify the independent maximum response map of each convolution kernel in the model;

Step 4, calculate the replaceability of the activation expression of each convolution kernel;

Step 5, trim the model convolution kernel according to the replaceability of activation expression of each convolution kernel, and retain the replaceability of small activation expression;

Step 6, use the pruned deep neural network model to identify buildings in remote sensing images.

In a remote sensing image building recognition task, the training set is assumed to be D={X, y}. X represents the nth remote sensing image, and y represents the building label corresponding to the nth remote sensing image. and the weights Θ={W, b} of the deep neural network that needs to be trained. W represents the lth convolutional layer, and similarly b represents the bias on the lth layer. By defining the loss function of the recognition task and using the BP algorithm, Θ ^★ can be obtained so that the model can obtain a high recognition accuracy rate on the dataset D while maintaining a small error value. After the model converges, for each new input image, the image can be converted into a corresponding feature vector by processing the convolution kernel. The recognizer at the end of the model can correctly recognize the image.

Each convolution kernel will respond differently to different features. The output of each convolution kernel is defined as h(X, θ), which represents the output of the ith convolution kernel on the lth layer. The maximum response graph algorithm proposed by Erhan et al. hopes to obtain the feature that can cause the largest output response of the convolution kernel. The formula is defined as follows:

X ^* = arg max h _{l, i} (X, Θ ^* )

The initial input X is random noise. For trained weights Θ ^★ . The corresponding weights are fixed, and the gradient ascent algorithm is used to iteratively update X ^* so that it can maximize the output activation value of the i-th convolution kernel in the l-th layer. The resulting X ^* represents the visual representation that elicits the maximum response from the convolution kernel.

However, in the maximum response graph algorithm, only maximizing the activation value of the target convolution kernel is considered. We found that the obtained visual representation also causes other convolution kernels in the same layer to produce high responses, that is to say, the obtained visual representation is entangled with the expressions of other convolution kernels in features. The generated images do not well represent the representation of the corresponding convolution kernels in the feature space. Therefore, in order to better obtain the disentangled features of the convolution kernel in the feature space, this embodiment modifies the target of the maximum response graph algorithm. While maximizing the output activation value of the target convolution kernel, try to ensure that other convolution kernels get less input.

Further, in the calculation process of the independent maximum response graph described in step 3, its objective function is

Among them, the initial input X is random noise, ^Θ is the trained weight, J represents the number of all convolution kernels in the l layer, and the output of each convolution kernel is h _{l, i} (X, Θ ^* ), which is Represents the output of the i-th convolution kernel on the l-th layer, h _{l, i} (X, Θ ^* ) represents other feature maps excluding the output activation values of the target i convolution kernels, arg max(*) represents the maximum output value Response map, X ^* represents the independent maximum response map of the final output;

Fix the corresponding weights, use the gradient ascent algorithm to iteratively update X ^* , so that it can maximize the output activation value of the ith convolution kernel of the lth layer, and the X ^* obtained by the objective function can make the output of the target convolution kernel as far as possible At the same time, it also ensures that the overall output of other convolution kernels is small, and the final X ^* is the independent maximum response map of the corresponding convolution kernel in the feature space.

For building recognition from remote sensing images, the model relies on a variety of features to predict new data. When the prediction result of the new remote sensing image is good, it shows that the model has strong generalization ability. Therefore, by measuring the characteristics of the expression of the model in the feature space, it can help us measure the generalization ability of the model.

Currently, the disentangled features of each convolution kernel can be obtained through the independent maximum response graph algorithm. However, different convolution kernels may produce repeated expressions, and there are other convolution kernels that have a higher response to the disentanglement feature. The method of the present invention proposes a Representational Substitution (RS) method to quantify the repeatability of the expression of other convolution kernels, that is, the replacement of the expression of the convolution kernel in the feature space.

The disentangled features of the target convolution kernel obtained using the independent maximum response graph algorithm are forwarded to the layer where the target convolution kernel is located. If the activation value of the target convolution kernel in the corresponding layer is the largest, then the feature representing the convolution kernel cannot be replaced by other convolution kernels. On the contrary, when the activation value of other convolution kernels in the same layer is greater than the activation value of the target convolution kernel, the features representing the target convolution kernel can be replaced. The replaceable feature loss also has other convolution kernels expressing the feature, so the convolution kernel is not important.

Further, the calculation of the replaceability of the activation expression includes steps 401 and 402 .

In step 401, the expression replaceability is calculated, and the formula for expressing the replaceability is as follows

RS(l, i) represents the feature that the disentangled feature of the ith convolution kernel in the lth layer can be replaced by other convolution kernels in the same layer, where |{x _l }| represents the total number of convolution kernels in the lth layer , IAM(l, i) represents the feature representation of the independent maximum response map generated by the ith convolution kernel of the lth layer, f(IAM(l, i)) represents forwarding the generated independent maximum response map feature representation The activation value of the i-th convolution kernel in the l-th layer obtained by propagation; |{x _{l, j} : x _{l, j} > x _{l, i} }| indicates that in the l-th layer, the activation value is greater than the j-th convolution kernel activation value The number of convolution kernels; expression replaceability quantifies the replaceability of the expression of the convolution kernels on the same layer, and its metric value ranges from [0, 1].

A convolution kernel with low expression replaceability can indicate that the expression is not easily replaceable. And when the expression replaceability is high, there are two meanings. One is that the expression of the convolution kernel is easily replaced by other convolution kernels. The other means that the convolution kernel did not learn any features. Because when the target convolution kernel does not respond to any feature, according to the independent maximum response graph formula, f(x) is close to 0, and the result is that the average response value of other convolution kernels is the smallest. So these convolution kernels will produce almost similar results. However, the convolution kernel does not learn any features, which will also cause a higher response of other convolution kernels and make the expression replaceability value of the convolution kernel higher. So expressing replaceability also means that the convolution kernel does not learn features.

The present invention further proposes Activated Representational Substitution (ARS), which can uniformly represent the replaceability of convolution kernel expressions. According to the independent maximum response map formula, the output of the target convolution kernel is 0 when no features have been learned. Use expression replaceability to represent the activation of the convolution kernel, representing the proportion of non-zero values in the output feature map.

Step 402, combining the expression replaceability of the convolution kernel and the activation response value, calculate the expression replaceability of activation, and the expression replaceability of activation is defined as:

AR(l, i) represents the proportion of the activation value of the corresponding convolution kernel that is not 0, and the replacement of the activation expression represents the replacement of the expression of the effective activation value output by the target convolution kernel in the feature space sex.

The convolutional kernel with low replacement of activated expressions means that the expressions activated on the same layer are less likely to be replaced, which is important for the generalization of the model. As the value of ARS becomes larger, it means that the convolution kernel changes from repeated expression to meaningless expression. Convolutional kernels with high ARS are not important for model generalization.

The realization principle of Activated Representation Replaceability (ARS) is to measure the feature richness of the remote sensing image building model, so as to measure the generalization characteristics of the model, and improve the recognition results of the recognition model in a targeted manner. This method can also improve the recognition accuracy of remote sensing image building models. A schematic diagram of the principle of Activated Representation Replaceability (ARS) is shown in Figure 2, where grayscale and shape represent different features learned by the deep learning model of remote sensing images. Because the number of model convolution kernels is fixed, when the expression of each convolution kernel is not easily replaced by the expression of other convolution kernels, there can be richer features for the entire model. Therefore, the expression replaceability of the activation of the convolution kernel has a very strong relationship with the generalization of the model.

As shown in Figure 2, the input images are buttons and faces composed of different features. In Model 1, all three convolution kernels show expressions that other convolution kernels are not easy to replace, that is, the expression replaceability of the activation of each convolution kernel is low, so that the input image can be correctly recognized. However, the model 2 convolution kernels have similar expressions, and the activation expressions of some convolution kernels are highly replaceable and cannot correctly recognize faces.

It can be seen from the content of the invention and the embodiments that the method of the present invention proposes an activated expression replaceability index method for a building recognition model based on deep learning, which can quantify the replaceability of each convolution kernel on the same layer expressed in the feature space. . The more important the activation of the convolution kernel, the lower the expression replaceability value, which means that it is irreplaceable in the feature space. This method can selectively prune the convolution kernel, thereby effectively improving the recognition accuracy of the remote sensing image building recognition model. .

Claims (4)

1. The method for identifying the replaceable remote sensing image building based on the activation expression is characterized by comprising the following steps of:

step 1, obtaining a building data set of a remote sensing image;

step 2, training a common deep neural network model;

step 3, calculating and identifying an independent maximum response graph of each convolution kernel in the model;

step 4, calculating the activation expression replaceability of each convolution kernel;

step 5, pruning the model convolution kernels according to the activation expression replaceability of each convolution kernel, and keeping small activation expression replaceability;

and 6, using the trimmed deep neural network model to identify the remote sensing image building.

2. A remote sensing image building identification method according to claim 1, characterized in that in the training process of the deep neural network model in step 2, the training set is represented as D ═ { X, y }, X represents the nth remote distanceThe method comprises the steps of obtaining a perception image, wherein y represents a building label corresponding to an nth remote sensing image, theta is { W, b } represents weight of a deep neural network needing training, W represents an ith convolutional layer, b represents bias on the ith layer, and theta is obtained by defining a loss function of an identification task and using a BP algorithm^*So that the model can be made static with small error values while achieving a high recognition accuracy on the data set D.

3. A method for identifying buildings based on remote sensing images as claimed in claim 2, wherein the objective function of the independent maximum response graph in the step 3 is

Wherein, X of the initial input is random noise theta^★For the trained weights, J represents the number of all convolution kernels in the l layers, and the output of each convolution kernel is h_l，i(X，Θ^*) Which represents the output of the ith convolution kernel at the l-th layer, h_l，-i(X，Θ^*) Other feature maps representing the activation values of the output excluding the target i convolution kernels, argmax (X) representing the maximum response map of the output, X^*Then the independent maximum response graph of the final output is represented;

fixing corresponding weight value, using gradient rising algorithm to iteratively update X^*So that it can make the output activation value of ith convolution kernel of l layer be maximum and X obtained by target function^*The target convolution kernel output can be enabled to be as large as possible, and simultaneously, the integral output of other convolution kernels is ensured to be small, and the final X is obtained^*Then the independent maximum response map in feature space for the corresponding convolution kernel.

4. A remote sensing image building identification method as claimed in claim 3, characterized in that said calculation of the alternative of the activation expression comprises the following steps:

step 401, calculating the expression replaceability, wherein the formula of the expression replaceability is as follows

RS (l, i) represents the property that the unwrapping feature of the ith convolution kernel of the ith layer can be replaced by other convolution kernels on the same layer, wherein | { x_lL represents the total number of the ith layer of convolution kernels, IAM (l, i) represents the characteristic representation of an independent maximum response graph generated by the ith layer of convolution kernels, and f (IAM (l, i)) represents the activation value of the ith layer of convolution kernels obtained by forward propagation of the characteristic representation of the generated independent maximum response graph; i { x_l，j：x_l，j＞x_l，iThe j represents the number of convolution kernels which are larger than the j activation value of the convolution kernel in the l layer; expression replaceability quantifies the replaceability of the convolution kernel expression on the same layer, and the metric value ranges from 0 to 1]；

Step 402, calculating the expression replaceability of activation, wherein the expression replaceability of activation is defined as:

AR (l, i) represents the ratio of the activation values of the corresponding convolution kernels, wherein the ratio is not a 0 value, and the expression replaceability of activation represents the replaceability of the expression of the effective activation value output by the target convolution kernel in the feature space.

Images