TWI774411B - Model compression method and model compression system - Google Patents

Abstract

A model compression method includes: performing a model pruning operation upon an original model with deep neural network architecture to generate a compressed model, feeding a same test data into the original model and the compressed model, estimating similarity between a first output data obtained from processing the test data by the original model and a second output data obtained from processing the test data by the compressed model, and determining how to further adjust the model pruning operation through reinforcement learning that uses the similarity as a reward.

Description

Model compression method and model compression system

The present invention relates to model compression, in particular to a model compression method and model compression system for adjusting model pruning operations through a reinforcement learning mechanism that uses similarity as a reward.

There is a technique in model compression to train a smaller model (student model) through the existing model (teacher model). The teacher model usually has a large number of parameters and is not easy to deploy on existing equipment, so this method is used to train a smaller model (student model). To train a small model with similar capabilities and deploy it to a mobile device, however, most of the methods still have to manually design the parameters of the student model, so there is an urgent need for a method that can automatically find a suitable student model (ie, the compressed model).

Therefore, one of the objectives of the present invention is to propose a model compression method and model compression system for performing model pruning operations through a reinforcement learning mechanism that uses similarity as a reward.

In an embodiment of the present invention, a model compression method is disclosed. The model compression method includes: performing a model pruning operation on an original model with a deep neural network structure to generate a compressed model; inputting the same test data into the original model and the compressed model respectively; calculating The similarity between a first output data obtained by the original model processing the test data and a second output data obtained by the compressed model processing the test data; and using the similarity as a reward, through reinforcement learning Determine how to further adjust the model pruning operation.

In another embodiment of the present invention, a model compression system is disclosed. The model compression system includes a storage device and a processor. The storage device is used for storing a code. The processor is used for loading and executing the code to perform the following operations: performing a model pruning operation on an original model having a deep neural network architecture to generate a compressed model; inputting the same test data respectively to the original model and the compressed model; calculating the similarity between a first output data obtained by processing the test data by the original model and a second output data obtained by processing the test data by the compressed model; and Using the similarity as a reward, reinforcement learning is used to determine how to further adjust the model pruning operation.

The model compression method of the present invention uses similarity as the basis for model pruning (model compression), so the user does not need to provide the marked data as the test data, which can reduce the cost and time of data marking. In addition, the user does not need to provide the test data The original code can be directly input into the model for compression, so it can effectively promote the application of model compression. Moreover, the generalization features left after compression are not prone to overfitting.

FIG. 1 is a schematic diagram of a model compression system according to an embodiment of the present invention. As shown in FIG. 1 , the model compression system 100 includes a processor 102 and a storage device 104 . The storage device 104 is used to store a program code Code_MC. For example, the storage device 104 may be a conventional hard disk, a solid-state hard disk, a memory, etc., but the invention is not limited thereto. The processor 102 can load and execute the code Code_MC to execute various steps in the model compression method shown in FIG. 2 .

Please refer to Figure 2 and Figure 3 together. FIG. 2 is a flowchart of a model compression method according to an embodiment of the present invention. FIG. 3 is a schematic diagram of the operation of the model compression method shown in FIG. 2 . Please note that if the same result can be obtained, the model compression method does not have to be executed in sequence according to the steps shown in Figure 2. In addition, according to design requirements and/or application requirements, the model compression method can also be modified and added. other steps. In step 202 , a model pruning operation is performed on an original model 302 having a deep neural network architecture to generate a compression model 304 . For example, the original model 302 may be a model obtained by training based on a convolutional neural network (CNN) architecture, and the goal of model pruning is to keep only important weights and delete less influential weights , in other words, compared with the original model 302, the compressed model 304 will have a smaller number of parameters, so the computational cost and storage space can be reduced, so that the compressed model 304 can be deployed to the product end with limited computing power , such as mobile phones, edge devices, etc. In addition, the model pruning operation of the present invention also hopes that the output of the compressed model 302 can be as close to the output of the original model 302 as possible, and further details will be described later.

In step 204, the same test data 308 is input to the original model 302 and the compressed model 304 for processing. In other words, based on the same test data 308, the output of the original model 302 and the output of the compressed model 304 can be used to evaluate the compressed model. Whether the model 304 is similar to the original model 302 .

In step 206, the similarity between the output data D1 obtained by processing the test data 308 in the original model 302 and the output data D2 obtained by processing the test data 308 in the compressed model 304 is calculated. Represents whether the output features of the pruned model are similar to the output features of the model before pruning.

In step 208, the similarity calculated in step 206 is used as a reward, and how to further adjust the model pruning operation is determined through reinforcement learning. For example, an agent of reinforcement learning can use a deep deterministic policy gradient (DDPG) algorithm to determine the action to take, where the action is used to select the parts to be compressed, and then To achieve the purpose of adjusting the model pruning operation. In other embodiments, other algorithms (eg, Truncated Natural Policy Gradient (TNPG) algorithm, Cross Entropy Method (CEM) algorithm, etc.) may also be used to determine the action to be taken.

For example, it is assumed that the original model 302 input by the user is a structure composed of three convolution layers (convolution layers) with channel sizes [32, 64, 128], respectively. The main body 306 initialized at the beginning gives the respective initial compression rates of the three layers [60%, 40%, 70%], therefore, the model compression method of the present invention uses reinforcement learning to compress the original model 302, so that the channel sizes of the compressed model 304 are [12, 38, 38] respectively, and the similarity is 0.3 , and then, the result with a similarity of 0.3 will be fed back to the main body 306, and the main body 306 will determine the next compression direction (for example, adjust the compression part), and adjust the desired compression rate of each layer based on the similarity and the model information. The compression operation compresses the original model 302 through the adjusted model pruning operation, so that the channel sizes of the compressed model 304 are [14, 32, 64] respectively, and the similarity is 0.4; the above model compression operation will Iterative execution is performed to obtain a compressed model 304 with higher similarity through reinforcement learning.

In one embodiment of the present invention, the model compression method can refer to the conventional framework of automatic machine learning based model compression (AutoML for Model Compression, AMC) to implement model pruning, but it is not limited thereto. The skilled person may know other various model compression methods, which will not be repeated here. The conventional model compression architecture uses the output of the compressed model as a reward, so that reinforcement learning can determine how to further compress the original model. Further, the conventional architecture uses accuracy as the reward for the main body of reinforcement learning. , in order to calculate the accuracy, the user needs to provide labeled data as the test data fed into the compressed model, so that the accuracy of the output of the compressed model can be known through the information provided by the labeling. However, , For users, the marking of data is quite time-consuming and labor-intensive. In addition, when calculating the accuracy, the maximum value in the output of the compressed model is generally compared with the marking. Therefore, the calculation of the accuracy does not care about compression at all. In addition to the maximum value in the output of the latter model, if the input data is difficult to judge, the values in the output of the compressed model will be very close to each other. Using the accuracy rate alone as the reward for the main body of reinforcement learning may cause the model Overconfidence and loss of judgment ability of some features, thus resulting in similar overfitting (overfitting) results or different outputs from the original model. Furthermore, in order to know which accuracy algorithm to use, the conventional architecture also The user is required to provide the test source code.

Compared with the conventional architecture that uses accuracy as the reward for the main body of reinforcement learning, the model compression method of the present invention uses similarity as the reward for the main body of reinforcement learning, and performs model pruning through reinforcement learning (such as DDPG algorithm). adjustment. The main purpose of model pruning (model compression) is to make the compressed model 304 similar to the original model 302 provided by the user. Therefore, the model compression method of the present invention can combine the output data D1 of the original model 302 with the compressed model 304 The similarity of the output data D2 is compared as a basis for model pruning (model compression).

In an embodiment of the present invention, the similarity can be obtained by calculating the Pearson's correlation coefficient between the output of the original model X and the output of the compressed model Y, for example:

Output matrix of X= [1.0, 2.0, 3.0]

Output matrix of Y= [2.0, 20.0, 38.0]

=1.0 Pearson's correlation coefficient ρ(X,Y) =

=1.0

In another embodiment of the present invention, the similarity can be obtained by calculating the cosine similarity between the output of the original model X and the output of the compressed model Y, for example:

Output matrix of X= [1.0, 2.0, 3.0]

Output matrix of Y= [2.0, 20.0, 38.0]

=0.9698612260388879 Cosine similarity (X,Y) =

=0.9698612260388879

However, the above is only used as an example and is not used as a limitation of the present invention. In fact, the model compression method of the present invention can also adopt other suitable similarity algorithms according to design requirements and/or application requirements. Variations also fall within the scope of the present invention.

As mentioned above, the calculation of the similarity is based on the respective output data D1 and D2 of the original model 302 and the compressed model 304. In this embodiment, if the original model 302 is a model obtained by training based on the convolutional neural network architecture , then the model pruning operation (the part to be compressed is selected by the action taken by the reinforcement learning main body 306) will only be applied to the convolutional layer, so the output data D1 and D2 can be the convolutional neural network architecture located in the volume The output of any layer after the stack. FIG. 4 is a schematic diagram of the convolutional neural network architecture of the original model 302 and the compressed model 304 shown in FIG. 3 . As shown in the figure, the convolutional neural network architecture 400 includes an input layer, convolutional layers 404_1 to 404_N (N≧1), pooling layers 406_1 to 406_N (N≧1), fully connected Layers (filled-connected layers) 408_1 to 408_M (M≧1) and an output layer (output layer) 410 . In an embodiment of the present invention, the output data D1 may be the output of a fully connected layer (eg, 408_i, 1≦i≦M) of the original model 302 , and the output data D2 may be the same fully connected layer of the compressed model 304 . (eg 408_i, 1≦i≦M) output. In another embodiment of the present invention, the output layer 410 is the last layer, and the Softmax function will be executed to make the sum of the probability distributions of the outputs of all nodes in the fully connected layer 408_M to be 1. In addition, the output data D1 may be the original model 302. The Softmax function output of the last layer of the model 304, and the output data D2 may be the Softmax function output of the last layer of the compressed model 304. Please note that the convolutional neural network architecture 400 shown in FIG. 4 is only used for illustrative purposes, not a limitation of the present invention. In practice, the model compression method of the present invention can also be applied to other neural network architectures. These design variations also fall within the scope of the present invention.

To sum up, the similarity calculation is based on the respective output data of the original model and the compressed model, so there is no need to compare the output data of the compressed model with the labels of the test data. As the reward for the main body of reinforcement learning, the user needs to provide the marked data as the test data. The model compression method of the present invention uses similarity as the reward for the main body of the reinforcement learning and can not use the marked data as the test data. (That is, the test data 308 is non-labeled data). Since the test data 308 does not include labels, the cost and time of data labeling can be reduced. In addition, the model compression method of the present invention adopts the calculation of similarity, so the user does not need to provide the original test code, and can directly input the model for compression, so the application of model compression can be effectively promoted. Furthermore, the model compression method of the present invention uses similarity as the reward for the main body of reinforcement learning, so the generalization features left after compression are not prone to overfitting. The above descriptions are only preferred embodiments of the present invention, and all equivalent changes and modifications made according to the scope of the patent application of the present invention shall fall within the scope of the present invention.

100: Model Compression System 102: Processor 104: Storage Device 202, 204, 206, 208: steps 302: Original Model 304: Compressed model 306: Subject 308: Test data 400: Convolutional Neural Network Architecture 402: Input layer 404_1, 404_N: Convolutional layer 406_1, 406_N: Pooling layer 408_1, 408_M: Fully connected layer 410: output layer Code_MC: code D1, D2: output data

FIG. 1 is a schematic diagram of a model compression system according to an embodiment of the present invention. FIG. 2 is a flowchart of a model compression method according to an embodiment of the present invention. FIG. 3 is a schematic diagram of the operation of the model compression method shown in FIG. 2 . Figure 4 is a schematic diagram of the convolutional neural network architecture of the original model and the compressed model shown in Figure 3.

202, 204, 206, 208: steps

Claims (12)

A model compression method, including: performing a model pruning operation on an original model having a deep neural network architecture to generate a compressed model; Input the same test data to the original model and the compressed model respectively; calculating a similarity between a first output data obtained by processing the test data by the original model and a second output data obtained by processing the test data by the compressed model; and Using the similarity as a reward, reinforcement learning is used to determine how to further adjust the model pruning operation. The model compression method of claim 1, wherein the test data is untagged data. The model compression method of claim 1, wherein the similarity is obtained by calculating a Pearson correlation coefficient between the first output data and the second output data. The model compression method of claim 1, wherein the similarity is obtained by calculating a cosine similarity between the first output data and the second output data. The model compression method of claim 1, wherein the first output data is an output of a fully connected layer of the original model, and the second output data is an output of a fully connected layer of the compressed model. The model compression method of claim 1, wherein the first output data is the Softmax function output of the last layer of the original model, and the second output data is the Softmax function of the last layer of the compressed model output. A model compression system comprising: a storage device for storing a code; and a processor to load and execute the code to perform the following operations: performing a model pruning operation on an original model having a deep neural network architecture to generate a compressed model; Input the same test data to the original model and the compressed model respectively; calculating a similarity between a first output data obtained by processing the test data by the original model and a second output data obtained by processing the test data by the compressed model; and Using the similarity as a reward, reinforcement learning is used to determine how to further adjust the model pruning operation. The model compression system of claim 7, wherein the test data is unlabeled data. The model compression system of claim 7, wherein the similarity is obtained by calculating a Pearson correlation coefficient between the first output data and the second output data. The model compression system of claim 7, wherein the similarity is obtained by calculating a cosine similarity between the first output data and the second output data. The model compression system of claim 7, wherein the first output data is an output of a fully connected layer of the original model, and the second output data is an output of a fully connected layer of the compressed model. The model compression system of claim 7, wherein the first output data is the Softmax function output of the last layer of the original model, and the second output data is the Softmax function of the last layer of the compressed model output.

Images