HK40048702B - Method and computer system for compressing neural network, and storage medium - Google Patents

Description

Methods, computer systems, and storage media for compressing neural network models

Cross-references to related applications

This application claims priority to U.S. Provisional Patent Application No. 62/964,996, filed January 23, 2020, with the U.S. Patent and Trademark Office, and U.S. Patent Application No. 17/088,061, filed November 3, 2020, with the U.S. Patent and Trademark Office, the entire contents of which are incorporated herein by reference.

Technical Field

This disclosure generally relates to the field of data processing, and more particularly to neural networks.

Background Technology

The International Organization for Standardization (ISO), the International Electrotechnical Commission (IEC), the Moving Picture Experts Group (MPEG) (JTC 1/SC 29/WG 11) have been actively seeking potential standardization needs for future video codec technologies for visual analysis and understanding. In 2015, ISO adopted the Compact Descriptors for Visual Search (CDVS) standard as a still image standard, extracting feature representations for image similarity matching. The CDVS standard is listed as Part 15 of MPEG 7 and ISO/IEC 15938-15 and finalized in 2018. This standard extracts global and local, manually designed, feature descriptors based on deep neural networks (DNNs) from video clips. The success of DNNs in numerous video applications such as semantic classification, object detection/recognition, object tracking, and video quality enhancement has created a strong demand for compressed DNN models.

Summary of the Invention

This disclosure relates to methods, systems, and computer-readable storage media for compressing neural network models, which can compress neural network models and improve the computational efficiency of neural network models.

According to one aspect, a method for compressing a neural network model is provided. The method may include reordering at least one index corresponding to a multidimensional tensor associated with the neural network; determining a set of weight coefficients associated with the at least one reordered index; and compressing the neural network model based on the determined set of weight coefficients.

According to another aspect, a computer system for compressing a neural network model is provided. The computer system may include a reordering module for reordering at least one index corresponding to a multidimensional tensor associated with the neural network. A set of weight coefficients associated with the at least one reordered index is determined. The model of the neural network is compressed based on the determined set of weight coefficients.

According to another aspect, a non-volatile computer-readable medium is provided for compressing a neural network model. The non-volatile computer-readable medium may include a processor and a memory; the memory stores a computer program that, when executed by the processor, causes the processor to execute program instructions stored on at least one computer-readable storage device and at least one of at least one tangible storage devices, the program instructions being executable by the processor. The program instructions, executable by the processor, are used to perform a method that may accordingly include reordering at least one index corresponding to a multidimensional tensor associated with the neural network; determining a set of weight coefficients associated with the at least one reordered index; and compressing the neural network model based on the determined set of weight coefficients.

The method, system, and computer-readable storage medium for compressing neural network models provided in this disclosure can improve the efficiency of compressing learning weight coefficients, thereby accelerating the calculation of optimized weight coefficients. This can significantly compress neural network models and improve the computational efficiency of neural network models.

Attached Figure Description

These and other objects, features, and advantages will become apparent from the following detailed description of illustrative embodiments, which will be read in conjunction with the accompanying drawings. Since the drawings are intended to facilitate clear understanding by those skilled in the art in conjunction with the detailed description, various features in the drawings are not drawn to scale. In the drawings:

Figure 1 illustrates a networked computer environment according to at least one embodiment;

Figure 2 is a block diagram of a neural network model compression system according to at least one embodiment;

Figure 3 shows an operation flowchart of the steps performed by a program of a compressed neural network model according to at least one embodiment;

Figure 4 is a block diagram of the internal and external components of the computer and server depicted in Figure 1 according to at least one embodiment;

Figure 5 is a block diagram of an exemplary cloud computing environment including the computer system depicted in Figure 1 according to at least one embodiment; and

Figure 6 is a block diagram of the functional layers of an exemplary cloud computing environment of Figure 5 according to at least one embodiment.

Detailed Implementation

This document discloses detailed embodiments of the claimed structures and methods; however, it is understood that the disclosed embodiments are merely illustrative of the claimed structures and methods that can be implemented in various forms. These structures and methods can be implemented in many different forms and should not be construed as limited to the exemplary embodiments set forth herein. Rather, these exemplary embodiments are provided to make this disclosure clear and complete, and to fully convey the scope to those skilled in the art. Details of well-known features and techniques may be omitted in the description to avoid unnecessarily obscuring the presented embodiments.

This disclosure generally relates to the field of data processing, and more particularly to neural networks. The exemplary embodiments described below provide systems, methods, and computer programs for compressing neural network models. Therefore, some embodiments have the ability to improve computational efficiency by allowing for increased compression efficiency of learned weight coefficients, which can significantly reduce the size of deep neural network models.

As mentioned earlier, ISO/IEC MPEG (JTC 1/SC 29/WG 11) has been actively seeking potential standardization needs for future video codec technologies for visual analysis and understanding. In 2015, ISO adopted the CDVS standard as a still image standard, which extracts feature representations for image similarity matching. The CDVS standard is listed as Part 15 of MPEG 7 and ISO/IEC 15938-15 and was finalized in 2018. This standard extracts global and local, manually designed, DNN-based feature descriptors from video segments. The success of DNNs in numerous video applications such as semantic classification, object detection/recognition, object tracking, and video quality enhancement has created a strong demand for compressed DNN models.

Therefore, MPEG is actively working on the coded representation of the Neural Network standard (NNR), which encodes DNN models to save storage and computation. Several methods exist for learning compact DNN models. The goal is to remove unimportant weight coefficients, assuming that the smaller the value of the weight coefficient, the less important they are. Several network pruning methods have been proposed to explicitly achieve this goal, either by adding sparsity-enhancing regularization terms to the network training objective or by greedily removing network parameters. From the perspective of compressing DNN models, after learning a compact network model, the weight coefficients can be further compressed by quantization followed by entropy encoding/decoding. Such further compression can significantly reduce the storage size of DNN models, which is essential for deploying models on mobile devices, chips, etc.

Weight uniform regularization can improve compression efficiency in subsequent compression processing. An iterative network retraining/correction framework is used to jointly optimize the original training objective and the weight uniform loss, which includes compression ratio loss, uniform distortion loss, and computation speed loss. This ensures that the learned network weight coefficients maintain the original target performance, are suitable for further compression, and accelerate the computation of the learned weight coefficients. The proposed method can be applied to compressing original pre-trained DNN models. It can also be used as an additional processing module to further compress any pruned DNN model.

Unified regularization can improve the efficiency of further compressing the weight coefficients during learning, thereby accelerating the computation of optimized weight coefficients. This can significantly reduce the size of the DNN model and accelerate inference computation. Through an iterative retraining process, the performance of the original training objective can be maintained, allowing for both compression and computational efficiency. The iterative retraining process also provides the flexibility to introduce different losses at different times, allowing the system to focus on different objectives during the optimization process. The methods, computer systems, and computer programs disclosed in this paper can generally be applied to datasets with different data formats. The input/output data is typically a 4D tensor, which can be a real video clip, an image, or an extracted feature map.

This document describes aspects with reference to flowchart illustrations and/or block diagrams of methods, apparatus (systems), and computer-readable media according to various embodiments. It should be understood that each block of the flowchart illustrations and/or block diagrams, and combinations of blocks in the flowchart illustrations and/or block diagrams, can be implemented by computer-readable program instructions.

Referring now to Figure 1, a functional block diagram of a networked computer environment illustrates a neural network model compression system 100 (hereinafter referred to as the "System") for compressing neural network models. It should be understood that Figure 1 is merely an illustration of one embodiment and does not imply any limitation on the environment in which different embodiments may be implemented. Many modifications can be made to the depicted environment based on design and implementation requirements.

System 100 may include computer 102 and server computer 114. Computer 102 may communicate with server computer 114 via communication network 110 (hereinafter referred to as "network"). Computer 102 includes processor 104 and software program 108 stored on data storage device 106, and is capable of interfacing with a user and communicating with server computer 114. As will be discussed below with reference to FIG4, computer 102 may include internal component 800A and external component 900A, and server computer 114 may include internal component 800B and external component 900B, respectively. Computer 102 may be, for example, a mobile device, telephone, personal digital assistant, netbook, laptop computer, tablet computer, desktop computer, or any type of computing device capable of running programs, accessing networks, and accessing databases.

As discussed below in conjunction with Figures 5 and 6, server computer 114 can also operate in cloud computing service models, such as Software as a Service (SaaS), Platform as a Service (PaaS), or Infrastructure as a Service (IaaS). Server computer 114 can also reside in cloud computing deployment models, such as private clouds, community clouds, public clouds, or hybrid clouds.

The server computer 114, used for compressing neural network models, is capable of running a neural network model compression program 116 (hereinafter referred to as the "program") that interacts with the database 112. The neural network model compression program method will be explained in more detail below with reference to Figure 3. In one embodiment, the computer 102 may operate as an input device including a user interface, while the program 116 may run primarily on the server computer 114. In an alternative embodiment, the program 116 may run primarily on at least one computer 102, while the server computer 114 may be used to process and store the data used by the program 116. It should be noted that the program 116 may be a standalone program or may be integrated into a larger neural network model compression program.

However, it should be noted that in some instances, processing of program 116 can be shared between computer 102 and server computer 114 in any proportion. In another embodiment, program 116 may operate on more than one computer, server computer, or a combination of computers and server computers, for example, multiple computers 102 communicating with a single server computer 114 via network 110. In another embodiment, for example, program 116 may operate on multiple server computers 114 communicating with multiple client computers via network 110. Alternatively, the program may run on a network server that communicates with servers and multiple client computers via a network.

Network 110 may include wired connections, wireless connections, fiber optic connections, or a combination thereof. Typically, network 110 may be any combination of connections and protocols that support communication between computer 102 and server computer 114. Network 110 may include various types of networks, such as, for example, Local Area Network (LAN), Wide Area Network (WAN) (e.g., the Internet), telecommunications networks (e.g., Public Switched Telephone Network (PSTN)), wireless networks, public switched networks, satellite networks, cellular networks (e.g., the fifth generation (5G), Long Term Evolution (LTE), the third generation (3G), Code Division Multiple Access (CDMA), etc.), Public Land Mobile Network (PLMN), Metropolitan Area Network (MAN), private networks, self-organizing networks, intranets, fiber-optic networks, etc., and/or combinations of these or other types of networks.

The number and arrangement of devices and networks shown in Figure 1 are provided as examples. In practice, additional devices and/or networks, fewer devices and/or networks, different devices and/or networks, or devices and/or networks with different arrangements than those shown in Figure 1 may exist. Furthermore, the two or more devices shown in Figure 1 may be implemented in a single device, or the single device shown in Figure 1 may be implemented as multiple distributed devices. Additionally or alternatively, a group of devices in system 100 (e.g., at least one device) may execute at least one function, which is described as being executed by another group of devices in system 100.

Referring now to Figure 2, a neural network model compression system 200 is described. The neural network model compression system 200 can be used as a framework for an iterative learning process. The neural network model compression system 200 may include a unified index order and method selection module 202, a weight unification module 204, a network forwarding calculation module 206, a target loss calculation module 208, a gradient calculation module 210, and a backpropagation and weight update module 212.

Let denot be the dataset that assigns the objective y to the input x. Let Θ = {w} denote the set of weight coefficients of the DNN. The goal of training the neural network is to learn the optimal set of weight coefficients Θ*, thereby minimizing the objective loss. For example, in previous network pruning methods, the objective loss has two parts: empirical data loss and sparsity-promoting regularization loss ￡ _R (Θ):

Where _λR ≥0 is a hyperparameter that balances the contributions of data loss and regularization loss.

Sparsity facilitates regularization loss by placing regularization over the entire set of weights, and the resulting sparse weights have a weak correlation with inference efficiency or computational speedup. From another perspective, after pruning, the sparse weights can undergo another network training process where an optimal set of weights can be learned, which can improve the efficiency of further model compression.

This disclosure proposes a weighted uniform loss that is optimized together with the original target loss:

Where _λU ≥ 0 is a hyperparameter balancing the contributions of the original training objective and weight unification. Joint optimization yields an optimal set of weight coefficients, which significantly improves the efficiency of further compression. The weighted unification loss considers how convolution operations are performed as a fundamental process of GEMM matrix multiplication, resulting in optimized weighting coefficients that greatly accelerate computation. Notably, our weighted unification loss can be viewed as an additional regularization term to the general objective loss, with (when _λR > 0) or without (when _λR = 0) general regularization. Moreover, our method is flexible enough to be applied to any regularized loss _εR (Θ).

In at least one embodiment, the weight unification loss £ _U (Θ) further includes a compression ratio loss £ _C (Θ), a unification distortion loss £ _I (Θ), and a computation speed loss £ _S (Θ):

￡ _U (Θ)＝￡ _I (Θ)+λ _C ￡ _C (Θ)+λ _S ￡ _S (Θ),

Detailed explanations of these loss terms will be provided in later sections. For both learning effectiveness and learning efficiency, an iterative optimization process is further proposed. The partial weight coefficients that satisfy the desired structure can be fixed. The non-fixed parts of the weight coefficients can be updated by backpropagating the training loss. By iteratively performing these two steps, more and more weights can be gradually determined, and the joint loss can be effectively and progressively optimized.

Moreover, in at least one embodiment, each layer is compressed individually, and can be further written as:

Where _LU ( ^Wj ) is the uniform loss defined on the j-th layer; N is the total number of layers for measuring quantization loss; and ^Wj represents the weight coefficient of the j-th layer. Similarly, since _LU ( ^Wj ) is calculated independently for each layer, script j is omitted in the remainder of this disclosure without loss of generality.

For each network layer, its weight coefficients W are general 5D tensors of size ( _ci , _k1 , _k2 , _k3 , _co ). The input to the layer is a 4D tensor A of size ( _hi , _wi , _di , _ci ), and the output of the layer is a 4D tensor B of size ( _ho , _wo , _do , _co ). Sizes ci, _{k1 , k2 , k3} , _co , _hi , _wi , _di , _ho , _wo , and _do are integers greater than or equal to 1. When any of the sizes _ci , _k1 , _k2 , _k3 , _co , hi, _wi , _di , _ho , _wo , _{and do} are 1, the corresponding tensor is reduced to a lower dimension. Each item in each tensor is a floating-point number. Let M represent a 5D binary mask of the same size as W, where each term in M is a binary number 0 or 1, indicating whether the corresponding weight coefficient is pruned or retained. M is introduced in association with W to address the case where W comes from a pruned DNN model, where some connections between neurons in the network are removed from computation. When W comes from the original, unpruned pre-trained model, all terms in M are set to 1. The output B is computed via a convolution operation ⊙ based on A, M, and W.

Parameters h _{_i}, w_{_i}, and _di (h _₀ , w _{_o}, and do_{_o}) are the height, weight, and depth of the input tensor A (output tensor B). Parameter c_{_i}(co_{_o} ) is the number of input (output) channels. Parameters k _₁, k _₂ , and k_₃ are the sizes of the convolution kernels corresponding to the height, weight, and depth axes, respectively. That is, for each output channel v = 1, ..., co_{_o} , this operation can be viewed as convolving with the input A into a 4D weighted tensor W_v of size (c _{_i} , k₁, k _₂ , k_₃ ).

The order of the summation operations can be changed. A 5D weighted tensor can be reshaped into a 3D tensor of size ( _ci , _co , k), where k = _k1 · _k2 · _k3 . The order of the reshaping indices along the k-axis is determined by the reshaping algorithm during the reshaping process, which will be described in detail later.

The desired structure of the weight coefficients can be designed by considering two aspects. First, the structure of the weight coefficients should be consistent with the basic GEMM matrix multiplication process used to implement convolution operations, in order to accelerate inference computation using the learned weight coefficients. Second, the structure of the weight coefficients can contribute to improving the efficiency of quantization and entropy encoding/decoding for further compression. In at least one embodiment, a block-wise structure of weight coefficients in each layer can be used in the 3D reshaping of the weight tensor. Specifically, the 3D tensor can be partitioned into blocks of size ( _gi , _go , _gk ), and all coefficients within a block can be unified. The unified weights within a block are set to follow a predefined unification rule, for example, setting all values to be the same, such that a single value can be used to represent the entire block in a highly efficient quantization process. There can be multiple rules for unifying weights, each associated with a unified distortion loss, which measures the error introduced by adopting that rule. For example, instead of setting the weights to be the same, the weights can be set to have the same absolute value while preserving their original signs. Given the structure of the design, during iteration, the partial weight coefficients to be fixed can be determined by considering the uniform distortion loss, the estimated compression ratio loss, and the estimated velocity loss. A neural network training process is then performed to update the remaining non-fixed weight coefficients via backpropagation.

The overall framework of the iterative retraining/fine-tuning process can iteratively alternate steps to gradually optimize the joint loss. Given a pre-trained DNN model with weight coefficients W and a mask M, which can be a pruned sparse model or an unpruned non-sparse model, in the first step, the order of indices I(W) = [i _₀ , ..., i_{_k} ] can be determined by a unified index ordering and method selection module 202 to reshape the weight coefficients W (and the corresponding mask M), where k = k _₁ · k _₂ · k_₃ is the 3D tensor of the reshaped weights W. Specifically, the 3D tensor of the reshaped weights W can be partitioned into superblocks of size (g _{_i} , g _{_o} , g_{_k} ). Let S denote a superblock. _{I(W) is determined individually for each superblock S based on the weight unified loss of the weight coefficients within the superblock S, i.e., based on the weight unified loss ε_T} (Θ). The size of the superblock is typically selected according to a compression method later. For example, in at least one embodiment, a superblock of size (64,64,2) can be selected to conform to the 3-Dimension Coding Tree Unit (CTU3D) used in the subsequent compression process.

Each superblock S is further divided into blocks of size (d _{_i} , d_{_o} , d_{_k} ). Weight unification occurs within each block. For each superblock S, a weight unifier is used to unify the weight coefficients within the blocks of S. Let b denote a block in S; there are different ways to unify the weight coefficients in b. For example, the weight unifier can set all weights in b to be the same, for example, set to the average of all weights in b. In this case, the L _{_N} norm of the weight coefficients in b (e.g., the L_₂ norm as the variance of the weights in b) reflects the unification distortion loss ε_{_I}(b) of representing the entire block using the average. Alternatively, the weight unifier can set all weights to have the same absolute value while preserving the original sign. In this case, the L_{_N} norm of the absolute values of the weights in b can be used to measure _LI (b). In other words, given a weight unification method u, the weight unifier can unify the weights in b using a method u with a corresponding unification distortion loss _LI (u,b). The uniform distortion loss £ _I (u,S) of the entire superblock S can be determined by averaging _LI (u,b) over all blocks in S, i.e., _LI (u,S)＝average _b ( _LI (u,b)).

Similarly, the compression ratio loss £ _C (u,S) reflects the compression efficiency of uniform weights in superblock S using method u. For example, when all weights are set to be the same, only one number is used to represent the entire block, and the compression ratio is r _compression = _gi · _go · g _k . £ _C (u,S) can be defined as 1/r _compression .

The speed loss £ _S (u,S) reflects the estimated computation speed of using method u with uniform weighting coefficients in S, and £ _S (u,S) is a function of the number of multiplication operations in the computation using uniform weighting coefficients.

Up to this point, for each possible way of reordering the index to generate the 3D tensor of weights W, and for each possible method u of unifying the weights by the weight unifier, the weight unification loss £U(u,S) can be calculated based on £ _I (u,S), £ _C (u,S), and £ _S (u, _S ). An optimal weight unification method u* and an optimal reordered index I*(W) can be chosen, whose combination has the minimum weight unification loss £ _U *(u,S). When k is small, the optimal I*(W) and u* can be thoroughly searched. For large k, other methods can be used to find suboptimal I*(W) and u*. This invention does not impose any restrictions on the specific way of determining I*(W) and u*.

Once the order of indices I*(W) and the weight unification method u* are determined for each superblock S, the objective shifts to finding the updated optimal set of weight coefficients W* and the corresponding weight mask M* by iteratively minimizing the joint loss. Specifically, for the t-th iteration, the current weight coefficients W(t-1) and mask M(t-1) can be used. Furthermore, a weight unification mask Q(t-1) can be maintained throughout the training process. The weight unification mask Q(t-1) has the same shape as W(t-1), which records whether the corresponding weight coefficients are unified. Then, the unified weight coefficients W _U (t-1) and the new unification mask Q(t-1) are calculated by the weight unification module 204. In the weight unification module 204, the weight coefficients in S can be reordered according to the determined order of indices I*(W), and the superblocks can be sorted in ascending order based on the unification loss ￡ _U (u*,S) of the superblocks. Given a hyperparameter q, the top q superblocks are selected for unification. Furthermore, the weight unifier unifies the blocks in the selected superblock S using the corresponding determined method u*, obtaining unified weights W _{_U} (t-1) and weight masks _MU (t-1). The corresponding entries in the unification mask Q(t-1) are marked as unified. In at least one embodiment, _MU (t-1) differs from M(t-1), wherein for blocks with both pruned and unpruned weight coefficients, the weight unifier resets the initially pruned weight coefficients to have non-zero values and modifies the corresponding entries in _MU (t-1). For other types of blocks, _MU (t-1) naturally remains unchanged.

The uniform weight coefficients in Q(t-1) can be fixed, and the remaining unfixed weight coefficients in W(t-1) can be updated through the neural network training process to obtain updated W(t) and M(t).

Let represent the training dataset, which can be the same as the original dataset, based on which pre-trained weight coefficients W are obtained. It can also be a different dataset, but with the same data distribution as the original dataset. In the second step, each input x is passed through the current network via the network forwarding computation module 206 using the current unified weight coefficients W _{_U} (t-1) and mask _MU (t-1), producing an estimated output. Based on the ground truth annotation y and the estimated output, the target loss computation module 208 calculates the target training loss. The gradient of the target loss G(W_{_U}(t-1)) can then be calculated by the gradient computation module 210. Automatic gradient computation methods used by deep learning frameworks (such as TensorFlow or PyTorch) can be used to compute G(W_{_U}(t-1)). Based on the gradient G(W_{_U} (t-1)) and the unified mask Q(t-1), the non-fixed weight coefficients of W _{_U} (t-1) and the corresponding mask _MU (t-1) can be updated via backpropagation using the backpropagation and weight update module 212. The retraining process can be an iterative process. Generally, multiple iterations are performed to update the non-fixed part of W _{_U} (t-1) and the corresponding M(t-1), for example, until the target loss converges. Then the system enters the next iteration t, where given new hyperparameters q(t), new unified weight coefficients _{W U}(t), mask MU(t), and the corresponding unified mask Q(t) are calculated through a weight unification process based on W _{_U}( _t -1), u*, and I*(W).

In at least one embodiment, the value of the hyperparameter q(t) increases with t during each iteration, thereby unifying and fixing more and more weight coefficients throughout the iterative learning process.

Referring now to Figure 3, an operational flowchart illustrating the steps of the method 300 for compressing a neural network model is depicted. In some embodiments, at least one process block of Figure 3 may be executed by computer 102 (Figure 1) and server computer 114 (Figure 1). In some embodiments, at least one process block of Figure 3 may be executed by another device or a set of devices, said other device or set of devices being separate from or including computer 102 and server computer 114.

At 302, method 300 includes reordering at least one index corresponding to a multidimensional tensor associated with a neural network.

At 304, method 300 includes determining a set of weight coefficients associated with at least one reordered index.

In some embodiments, determining the set of weight coefficients associated with at least one reordered index includes: quantizing the weight coefficients; and selecting weight coefficients that minimize the unified loss value, wherein the unified loss value is associated with the weight coefficients.

In some embodiments, the minimized uniform loss value is backpropagated, and the neural network is trained based on the minimized uniform loss value obtained through backpropagation.

In some embodiments, the minimized unified loss value is backpropagated, and at least one of the weighting coefficients is fixed based on the minimized unified loss value obtained from the backpropagation.

In some embodiments, a gradient and a uniform mask associated with the set of weight coefficients are determined, and at least one non-fixed weight coefficient among the weight coefficients is updated based on the gradient and the uniform mask.

In some embodiments, the set of weight coefficients is compressed by quantizing and entropy encoding/decoding the weight coefficients.

In some embodiments, the uniform set of weight coefficients includes at least one weight coefficient having the same absolute value.

At 306, method 300 includes compressing a model of a neural network based on a determined set of weight coefficients.

It is understood that Figure 3 is merely an illustration of one implementation scheme and does not imply any limitation on how different embodiments can be implemented. Many modifications can be made to the depicted environment based on design and implementation requirements.

The method for compressing neural network models provided in this disclosure improves the efficiency of compressing learning weight coefficients, thereby accelerating the calculation of optimized weight coefficients. This can significantly compress neural network models and improve the computational efficiency of neural network models.

Figure 4 is a block diagram 400 of the internal and external components of the computer depicted in Figure 1 according to an exemplary embodiment. It should be understood that Figure 4 is merely an illustration of one embodiment and does not imply any limitation on the environment in which different embodiments may be implemented. Many modifications can be made to the depicted environment based on design and implementation requirements.

Computer 102 (FIG. 1) and server computer 114 (FIG. 1) may include corresponding groups of internal components 800A, 800B and external components 900A, 900B as shown in FIG. 4. Each of the groups of internal components 800 includes at least one processor 820, at least one computer-readable RAM 822, and at least one computer-readable ROM 824 on at least one bus 826, at least one operating system 828, and at least one computer-readable tangible storage device 830.

Processor 820 is implemented in hardware, firmware, or a combination of hardware and software. Processor 820 is a Central Processing Unit (CPU), Graphics Processing Unit (GPU), Accelerated Processing Unit (APU), microprocessor, microcontroller, Digital Signal Processor (DSP), Field-Programmable Gate Array (FPGA), Application-Specific Integrated Circuit (ASIC), or another type of processing unit. In some embodiments, processor 820 includes at least one processor capable of being programmed to execute functions. Bus 826 includes components that allow communication between internal components 800A, 800B.

At least one operating system 828, software program 108 (FIG. 1), and neural network model compression program 116 (FIG. 1) on server computer 114 (FIG. 1) are stored in at least one corresponding computer-readable tangible storage device 830 for execution by at least one corresponding processor 820 via at least one corresponding RAM 822 (which generally includes cache memory). In the embodiment shown in FIG. 4, each computer-readable tangible storage device 830 is a disk storage device of an internal hard disk drive. Alternatively, each computer-readable tangible storage device 830 is a semiconductor storage device, such as ROM 824, EPROM, flash memory, optical disc, magneto-optical disc, solid-state drive, compact disc (CD), digital versatile disc (DVD), floppy disk, cassette disk, magnetic tape, and/or another type of non-volatile computer-readable tangible storage device capable of storing computer programs and digital information.

Each set of internal components 800A, 800B also includes an R/W drive or interface 832 for reading from or writing to at least one portable computer-readable physical storage device 936 (e.g., CD-ROM, DVD, Memory Stick, magnetic tape, disk, optical disc, or semiconductor storage device). Software programs (such as software program 108 (FIG. 1) and neural network model compression program 116 (FIG. 1)) may be stored on at least one corresponding portable computer-readable physical storage device 936, read via the corresponding R/W drive or interface 832, and loaded into the corresponding hard disk drive 830.

Each set of internal components 800A, 800B also includes a network adapter or interface 836 (e.g., a TCP/IP adapter card), a wireless Wi-Fi interface card; or a 3G, 4G, or 5G wireless interface card or other wired or wireless communication links. Software program 108 (Figure 1) and neural network model compression program 116 (Figure 1) on server computer 114 (Figure 1) can be downloaded to computer 102 (Figure 1) from an external computer via a network (e.g., the Internet, a local area network, or other wide area network) and the corresponding network adapter or interface 836. From the network adapter or interface 836, software program 108 and neural network model compression program 116 on server computer 114 are loaded into the corresponding hard disk drive 830. The network may include copper wire, fiber optic, wireless transmission, routers, firewalls, switches, gateway computers, and/or edge servers.

Each set of external components 900A, 900B may include a computer display monitor 920, a keyboard 930, and a computer mouse 934. External components 900A, 900B may also include a touchscreen, virtual keyboard, touchpad, pointing device, and other human-machine interface devices. Each set of internal components 800A, 800B also includes a device driver 840 for interfacing with the computer display monitor 920, keyboard 930, and computer mouse 934. Device driver 840, R/W driver or interface 832, and network adapter or interface 836 include hardware and software (stored in storage device 830 and/or ROM 824).

It should be understood in advance that although this disclosure includes a detailed description of cloud computing, the implementations of the teachings cited herein are not limited to cloud computing environments. Rather, some embodiments can be implemented in conjunction with any other type of computing environment now known or developed hereafter.

Cloud computing is a service delivery model that enables convenient, on-demand network access to a shared pool of configurable computing resources (e.g., networks, network bandwidth, servers, processing power, storage, applications, virtual machines, and services) that can be rapidly provisioned and released with minimal management effort or interaction with service providers. This cloud model may include at least five features, at least three service models, and at least four deployment models.

The features are as follows:

On-demand self-service: Cloud users can automatically and unilaterally provide computing functions, such as server time and network storage, as needed, without any manual interaction with the service provider.

Broad network access: Functionality is available over a network and accessed through standard mechanisms that facilitate the use of heterogeneous thin or thick client platforms (e.g., mobile phones, laptops, and PDAs).

Resource pooling: A provider's computing resources can be pooled through a multi-tenant model to serve multiple consumers, dynamically allocating and reallocating different physical and virtual resources according to demand. Typically, consumers cannot control or know the exact location of the provided resources, but can specify locations at a higher level of abstraction (e.g., country, state, or data center), thus exhibiting location independence.

Rapid elasticity: In some cases, features can be rapidly and elastically scaled up; in others, they can be automatically and quickly scaled out and quickly released to scale in. To the user, the available features often appear unlimited and can be purchased in any quantity at any time.

Measured service: Cloud systems automatically control and optimize resource usage by leveraging metering capabilities at a certain level of abstraction appropriate to the service type (e.g., storage, processing, bandwidth, and active user accounts). Resource usage can be monitored, controlled, and reported, providing transparency to both service providers and users.

The service model is as follows:

Software as a Service (SaaS): This provides consumers with the functionality to use applications running on a provider's cloud infrastructure. These applications can be accessed from various client devices via thin client interfaces such as web browsers (e.g., web-based email). Aside from potentially limiting user-specific application configuration settings, users neither manage nor control the underlying cloud infrastructure, including the network, servers, operating system, storage, or even individual application functionalities.

Platform as a Service (PaaS): This provides consumers with the ability to deploy applications created or acquired by the consumer onto cloud infrastructure using programming languages and tools supported by the provider. Users neither manage nor control the underlying cloud infrastructure, including networks, servers, operating systems, or storage, but they have control over the deployed applications and may also have control over the configuration of the application hosting environment.

Infrastructure as a Service (IaaS) provides consumers with processing, storage, networking, and other basic computing resources, enabling them to deploy and run arbitrary software, including operating systems and applications. Users neither manage nor control the underlying cloud infrastructure, but have control over the operating system, storage, deployed applications, and may have limited control over the selection of network components (e.g., host firewalls).

The deployment model is as follows:

Private cloud: A cloud infrastructure that operates exclusively for a single organization. It can be managed by that organization or a third party, and can exist either on-premises or off-premises.

Community cloud: A cloud infrastructure shared by several organizations and supporting a specific community with common concerns (e.g., tasks, security requirements, policies, and compliance considerations). It may be managed by the organizations or third parties and may exist internally or externally.

Public cloud: Cloud infrastructure available to the public or large industrial groups and owned by organizations that sell cloud services.

Hybrid cloud: A cloud infrastructure consisting of two or more clouds (private, community, or public) that remain a single entity but are bound together by a standardized or proprietary technology that enables data and application portability (e.g., cloud explosion with load balancing between clouds).

Cloud computing environments are services focused on statelessness, loose coupling, modularity, and semantic interoperability. At the heart of cloud computing is an infrastructure comprising a network of interconnected nodes.

Referring to Figure 5, an exemplary cloud computing environment 500 is depicted. As shown, the cloud computing environment 500 includes at least one cloud computing node 10, and local computing devices used by cloud users can communicate with the at least one cloud computing node. These local computing devices are, for example, a Personal Digital Assistant (PDA) or cellular phone 54A, a desktop computer 54B, a laptop computer 54C, and/or an automotive computer system 54N. The cloud computing nodes 10 can communicate with each other. They can be physically or virtually grouped in at least one network (not shown), such as a private cloud, community cloud, public cloud, or hybrid cloud, or a combination thereof, as described above. This allows the cloud computing environment 500 to provide Infrastructure as a Service, Platform as a Service, and/or Software as a Service, without requiring cloud consumers to maintain resources on their local computing devices. It should be understood that the types of computing devices 54A-N shown in Figure 5 are intended to be exemplary only, and the cloud computing nodes 10 and the cloud computing environment 500 can communicate with any type of computerized device via any type of network and/or network-addressable connectivity (e.g., using a web browser). Referring to Figure 6, a set of functional abstraction layers 600 provided by the cloud computing environment 500 (Figure 5) is shown. It should be understood beforehand that the components, layers, and functions shown in Figure 6 are merely exemplary, and the embodiments are not limited thereto. As shown, the following layers and corresponding functions are provided:

The hardware and software layer 60 includes hardware components and software components. Examples of hardware components include: a host 61; a server 62 based on a Reduced Instruction Set Computer (RISC) architecture; a server 63; a blade server 64; a storage device 65; and a network and networking component 66. In some embodiments, the software components include network application server software 67 and database software 68.

The virtualization layer 70 provides an abstraction layer from which the following examples of virtual entities can be provided: virtual server 71; virtual storage 72; virtual network 73, including virtual private network; virtual application and operating system 74; and virtual client 75.

In one example, management layer 80 provides the functionality described below. Resource provisioning 81 provides dynamic acquisition of computing resources and other resources used to perform tasks within the cloud computing environment. Metering and pricing 82 tracks the cost of resource usage and provides billing and invoicing for the consumption of these resources when they are utilized in the cloud computing environment. In one example, these resources may include application software licenses. Security provides authentication for cloud users and tasks, as well as protection for data and other resources. User portal 83 provides access to the cloud computing environment for users and system administrators. Service level management 84 provides cloud resource allocation and management to meet required service levels. Service Level Agreement (SLA) planning and implementation 85 provides pre-scheduling and procurement of cloud resources for anticipated future needs according to the SLA.

Workload layer 90 provides examples of functionalities that can leverage cloud computing environments. Examples of workloads and functionalities that can be provided from this layer include: mapping and navigation 91; software development and lifecycle management 92; virtual classroom delivery 93; data analysis and processing 94; transaction processing 95; and neural network model compression 96. Neural network model compression 96 can compress neural network models.

Some embodiments may relate to systems, methods, and/or computer-readable media at any possible level of technical detail integration. A computer-readable medium may include a computer-readable non-volatile storage medium having computer-readable program instructions for causing a processor to perform operations.

Computer-readable storage media can be tangible devices that hold and store instructions for use by an instruction execution device. Computer-readable storage media can be, for example, but not limited to, electronic storage devices, magnetic storage devices, optical storage devices, electromagnetic storage devices, semiconductor storage devices, or any suitable combination of the foregoing. A non-exhaustive list of more specific examples of computer-readable storage media includes the following: portable computer floppy disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), static random access memory (SRAM), compact disc read-only memory (CD-ROM), digital versatile disk (DVD), memory sticks, floppy disks, mechanical encoding devices, such as punched cards or raised structures in recesses on which instructions are recorded, and any suitable combination of the foregoing. As used herein, a computer-readable storage medium is not to be construed as a transient signal itself, such as radio waves or other freely propagating electromagnetic waves, electromagnetic waves propagating through waveguides or other transmission media (e.g., light pulses through optical fibers), or electrical signals transmitted through wires.

The computer-readable program instructions described herein can be downloaded from a computer-readable storage medium to a suitable computing/processing device, or downloaded to an external computer or external storage device via a network (e.g., the Internet, a local area network, a wide area network, and/or a wireless network). The network may include copper cables, optical fibers, wireless transmissions, routers, firewalls, switches, gateway computers, and/or edge servers. A network adapter card or network interface in each computing/processing device receives the computer-readable program instructions from the network and forwards them to a computer-readable storage medium within the suitable computing/processing device.

The computer-readable program code/instructions that perform the operation can be assembly instructions, instruction-set-architecture (ISA) instructions, machine instructions, machine-dependent instructions, microcode, firmware instructions, status setting data, integrated circuit configuration data, or source code or object code written in any combination of one or more programming languages, including object-oriented programming languages (such as Smalltalk, C++, etc.) and procedural programming languages (such as the "C" programming language or similar programming languages). The computer-readable program instructions can execute entirely on the user's computer, partially on the user's computer, as a standalone software package partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In the latter case, the remote computer can be connected to the user's computer via any type of network (including local area network (LAN) or wide area network (WAN)) or can be connected to an external computer (e.g., via the Internet provided by an Internet service provider). In some embodiments, electronic circuits including, for example, programmable logic circuits, field-programmable gate arrays (FPGAs), or programmable logic arrays (PLAs) can execute computer-readable program instructions to personalize the electronic circuits by utilizing state information of computer-readable program instructions, thereby performing aspects or operations of this disclosure.

These computer-readable program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create methods for implementing the functions/operations specified in the blocks or blocks of the flowchart and/or block diagram. These computer-readable program instructions can also be stored in a computer-readable storage medium that can instruct a computer, programmable data processing apparatus, and/or other device to operate in a particular manner, such that the computer-readable storage medium storing the instructions includes an article of manufacture comprising instructions that implement aspects of the functions/operations specified in the blocks or blocks of the flowchart and/or block diagram.

Computer-readable program instructions may also be loaded onto a computer, other programmable data processing apparatus or other device to perform a series of operational steps on the computer, other programmable apparatus or other device to produce computer-implemented processing, such that the instructions executed on the computer, other programmable apparatus or other device implement the functions/operations specified in the boxes or blocks of the flowchart and/or block diagram.

The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer-readable media according to various embodiments. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of instructions, including at least one executable instruction for implementing a specified logical function. Methods, computer systems, and computer-readable media may include additional blocks, fewer blocks, different blocks, or blocks arranged differently from those depicted in the drawings. In some alternative embodiments, the functions indicated in the blocks may not be performed in the order shown in the drawings. For example, two blocks shown consecutively may actually be performed simultaneously or substantially simultaneously, or these blocks may sometimes be performed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and/or flowcharts, and combinations of blocks in the block diagrams and/or flowcharts, may be implemented by a hardware-based dedicated system that performs the specified function or operation, or performs a combination of dedicated hardware and computer instructions.

Clearly, the systems and/or methods described herein can be implemented in various forms of hardware, firmware, or combinations of hardware and software. The actual dedicated control hardware or software code for implementing these systems and/or methods is not limited to these implementations. Therefore, while the operation and behavior of the systems and/or methods are described herein without reference to specific software code—it should be understood that software and hardware can be designed to implement the systems and/or methods based on the description herein.

Elements, actions, or instructions used herein should not be construed as critical or essential unless explicitly stated otherwise. Furthermore, as used herein, the articles “a” and “an” are intended to include at least one item and are interchangeable with “at least one.” Additionally, as used herein, the term “group” is intended to include at least one item (e.g., related items, unrelated items, a combination of related and unrelated items, etc.) and is interchangeable with “at least one.” The term “a” or similar language is used when referring to only one item. Furthermore, as used herein, the terms “has,” “have,” “having,” etc., are intended to be open-ended terms. Further, the phrase “according to” is intended to mean “at least partially according to,” unless otherwise explicitly stated.

Descriptions of various aspects and embodiments have been presented for illustrative purposes, but such description is not intended to be exhaustive or limited to the disclosed embodiments. While combinations of features are referenced in the claims and/or disclosed in the specification, these combinations are not intended to limit the disclosure of possible embodiments. In fact, many of these features can be combined in ways not specifically listed in the claims and/or not disclosed in the specification. While each dependent claim listed below may be directly subordinated to only one claim, the disclosure of possible embodiments includes each dependent claim combined with each other claim in the group of claims. Many modifications and variations will be apparent to those skilled in the art without departing from the scope of the described embodiments. The terminology used herein has been chosen to best explain the principles of the embodiments, their practical application, or improvements relative to technology found in the market, or to enable those skilled in the art to understand the embodiments disclosed herein.

Claims (9)

1. A method for compressing a neural network model used for video processing, characterized in that the input and output of the neural network model are video or images, the original pre-trained neural network model is compressed through weight uniform regularization, and feature descriptors are extracted using the compressed neural network model, the method comprising: At least one index corresponding to a multidimensional tensor is reordered, where the multidimensional tensor is the multidimensional tensor corresponding to the weight coefficients of each network layer of the neural network; the at least one index is determined based on the weight uniform loss of the weight coefficients. The weight coefficients are quantized, and the weight coefficients that minimize the unified loss value are selected to determine the set of weight coefficients associated with at least one reordered index, wherein the unified loss value is associated with the weight coefficients; and The model of the neural network is compressed according to the determined set of weight coefficients; The method further includes: The minimized uniform loss value is backpropagated, and at least one of the weight coefficients is fixed based on the minimized uniform loss value obtained from the backpropagation. The uniform loss value includes compression ratio loss, uniform distortion loss, and computation speed loss. Determine the gradient and uniform mask associated with the set of weight coefficients, and update at least one non-fixed weight coefficient among the weight coefficients based on the gradient and the uniform mask. 2. The method according to claim 1, further comprising backpropagating the minimized uniform loss value and training the neural network based on the minimized uniform loss value obtained from the backpropagation. 3. The method according to any one of claims 1-2, characterized in that it further includes compressing the set of weight coefficients by quantizing and entropy encoding/decoding the weight coefficients. 4. The method according to any one of claims 1-2, characterized in that the uniform set of weight coefficients includes at least one weight coefficient having the same absolute value. 5. A computer system for compressing a neural network model used for video processing, characterized in that the input and output of the neural network model are video or images, the original pre-trained neural network model is compressed through weight uniform regularization, and feature descriptors are extracted using the compressed neural network model, the computer system comprising: A reordering module is used to reorder at least one index corresponding to a multidimensional tensor, wherein the multidimensional tensor is the multidimensional tensor corresponding to the weight coefficients of each network layer of the neural network; the at least one index is determined based on the weight uniform loss of the weight coefficients. A unification module for determining a set of weight coefficients associated with at least one reordered index, comprising a quantization module for quantizing the weight coefficients; and a selection module for selecting weight coefficients that minimize a unification loss value, wherein the unification loss value is associated with the weight coefficients; and A compression module is used to compress the model of the neural network according to a determined set of weight coefficients; The computer system further includes an update module, which is used to backpropagate the minimized uniform loss value and fix at least one of the weight coefficients based on the backpropagated minimized uniform loss value. The uniform loss value includes compression ratio loss, uniform distortion loss and computation speed loss. The update module is further configured to determine the gradient and uniform mask associated with the set of weight coefficients, and update at least one non-fixed weight coefficient among the weight coefficients based on the gradient and the uniform mask. 6. The computer system according to claim 5, further comprising a training module, the training module being used to backpropagate the minimized uniform loss value and train the neural network based on the minimized uniform loss value obtained from the backpropagation. 7. The computer system according to claim 5, further comprising a compression module, the compression module being used to compress the set of weight coefficients by quantizing and entropy encoding/decoding the weight coefficients. 8. A non-volatile computer-readable medium, characterized in that it stores thereon a computer program for compressing a neural network model, the computer program being used to cause at least one computer processor to perform the method as described in any one of claims 1-4. 9. A computing device, characterized in that it comprises a processor and a memory; the memory stores a computer program, which, when executed by the processor, causes the processor to perform the method as described in any one of claims 1 to 4.