CN110097186B - A Neural Network Heterogeneous Quantization Training Method - Google Patents

Abstract

The invention provides a neural network heterogeneous quantitative training method, which belongs to the technical field of artificial neural network. On the basis of the traditional training framework based on CPU or GPU or a combination of the two, the invention adds high-speed interface logic, and connects hardware computing through the high-speed interface logic. In the acceleration module, a specific step or steps in the middle of the training process is transferred to the hardware computing acceleration module. After the calculation is completed, the result is returned to the source training master through the high-speed interface logic, and the training process with specific customization functions is completed. Quickly implement and deploy cutting-edge new structures or algorithms into training, improve system flexibility, reduce storage and bandwidth requirements, reduce resource requirements in the forward prediction process, reduce training complexity, improve training efficiency, and ensure that current training devices can be more efficient. Well adapted to the latest neural network architectures.

Description

A Neural Network Heterogeneous Quantization Training Method

technical field

The invention relates to the technical field of artificial neural networks, in particular to a neural network heterogeneous quantitative training method.

Background technique

Neural network training sends a set of training sets into the network, and adjusts the weights according to the difference between the actual output of the network and the expected output. The training process includes: defining the structure of the neural network and the output of forward propagation, finding the error between the result and the expected value, and then returning the error layer by layer, and then updating the weights. Adjust the network weights by training samples and expected values.

CPU is good at logic control, serial operation and general type data operation, while GPU focuses on processing multiple tasks of large-scale parallel computing. Both CPU and GPU can efficiently complete tasks in their respective fields, and can also be used as the mainstream method of current neural network training.

With the deepening of research, more and more new structures and new algorithms are constantly being proposed, which brings higher requirements and challenges to general-purpose CPU and GPU training methods. It is difficult to quickly realize specific detailed structures, and the training time may become more lengthy. .

Contents of the invention

In order to solve the above technical problems, the present invention proposes a neural network heterogeneous quantization training method. Heterogeneous methods are used to accelerate the original training process, and cutting-edge new structures, such as special convolution types, or new algorithms, such as model parameter quantization, can be quickly implemented and deployed in training, improving system flexibility and reducing storage And bandwidth requirements, reduce resource requirements in the forward prediction process, reduce training complexity, improve training efficiency, and ensure that the current training device can better adapt to the latest neural network structure.

Technical scheme of the present invention is:

A neural network heterogeneous quantization training method, on the basis of the traditional training architecture based on CPU or GPU or a combination of the two, add high-speed interface logic, connect the hardware quantization acceleration module through the high-speed interface logic, add quantization steps in the training process, and The quantitative calculation process of model parameters and feature map results is delegated to the hardware quantization acceleration module, and the results after the quantitative calculation are completed are returned to the source training main control through the high-speed interface logic, the quantized model parameters are updated, and the model with the model is iteratively completed. The training process of the parameter and feature map result quantization function.

Further, the hardware quantization acceleration module is responsible for completing the low-bit quantization of neural network model parameters and neural network feature map results, which is implemented by a dedicated circuit and forms a heterogeneous structure with the traditional training subject CPU or GPU.

Further, the data quantization operation includes: data temporary storage, data statistical sorting, data compression and decompression, data hashing and table lookup, floating-point number to fixed-point number with specific digits, floating-point number shift scaling and interception, data inversion quantification etc.

The specific customization functions include but are not limited to: model parameter quantization, conversion of floating-point numbers to fixed-point numbers, special convolution operations, such as dilated convolution, deep-wise convolution, 1x1 multiplier array, fully connected multiplier-adder array, etc.; The specific custom function is realized by the hardware computing acceleration module, and the module may be enabled multiple times or only once during the training process to complete the specific function.

Specifically include the following steps:

1) Under the traditional CPU or GPU-based training framework, set the initial values of neural network model parameters and hyperparameters, initialize the hardware quantization acceleration module at the same time, and start training;

2) After the first round of backpropagation has updated the last layer parameters of the neural network, the updated weight parameters are passed into the hardware quantization acceleration module, and the weight parameters are initially compressed and compressed by general data compression methods such as GZIP or entropy coding. Store, then sort the data statistically, further shift and intercept the data according to the expected fixed-point digits, and limit the maximum and minimum values of the data, obtain the quantized weight parameters, and send them back to the traditional framework to continue backpropagation The parameters of the previous layer are updated until the first round of backpropagation is completed, and all weight parameters are obtained;

3) Repeat step 2 to update the weights, complete multiple rounds of backpropagation, until the model Loss requirements are met, and the training is completed;

4) In addition to the weight parameters, the results of the feature maps of each layer of the neural network can also be quantified to further quantify the reasoning of the entire model;

5) According to the needs, the weight data can be hashed to obtain the index value, and the data can be further compressed, or the data can be dequantized immediately after the quantization is completed, so as to reduce the quantization loss.

The hardware computing acceleration module adopts FPGA or ACAP to realize through logical configuration, external non-volatile memory device, different customized functions can be stored at the same time, and FPGA or ACAP is configured in real time according to training requirements to complete different functions in the same training process.

The high-speed interface logic includes but is not limited to PCIE interface, USB3.0 interface, 10 Gigabit Ethernet interface, etc., and communicates and interacts with the original training master control.

The beneficial effects of the present invention are

Heterogeneous methods are used to accelerate the original training process, and cutting-edge new structures, such as special convolution types, or new algorithms, such as model parameter quantization, can be quickly implemented and deployed in training, improving system flexibility and reducing storage And bandwidth requirements, reduce resource requirements in the forward prediction process, reduce training complexity, improve training efficiency, and ensure that the current training device can better adapt to the latest neural network structure.

Detailed ways

In order to make the purpose, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention are clearly and completely described below. Obviously, the described embodiments are part of the embodiments of the present invention, not all Embodiments, based on the embodiments of the present invention, all other embodiments obtained by persons of ordinary skill in the art without making creative efforts belong to the protection scope of the present invention.

A neural network heterogeneous quantization training method of the present invention, on the basis of the traditional training framework based on CPU or GPU or a combination of the two, adds high-speed interface logic, connects the hardware quantization acceleration module through the high-speed interface logic, and adds quantization during the training process step, delegating the quantization calculation process of model parameters and feature map results to the hardware quantization acceleration module, returning the results of the quantization calculation to the source training master through the high-speed interface logic, updating the quantized model parameters, and iterating Complete the training process with the quantization function of model parameters and feature map results.

The hardware quantization acceleration module is responsible for completing the low-bit quantization of neural network model parameters and neural network feature map results. It is realized by a dedicated circuit and forms a heterogeneous structure with the traditional training main body CPU or GPU. Data quantization operations include: data temporary storage, data statistical sorting, data compression and decompression, data hashing and table lookup, floating-point numbers to fixed-point numbers with specific digits, floating-point number shift scaling and interception, data inverse quantization, etc.

Include the following steps:

1) Under the traditional CPU or GPU-based training framework, set the initial values of the neural network model parameters and hyperparameters, initialize the hardware quantization acceleration module at the same time, and start training;

2) After the first round of backpropagation has updated the last layer parameters of the neural network, the updated weight parameters are passed into the hardware quantization acceleration module, and the weight parameters are initially compressed and compressed by general data compression methods such as GZIP or entropy coding. Store, then sort the data statistically, further shift and intercept the data according to the expected fixed-point digits, and limit the maximum and minimum values of the data, obtain the quantized weight parameters, and send them back to the traditional framework to continue backpropagation The parameters of the previous layer are updated until the first round of backpropagation is completed, and all weight parameters are obtained;

3) Repeat step 2 to update the weights, complete multiple rounds of backpropagation, until the model Loss requirements are met, and the training is completed;

4) In addition to the weight parameters, the results of the feature maps of each layer of the neural network can also be quantified to further quantify the reasoning of the entire model;

5) According to the needs, the weight data can be hashed to obtain the index value, and the data can be further compressed, or the data can be inversely quantized immediately after the quantization is completed to reduce the quantization loss;

The hardware calculation acceleration module adopts FPGA or ACAP to realize through logical configuration, external non-volatile storage device, different customized functions can be stored at the same time, FPGA or ACAP is configured in real time according to training requirements, and different functions in the same training process are completed; The above-mentioned high-speed interface logic includes but is not limited to PCIE interface, USB3.0 interface, 10 Gigabit Ethernet interface, etc., and communicates with the original training master control.

The above descriptions are only preferred embodiments of the present invention, and are only used to illustrate the technical solution of the present invention, and are not used to limit the protection scope of the present invention. Any modification, equivalent replacement, improvement, etc. made within the spirit and principles of the present invention are included in the protection scope of the present invention.

Claims (6)

1. A neural network heterogeneous quantization training method, characterized in that, On the basis of the traditional training architecture based on CPU or GPU or a combination of the two, add high-speed interface logic, connect the hardware quantization acceleration module through the high-speed interface logic, and delegate the quantization calculation process of model parameters and feature map results to hardware quantization during the training process The acceleration module returns the result after the quantization calculation is completed to the source training main control via the high-speed interface logic, updates the quantized model parameters, and iteratively completes the training process with the quantization function of model parameters and feature map results; Specific steps are as follows: 1) Under the traditional CPU or GPU-based training framework, set the initial values of the neural network model parameters and hyperparameters, initialize the hardware quantization acceleration module at the same time, and start training; 2) After the first round of backpropagation updates the last layer parameters of the neural network, the updated weight parameters are passed to the hardware quantization acceleration module, and the weight parameters are compressed and stored for the first time through the data compression method, and then the data are counted Sorting, further shifting and intercepting the data according to the expected number of fixed-point digits, and limiting the maximum and minimum values of the data to obtain quantized weight parameters, which are sent back to the traditional framework to continue backpropagating the previous layer of parameter updates until Complete the first round of backpropagation and get all weight parameters; 3) Repeat step 2) to update the weights and complete multiple rounds of backpropagation until the model Loss requirements are met and the training is completed; In addition to the weight parameters, the results of the feature maps of each layer of the neural network can also be quantified to further quantify the entire model reasoning; According to needs, the weight data can be hashed to obtain the index value, and the data can be further compressed, or the data can be dequantized immediately after the quantization is completed to reduce the quantization loss. 2. The method of claim 1, wherein, The hardware quantization acceleration module is responsible for completing low-bit quantization of neural network model parameters and neural network feature map results. 3. The method of claim 2, wherein, The hardware quantization acceleration module is implemented by a circuit, and forms a heterogeneous structure with the traditional training subject CPU or GPU. 4. The method of claim 1, wherein, The data quantization operation includes: data temporary storage, data statistical sorting, data compression and decompression, data hashing and table lookup, floating-point number conversion to fixed-point number with specific digits, floating-point number shift scaling and interception, and data inverse quantization. 5. The method of claim 1, wherein, The hardware quantization acceleration module adopts FPGA or ACAP to realize through logical configuration, external non-volatile memory device, different customized functions can be stored at the same time, and FPGA or ACAP is configured in real time according to training requirements to complete different functions in the same training process. 6. The method of claim 1, wherein, The high-speed interface logic can be realized by using PCIE interface, USB3.0 interface or 10 Gigabit Ethernet interface, and communicates and interacts with the original training master control.