JP2020060967A - Neural network processing device, and neural network processing method - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a neural network processing apparatus capable of suppressing a decrease in a processing speed of a neural network even when embedded hardware is used, and a neural network processing method. A CNN processing device 1 includes an input buffer 10 for storing an input signal A given to a CNN, a weight buffer 11 for storing a weight U of a CNN, and a sum-of-product operation of an input signal A and a weight U. It includes a processor that performs a first process of performing a CNN convolution operation and a second process of performing quantization that reduces bit accuracy of at least a part of the data used in the first process. [Selection diagram] Fig. 1

Description

  The present invention relates to a neural network processing device and a neural network processing method.

  In recent years, a convolutional neural network (CNN) has attracted attention as a deep neural network for classifying images into a plurality of categories. CNN is characterized by having a convolutional layer in a deep neural network. In the convolutional layer, a filter is applied to the input data. More specifically, in the convolutional layer, the filter window is slid by a constant stride, the filter element is multiplied by the corresponding element of the input data, and the sum-of-products operation is performed.

  FIG. 11 is a diagram showing a general CNN signal processing flow. The CNN has an input layer, an intermediate layer, and an output layer (for example, see Non-Patent Document 1). In the intermediate layer, a convolution operation for weighting the input signal is performed. As shown in FIG. 11, in the intermediate layer, an activation function such as ReLU is applied to the result of the convolutional operation as needed, and further, normalization and pooling processing is performed in some cases. Further, the characteristics of the input signal extracted through the convolution operation are applied to the classifier including the fully connected layer, and the classification result is output from the output layer. As described above, one feature of a neural network such as CNN is that the product-sum calculation is repeatedly performed.

  Here, the input value and weight of the input data used for CNN may include a decimal point, but in the product-sum calculation of a conventional neural network such as CNN, “input signal”, “weight”, As shown in each value of “convolution operation” and “convolution operation”, the operation processing is performed in a form in which the number of digits of the operation result is secured.

Hideki Aso, et al., “Deep Learning Deep Learning”, Modern Science Company, November 2015

  However, when a conventional neural network such as CNN is implemented by built-in hardware such as FPGA (Field Programmable Gate Array) or a microcomputer, a reduction in processing speed due to a product-sum operation with a large number of digits has been a problem.

  The present invention has been made to solve the above-described problems, and a neural network processing device capable of suppressing a decrease in the processing speed of a neural network even when using embedded hardware, and It is an object to provide a neural network processing method.

  In order to solve the above-mentioned problems, a neural network processing device according to the present invention includes a first memory for storing an input signal given to a neural network, a second memory for storing a weight of the neural network, and the input signal. A first process for performing a convolution operation of the neural network including a product-sum operation of the weight and the weight, and a second process for performing a quantization for reducing the bit precision of at least a part of the data used in the first process. And a processor for performing.

  Further, in the neural network processing device according to the present invention, the second process performs quantization for reducing at least one bit precision of the input signal, the weight, and the operation result of the product-sum operation in the first process. May be included.

  Further, in the neural network processing device according to the present invention, the second memory may store a quantization weight obtained by previously quantizing the weight.

  Further, in the neural network processing device according to the present invention, the processor includes a quantization convolution operation unit that performs the first process in which the second process is incorporated, based on the weight and the input signal. Good.

  Further, in the neural network processing device according to the present invention, a third memory for storing a first function for quantizing the weight and a second function for implementing the first process in which the second process is incorporated are stored. A fourth memory may be further included, and the processor may read the first function to quantize the weight, read the second function, and perform the first process in which the second process is incorporated. .

  In the neural network processing device according to the present invention, the neural network may be a multilayer neural network having at least one intermediate layer.

  In order to solve the above-mentioned problems, a neural network processing method according to the present invention stores a first step of storing an input signal given to a neural network in a first memory and a weight of the neural network in a second memory. A second step, a first step in which the processor performs a convolution operation of the neural network including a product-sum operation of the input signal and the weight, and a bit precision of at least a part of data used in the first processing. And a third step of performing a second process of performing quantization for reduction.

  According to the present invention, since quantization is performed to reduce the bit precision of at least part of the data used in the first process that performs the convolution operation including the product-sum operation of the input signal and the weight given to the neural network, It is possible to suppress a decrease in the processing speed of the neural network even when the hardware for use is used.

FIG. 1 is a block diagram illustrating the functions of the CNN processing device according to the first embodiment of the present invention. FIG. 2 is a block diagram showing the hardware configuration of the CNN processing device according to the first embodiment. FIG. 3 is a diagram for explaining the flow of the CNN processing method according to the first embodiment. FIG. 4 is a diagram for explaining the CNN processing method according to the first embodiment. FIG. 5 is a block diagram illustrating the functions of the CNN processing device according to the second embodiment. FIG. 6 is a diagram for explaining the flow of the CNN processing method according to the second embodiment. FIG. 7 is a diagram for explaining the CNN processing method according to the second embodiment. FIG. 8 is a block diagram illustrating the function of the CNN processing device according to the third embodiment. FIG. 9 is a diagram for explaining the flow of the CNN processing method according to the third embodiment. FIG. 10 is a diagram for explaining the CNN processing method according to the third embodiment. FIG. 11 is a diagram for explaining a conventional CNN calculation process.

Hereinafter, preferred embodiments of the present invention will be described in detail with reference to FIGS. 1 to 10. In the following, a case where CNN is used as the “neural network” will be described as an example.
[First Embodiment]
First, the configuration of the CNN processing device (neural network processing device) 1 according to the first embodiment of the present invention will be described. FIG. 1 is a block diagram showing a functional configuration of the CNN processing device 1.

  The CNN processing device 1 according to the present embodiment is a processing device that performs a product-sum calculation of an input signal given to the CNN and the weight of the CNN and outputs a calculation result. This operation processing includes a product-sum operation of the convolutional layers forming the intermediate layer of CNN (hereinafter, may be referred to as “convolutional operation”). The result of the convolution operation performed by the CNN processing device 1 is multiplied by an activation function such as ReLU to obtain an output of the convolutional layer for one layer. In the following, for simplicity of explanation, it is assumed that the result of the product-sum operation of the convolutional layer, that is, the result of the convolutional operation is used as the input signal of the next convolutional layer. The CNN processing device 1 repeatedly performs the product-sum calculation of the input signal and the weight, and executes the product-sum calculation of the number of times corresponding to the number of convolutional layers included in the preset CNN model.

[Functional block of CNN processing device]
The CNN processing device 1 described above includes an input buffer (first memory) 10, a weight buffer (second memory) 11, a convolution operation unit 12, an operation result buffer 13, a quantization processing unit 14, an output buffer 15, and a storage unit 16. Equipped with.

  The input buffer 10 stores the input signal supplied to CNN. More specifically, the input buffer 10 stores an input signal such as image data given from the outside. The input buffer 10 also outputs an input signal such as image data to the quantization processing unit 14. The input signal supplied to the input buffer 10 may be pre-processed image data. Examples of preprocessing include monochrome conversion, contrast adjustment, and brightness adjustment. Further, the input signal may be reduced to have a bit depth set according to a CNN model preset in the CNN processing device 1. As the value of the input signal supplied to the input buffer 10, for example, a value including a decimal point represented by a 32-bit or 16-bit precision floating point array is used.

  The weight buffer 11 stores the weight of CNN. More specifically, the weight buffer 11 is loaded with CNN weight parameters stored in advance in a server (not shown) installed outside the CNN processing device 1 or the storage unit 16. In the present embodiment, as the weight value, a value including a decimal point represented by a 32-bit or 16-bit precision floating-point array is used. The weight buffer 11 outputs the buffered weight to the quantization processing unit 14 described later.

  The convolution operation unit 12 performs a CNN convolution operation including a product-sum operation of the input signal stored in the input buffer 10 and the weight stored in the weight buffer 11 (first processing). More specifically, the convolution operation unit 12 performs a product-sum operation of convolutional layers forming a CNN model preset in the CNN processing device 1. In the present embodiment, the convolution operation unit 12 performs the convolution operation based on the input signal quantized by the quantization processing unit 14 described below and the weight value. The calculation result output from the convolution calculation unit 12 is supplied to the calculation result buffer 13.

  The operation result buffer 13 buffers the result of the convolution operation by the convolution operation unit 12, and supplies the operation result to the quantization processing unit 14.

  The quantization processing unit 14 performs quantization for reducing the bit precision of at least part of the data used in the convolution operation process (first process) (second process). More specifically, the quantization processing unit 14 quantizes at least part of the data of the input signal, the weight, and the operation result of the convolution operation. In the present embodiment, the quantization processing unit 14 performs a quantization process on all data of the input signal, the weight, and the calculation result of the convolution calculation.

  Specifically, the quantization processing unit 14 quantizes the input signal read from the input buffer 10, quantizes the weight read from the weight buffer 11, and quantizes the result of the convolution operation read from the operation result buffer 13. I do. Further, the quantization processing unit 14 converts the input signal, the weight, and the calculation result stored in the input buffer 10, the weight buffer 11, and the calculation result buffer 13 into the quantized input signal, the weight, and the calculation result, respectively. Update to the value of.

  The quantization processing executed by the quantization processing unit 14 includes, for example, well-known fractional processing such as rounding, rounding up, rounding down, nearest rounding, and the like. On the other hand, restrictions such as converting the value including the decimal point to an integer are applied. There is a trade-off between the improvement of the processing speed associated with the quantization processing and the accuracy of the processing result. The quantization of each value performed by the quantization processing unit 14 is, for example, processing speed and accuracy by reducing the number of bits according to the number of bits that the processor 102 included in the CNN processing device 1 can handle at once, such as 8 bits or 16 bits. It is possible to achieve both. For example, in the case of FPGA, each value may be reduced to 1 bit. Note that the quantization processing unit 14 may be used only for some of the convolutional layers that are connected by the CNN processing device 1 and that are included in the CNN and are connected in multiple stages.

  The output buffer 15 temporarily stores the result of the convolution operation quantized by the quantization processing unit 14.

  The storage unit 16 stores the result of the quantized convolution operation temporarily stored in the output buffer 15. The result of the convolution operation stored in the storage unit 16 is stored as the output value of the convolutional layer of CNN and the input value of the next convolutional layer in the present embodiment. Further, the storage unit 16 stores in advance information indicating a fraction processing method used when the quantization processing unit 14 quantizes the input signal, the weight, and the value of the result of the convolution operation.

[Hardware configuration of CNN processing device]
Next, an example of the hardware configuration of the CNN processing device 1 having the above-described functions will be described with reference to the block diagram of FIG.

  As shown in FIG. 2, the CNN processing device 1 is, for example, a computer including a processor 102, a main storage device 103, a communication interface 104, an auxiliary storage device 105, and an input / output device 106, which are connected via a bus 101, and these. Can be realized by a program that controls the hardware resources of

  Programs for the processor 102 to perform various controls and calculations are stored in the main storage device 103 in advance. Each function of the CNN processing device 1 including the convolution operation unit 12 and the quantization processing unit 14 illustrated in FIG. 1 is realized by the processor 102 and the main storage device 103.

  The main storage device 103 implements the input buffer 10, the weight buffer 11, the calculation result buffer 13, and the output buffer 15 described with reference to FIG.

  The communication interface 104 is an interface circuit for communicating with various external electronic devices via the communication network NW. The input signal such as image data used by the CNN processing device 1 and the weight may be received from an external server or the like via the communication interface 104.

  The auxiliary storage device 105 includes a readable / writable storage medium and a drive device for reading / writing various information such as programs and data from / to the storage medium. For the auxiliary storage device 105, a semiconductor memory such as a hard disk or a flash memory can be used as a storage medium.

  The auxiliary storage device 105 has a storage area for storing input data and weights obtained from the outside, and a program storage area for storing a program for the CNN processing device 1 to perform CNN arithmetic processing such as convolution arithmetic. The auxiliary storage device 105 realizes the storage unit 16 described in FIG. 1. Furthermore, for example, it may have a backup area for backing up the above-mentioned data and programs.

  The input / output device 106 is configured by an I / O terminal that inputs a signal from an external device and outputs a signal to the external device. A display device (not shown) or the like may be provided via the input / output device 106 to display the calculation result output by the CNN processing device 1.

  Here, the program stored in the program storage area of the auxiliary storage device 105 may be a program that is processed in time series according to the order of the CNN processing method described in this specification, or in parallel. Alternatively, it may be a program that is processed at a necessary timing such as when it is called. Further, the program may be processed by one computer or may be processed in a distributed manner by a plurality of computers.

[CNN processing method]
Next, the operation of the CNN processing device 1 having the above configuration will be described with reference to FIGS. 3 and 4. First, the input buffer 10 and the weight buffer 11 temporarily store the input signal A and the weight U provided from a server or the like installed outside the CNN processing device 1 (steps S1 and S3).

  As shown in FIG. 4, the input signal A is stored in the input buffer 10. The input signal A is vectorized input image data and has vertical and horizontal dimensions. The value of the input signal A is represented by a value including a decimal point. The bold rectangle in FIG. 4 represents a filter. The weight buffer 11 stores the weight U. The weight U is an element of the kernel represented by a matrix, and is a parameter that is adjusted and updated by learning of CNN and finally determined. The value of the weight U also has dimensions in the vertical direction and the horizontal direction, and each element is represented by a value including a decimal point.

  Next, the quantization processing unit 14 reads the input signal A from the input buffer 10 and quantizes the value of each element (step S2). The quantization processing unit 14 also reads the weight U from the weight buffer 11 and quantizes the value of each element (step S4).

  More specifically, as shown in FIG. 4, the value of the input signal A including the decimal point is rounded off by the quantization processing unit 14, for example. The quantization processing unit 14 updates the value of the input signal A of the input buffer 10 with the value of the quantized input signal A ′.

  Further, as shown in FIG. 4, the value of the weight U including the decimal point is rounded off by the quantization processing unit 14, for example. The quantization processing unit 14 updates the value of the weight U of the weight buffer 11 with the value of the quantized weight U ′.

  Returning to FIG. 3, the convolution operation unit 12 reads the quantized input signal A ′ and the quantized weight U ′ from the input buffer 10 and the weight buffer 11, respectively, and performs the convolution operation (step S5). More specifically, the convolution operation unit 12 multiplies the vector of the quantized input signal A ′ by the matrix of the quantized weight U ′.

  Specifically, as shown in FIG. 4, the window of the CNN filter (in the example of FIG. 4, the thick line rectangle indicated by 2 × 2) is slid by a predetermined stride. The convolution operation unit 12 multiplies the element of the quantized weight U ′ by the corresponding element of the quantized input signal A ′ at each position of the filter, and obtains the sum.

  The convolution operation unit 12 stores the operation result B of the convolution operation by this product-sum operation in the corresponding location of the operation result buffer 13 (step S6). Specifically, as shown in FIG. 4, the calculation result buffer 13 stores a product-sum result “37” that does not include a decimal point.

  After that, the quantization processing unit 14 reads the operation result of the convolution operation obtained in step S5 from the operation result buffer 13 and performs the quantization process (step S7). Specifically, as illustrated in FIG. 4, the quantization processing unit 14 rounds the operation result “37” of the convolution operation, for example, and quantizes it to “40”. The quantization processing unit 14 updates the data in the operation result buffer 13 with the quantized operation result B '.

  Next, the processor 102 of the CNN processing device 1 reads the quantized convolution operation result B ′ from the operation result buffer 13 and applies an activation function such as ReLU (step S8). Specifically, the processor 102 outputs 0 through the ReLU function when the calculation result B ′ is a negative value, and outputs the positive calculation result B ′ as it is.

  Next, the processor 102 performs well-known pooling processing on the value output in step S8, and compresses the result of the convolution operation (step S9). The pooling process in step S9 may be performed as necessary. Further, the processor 102 may normalize the output of the activation function (ReLU) obtained in step S8 (see Non-Patent Document 1).

  The quantized operation result B'which has been subjected to the pooling processing is stored in the output buffer 15, read out by the processor 102 and output (step S10). The output value is input as an output of the feature extraction unit of the CNN to the all-connection layer that constitutes a subsequent classifier (not shown), and the image data of the input signal A is determined.

  As described above, according to the CNN processing apparatus 1 according to the first embodiment, the CNN input signal, the weight, and the operation result of the convolution operation are quantized. Even in this case, it is possible to suppress a decrease in CNN processing speed.

  Further, according to the CNN processing device 1, in particular, the calculation result of the convolution calculation is also quantized, so that the calculation load of the entire CNN including a large number of layers can be reduced and the signal processing can be speeded up. .

  In the embodiment described above, in the description of the product-sum calculation, the addition of bias is omitted for simplicity of description. However, the vector of the input signal may be multiplied by the weight matrix, and the bias value added after that may be quantized in the same manner.

  Moreover, in the embodiment described, the case where the quantization processing unit 14 quantizes all data of the input signal, the weight, and the calculation result of the convolution calculation has been described. However, the quantization processing unit 14 may be configured to quantize the data included in any one or two of the input signal, the weight, and the operation result of the convolution operation.

[Second Embodiment]
Next, a second embodiment of the present invention will be described. In the following description, the same components as those in the first embodiment described above will be designated by the same reference numerals and the description thereof will be omitted.

  In the first embodiment, the case has been described where the quantization processing unit 14 respectively performs the quantization processing on the input signal A, the weight U, and the operation result B of the convolution operation in the CNN. On the other hand, the CNN processing device 1A according to the second embodiment uses the weight U'quantized in advance. The CNN processing device 1A also includes a quantization convolution operation unit 12A that performs a convolution operation that incorporates a quantization process.

[Functional block of CNN processing device]
FIG. 5 is a block diagram showing a functional configuration of the CNN processing device 1A according to the second embodiment. The CNN processing device 1A includes an input buffer 10, a quantization weight buffer (second memory) 11A, a quantization convolution operation unit 12A, an operation result buffer 13, an output buffer 15, and a storage unit 16. Hereinafter, the configuration different from that of the first embodiment will be mainly described.

  The quantization weight buffer 11A stores weights (quantization weights) whose weights are quantized in advance. The quantization weight buffer 11A temporarily stores, for example, pre-quantized weights obtained from a server (not shown) installed outside the CNN processing device 1A. For example, the CNN processing device 1A may acquire the learned CNN weights from a server or the like installed outside in advance via the communication network NW and store the weights in the storage unit 16 in advance. In this case, the quantization weight buffer 11A reads the quantized weight from the storage unit 16 and temporarily stores it. Here, the quantized weight is a weight reduced to bit precision according to the processing capacity of the CNN processing device 1A, as in the first embodiment. For example, the weight including the decimal point is converted into an integer. Included.

  The quantization convolution operation unit 12A reads the input signal and the quantized weight from the input buffer 10 and the quantization weight buffer 11A, respectively, and performs the convolution operation. More specifically, the quantization convolution operation unit 12A also performs a signal quantization process when performing the convolution operation. For example, the quantization convolution operation unit 12A performs a process of limiting the number of digits of the operation result of the convolution operation to an integer in advance, or an operation of performing a bit shift before the calculation when the number of digits of the operation result is limited. Is performed when executing the convolution operation. The calculation result by the quantization convolution calculation unit 12A is temporarily stored in the calculation result buffer 13.

[CNN processing method]
Next, the operation of the CNN processing device 1A having the above configuration will be described with reference to FIGS. 6 and 7. First, it is assumed that the weight U ′ quantized by an external server or the like is acquired by the CNN processing device 1 via the communication network NW, and the quantized weight U ′ is stored in the storage unit 16.

  First, the input buffer 10 temporarily stores the input signal A acquired from a server or the like installed outside the CNN processing device 1 (step S20). In addition, the quantization weight buffer 11A reads the weight U'quantized in advance from the storage unit 16 and temporarily stores it (step S21).

  As shown in FIG. 7, the input signal A is stored in the input buffer 10. The input signal A is vectorized input image data and has vertical and horizontal dimensions. The value of the input signal A is represented by a value including a decimal point. The rectangle shown by the thick line in FIG. 7 represents a filter. The quantized weight buffer 11A stores the quantized weight U '. The weight U is an element of the kernel represented by a matrix, and is a parameter that is adjusted and updated by learning of CNN and finally determined. The quantized weight U ′ is a weight that represents, for example, the weight U including each element including a decimal point value by converting it into an integer to reduce the number of bits. Similarly to the weight U, the quantized weight U'has vertical and horizontal dimensions.

  Next, the quantization convolution operation unit 12A reads the input signal A from the input buffer 10 and the quantized weight U ′ from the quantization weight buffer 11A, and performs the convolution operation incorporating the quantization processing (step S22). ). For example, the quantization convolution operation unit 12A performs a process of limiting the number of digits of the operation result of the convolution operation to an integer. In addition, for example, the quantization convolution operation unit 12A performs a bit shift in advance on the input signal A and the quantized weight U ′ before performing the convolution operation, and performs processing on the number of digits of data.

  More specifically, as shown in the example of FIG. 7, the quantization convolution operation unit 12A performs a process of rounding down the decimal point during the convolution operation (C = floor (conv (A, U ')). Further, the quantization convolution operation unit 12A multiplies the element of the quantized weight U ′ and the corresponding element of the input signal A at each position of the filter, and obtains the sum. The operation result C of the quantized convolution operation by 12A is stored in the operation result buffer 13 (step S23).

  After that, the processor 102 of the CNN processing device 1A reads the quantized convolution operation result C from the operation result buffer 13 and applies an activation function such as ReLU (step S24). Specifically, the processor 102 outputs 0 through the ReLU function when the calculation result C has a negative value, and outputs the same value as the positive calculation result C.

  Next, the processor 102 performs well-known pooling processing on the value output in step S24, and compresses the result of the convolution operation (step S25). The pooling process in step S25 may be performed as needed. Further, the processor 102 may normalize the output of the activation function (ReLU) obtained in step S24 (see Non-Patent Document 1).

  The pooled quantized convolutional calculation result C is stored in the output buffer 15, further read by the processor 102 and output to the outside (step S26). The output value is input as an output of the feature extraction unit of the CNN to the subsequent all-connection layer that constitutes a classifier (not shown) to determine the image data of the input signal A.

  As described above, according to the CNN processing device 1A according to the second embodiment, it is possible to reduce the calculation processing in the CNN convolution calculation by using the quantized weight in advance. In addition, since the CNN processing device 1A incorporates the quantization processing when performing the convolution operation, it is possible to reduce the calculation processing associated with the quantization of the input signal and the weight.

[Third Embodiment]
Next, a third embodiment of the present invention will be described. In the following description, the same components as those in the above-described first and second embodiments will be designated by the same reference numerals and the description thereof will be omitted.

  In the second embodiment, the case where the quantized convolution operation unit 12A performs the convolution operation in which the quantization process is incorporated by using the quantized weight in advance has been described. On the other hand, in the third embodiment, when the weight quantization and the quantization convolution operation are performed, the function stored in the storage unit 16B in advance is read and the operation is performed.

[Functional block of CNN processing device]
FIG. 8 is a block diagram showing a functional configuration of the CNN processing device 1B according to the third embodiment. The CNN processing device 1B includes an input buffer 10, a quantization weight buffer 11A, a convolution operation unit 12, an operation result buffer 13, a quantization processing unit 14, an output buffer 15, and a storage unit (third memory, fourth memory) 16B. Prepare

The storage unit 16B stores a predefined weight quantization function (first function) 160 and a convolution operation quantization function (second function) 161.
The weight quantization function 160 is a function that realizes weight quantization. More specifically, in the weight quantization function 160, the quantization processing unit 14 performs preset fraction processing on the weight U, for example, integerization of the weight U including a decimal point to reduce the number of data. This is a function that realizes the quantization process.

  The convolution operation quantization function 161 is a function that realizes a convolution operation that incorporates quantization processing. More specifically, the convolutional operation quantization function 161 is a quantization process that limits the number of digits of the operation result when the convolutional operation unit 12 performs a convolutional operation, or performs bit shift of a value such as an input signal in advance. Is a function that realizes with the convolution operation.

  The quantization processing unit 14 calls the weight quantization function 160 from the storage unit 16B and executes the quantization of the weight U. The value of the weight U to be quantized is stored in the storage unit 16B in advance. The weight U'quantized by the quantization processing unit 14 using the weight quantization function 160 is temporarily stored in the quantization weight buffer 11A.

  The convolution operation unit 12 calls the convolution operation quantization function 161 stored in the storage unit 16B to execute the quantization convolution operation. More specifically, the convolution operation unit 12 reads the quantized weight U ′ from the quantization weight buffer 11A. The convolution operation unit 12 also reads the input signal A from the input buffer 10. Then, the convolution operation unit 12 performs a convolution operation in which a quantization process is incorporated using the convolution operation quantization function 161 that has been called based on the input signal A and the quantized weight U ′. The calculation result obtained by using the function by the convolution calculation unit 12 is stored in the calculation result buffer 13.

[CNN processing method]
Next, the operation of the CNN processing device 1B having the above configuration will be described with reference to FIGS. 9 and 10.

  First, the quantization processing unit 14 calls the weight quantization function 160 stored in advance in the storage unit 16B to quantize the weight U (step S30). The quantized weight U'is temporarily stored in the quantization weight buffer 11A (step S31). For example, as shown in FIG. 10, the weighting quantization function 160 corresponding to the quantized weight U ′ is called from the storage unit 16B by the quantization processing unit 14. In the example of FIG. 10, as the weight quantization function 160, the function indicated by “weight quantization function 1” is assigned to each element.

  Next, the input buffer 10 temporarily stores the input signal A acquired from a server or the like installed outside the CNN processing device 1 (step S32).

  Next, the convolution operation unit 12 calls the convolution operation quantization function 161 from the storage unit 16B and performs the convolution operation in which the quantization processing is incorporated (step S33). More specifically, the convolution operation unit 12 reads the input signal A from the input buffer 10 and the quantized weight U ′ from the quantization weight buffer 11A. The convolution operation unit 12 performs a convolution operation including a quantization process using the convolution operation quantization function 161 based on the input signal A and the quantized weight U ′. For example, the convolution operation unit 12 performs a process of limiting the number of digits of the operation result of the convolution operation to an integer according to the called function. In addition, for example, the quantization convolution operation unit 12A performs a bit shift in advance on the input signal A and the quantized weight U ′ before performing the convolution operation, and performs processing on the number of digits of data.

  More specifically, as shown in the example of FIG. 9, the convolution operation unit 12 performs a quantization process that rounds down the decimal point in the convolution operation according to the function 1 of the convolution operation quantization function 161 (floor ( conv)). Further, the convolution operation unit 12 multiplies the element of the quantized weight U ′ and the corresponding element of the input signal A at each position of the filter according to the convolution operation quantization function 161, and obtains the sum. . The operation result X of the quantized convolution operation performed by the quantization convolution operation unit 12A is temporarily stored in the operation result buffer 13 (step S34).

  Thereafter, the processor 102 of the CNN processing device 1B reads the quantized operation result X of the convolution operation from the operation result buffer 13 and applies an activation function such as ReLU (step S35). Specifically, when the calculation result X is a negative value, the processor 102 outputs 0 through the ReLU function, and the positive calculation result X outputs the same value.

  Next, the processor 102 performs well-known pooling processing on the value output in step S35, and compresses the result of the convolution operation (step S36). The pooling process in step S35 may be performed as needed. Further, the processor 102 may normalize the output of the activation function (ReLU) obtained in step S35 (see Non-Patent Document 1).

  The pooled quantized convolutional calculation result X is temporarily stored in the output buffer 15, is further read by the processor 102, and is output to the outside (step S37). The output value is input as an output of the feature extraction unit of the CNN to the subsequent all-connection layer that constitutes a classifier (not shown) to determine the image data of the input signal A.

  As described above, according to the CNN processing device 1B according to the third embodiment, the quantization of the weight and the quantization using the quantization function 160 of the weight defined in advance and the quantization function 161 of the convolution operation are performed. Performs a convolution operation. Therefore, in the operation of the convolutional layers connected in multiple stages, by exchanging the functions, all the convolutional operations can be defined, and the amount of programs can be suppressed and the decrease in CNN processing speed can be suppressed.

  The weight quantization function 160 and the convolution operation quantization function 161 according to the above-described embodiments may be programs that can be incorporated in hardware. For example, the source code of the FPGA or the firmware of the microcomputer corresponds to this, and in this case, the hardware IP or the like can be separately provided.

  Further, the weight quantization function 160 and the convolution operation quantization function 161 configured as a program that can be incorporated in hardware may be in a memory in the same hardware, or are stored in another network facility or the like. May be. By replacing the above program according to the form of the neural network, hardware that flexibly realizes the neural network having a desired processing function can be realized.

  Although the embodiments of the neural network processing apparatus and the neural network processing method of the present invention have been described above, the present invention is not limited to the above described embodiments, and is applicable within the scope of the invention described in the claims. Various modifications that can be envisioned by a trader can be made.

  For example, although the CNN has been described as an example of the neural network in the described embodiment, the neural network adopted by the neural network processing device is not limited to the CNN.

  It should be noted that the various functional blocks, modules, and circuits described in connection with the embodiments disclosed herein are general purpose processors, GPUs, digital signal processors (DSPs), application specific integrated circuits (ASICs), FPGAs. Alternatively, it may be implemented using other programmable logic devices, discrete gate or transistor logic, discrete hardware components, or any combination of the above designed to implement the functions described above.

  A general purpose processor may be a microprocessor, but in the alternative, the processor may be any conventional processor, controller, microcontroller, or state machine. The processor may be implemented, for example, as a combination of a DSP and a microprocessor, multiple microprocessors, one or more microprocessors connected to a DSP core, or a combination of computing devices of any such configuration. Is.

  DESCRIPTION OF SYMBOLS 1 ... CNN processing device, 10 ... Input buffer, 11 ... Weight buffer, 12 ... Convolution operation part, 13 ... Operation result buffer, 14 ... Quantization processing part, 15 ... Output buffer, 16 ... Storage part, 101 ... Bus, 102 ... processor, 103 ... main storage device, 104 ... communication interface, 105 ... auxiliary storage device, 106 ... input / output device, NW ... communication network, U ... weight, U '... quantized weight, A ... input signal, A '... Quantized input signal.

Claims (7)

A first memory for storing an input signal given to the neural network;
A second memory for storing the weights of the neural network;
A first process for performing a convolutional operation of the neural network including a product-sum operation of the input signal and the weight; and a second process for performing quantization for reducing bit precision of at least a part of data used in the first process. A neural network processing device comprising: a processor for performing processing. The neural network processing apparatus according to claim 1,
The said 2nd process includes the said input signal, the said weight, and the quantization which reduces the bit precision of at least 1 data of the calculation result of the said product sum calculation in the said 1st process. The neural network processing apparatus characterized by the above-mentioned. . The neural network processing apparatus according to claim 1,
The said 2nd memory memorize | stores the quantization weight which the said weight was quantized beforehand, The neural network processing apparatus characterized by the above-mentioned. The neural network processing device according to any one of claims 1 to 3,
The neural network processing apparatus, wherein the processor includes a quantization convolution operation unit that performs the first process in which the second process is incorporated, based on the weight and the input signal. The neural network processing apparatus according to claim 1,
A third memory storing a first function for quantizing the weights;
A fourth memory that stores a second function that implements the first process in which the second process is incorporated;
Further equipped with,
The neural network processing device, wherein the processor reads the first function, quantizes the weight, reads the second function, and performs the first process in which the second process is incorporated. The neural network processing device according to any one of claims 1 to 5,
The neural network processing device, wherein the neural network is a multilayer neural network having at least one intermediate layer. A first step of storing an input signal given to the neural network in a first memory;
A second step of storing the weights of the neural network in a second memory;
A first process in which the processor performs a convolution operation of the neural network including a product-sum operation of the input signal and the weight;
A second process of performing a quantization for reducing the bit precision of at least a part of the data used in the first process, and a third step of performing: a neural network processing method.

Images