TWI766193B - Convolutional neural network processor and data processing method thereof - Google Patents

Abstract

A convolutional neural network processor includes an Information Decode Unit (IDU) and a Convolutional Neural Network Inference Unit (CIU). The IDU receives a program input and a plurality of parameter inputs, and the IDU includes a decoding module and a parallel processing module. The decoding module receives the program input, and the decoding module produces an operating command according to the program input. The parallel processing module is electrically connected to the decoding module and receives the parameter inputs. The parallel processing module includes a plurality of parallel processing sub-modules. The parallel processing sub-modules produce a plurality of output weight parameters. The CIU is electrically connected to the IDU and includes a computing module. The computing module is electrically connected to the parallel processing module, and the computing module produces an output data according to an input data and the output weight parameter. Therefore, the convolutional neural network processor can perform a highly parallel computation.

Description

Convolutional neural network processor and data processing method thereof

The present invention relates to a convolutional neural network processor and a data processing method thereof, and more particularly, to a convolutional neural network processor and a data processing method thereof having a message decoding unit and a convolution judgment unit.

Convolutional Neural Networks (CNN) have recently been widely used in the fields of computer vision and image processing. However, recent applications are more focused on object recognition and object detection. Therefore, the hardware design of convolutional neural networks is not optimized for image processing networks, because the above applications do not consider (1) spatial resolution Rates are not heavily downsampled and (2) the model sparsity fails, resulting in extremely high memory bandwidth and extremely high computing power requirements.

In view of this, the present invention designs a convolutional neural network processor that can perform highly parallel operations and a data processing method thereof to provide high-performance operations.

The convolutional neural network processor and the data processing method thereof provided by the present invention can perform highly parallel operations through the information decoding unit and the convolution determination unit.

According to an embodiment of the present invention, a convolutional neural network processor is provided for computing input data, and the convolutional neural network processor includes a message decoding unit and a convolution judgment unit. The message decoding unit is used for receiving the input program and the complex input weight parameter, and includes a decoding module and a parallel processing module. The decoding module receives the input program and outputs the operation command according to the input program. The parallel processing module is electrically connected to the decoding module, and receives input weight parameters. The parallel processing module includes a plurality of parallel processing sub-modules, and the parallel processing sub-module generates complex output weight parameters according to the operation command and the input weight parameters. The convolution determination unit is electrically connected with the message decoding unit, and includes an operation module. The operation module is electrically connected with the parallel processing module, and the operation module generates output data according to the operation of the input data and the output weight parameter.

In this way, the convolutional neural network processor can perform highly parallel operations through the decoding module and parallel processing module of the message decoding unit, and the operation module of the convolution judgment unit, thereby providing high-performance and low-power computing .

The convolutional neural network processor according to the embodiment described in the preceding paragraph, wherein the decoding module includes a program memory and an instruction decoder. Program memory stores input programs. The instruction decoder is electrically connected with the program memory, and the instruction decoder decodes the input program to output the operation instruction.

According to the convolutional neural network processor of the embodiment described in the preceding paragraph, when the input weight parameter is a complex uncompressed input weight parameter, the parallel processing sub-module includes a complex parallel sub-memory and a complex parallel sub-processor. The complex parallel sub-memory stores the uncompressed input weight parameters in parallel. The plurality of parallel sub-processors are respectively electrically connected with the decoding module and the parallel sub-memory, and the parallel sub-processors receive the uncompressed input weight parameters in parallel according to the operation instruction, and generate the output weight parameters.

According to the convolutional neural network processor of the embodiment described in the preceding paragraph, when the input weight parameter is a complex compression input weight parameter, the parallel processing sub-module includes a complex parallel sub-memory and a complex parallel sub-processor. The complex parallel sub-memory stores the compressed input weight parameters in parallel. The plurality of parallel sub-processors are respectively electrically connected to the decoding module and the parallel sub-memory, and the parallel sub-processors receive and decompress and compress the input weight parameters in parallel according to the operation instruction, and generate the output weight parameters.

The convolutional neural network processor according to the embodiment described in the preceding paragraph, wherein the input weight parameter includes a complex number of first input weight parameters, and the output weight parameter includes a complex number of first output weight parameters. The parallel processing sub-module includes a plurality of parallel sub-memory and a plurality of parallel sub-processors. The plurality of parallel sub-memory stores the input weight parameters in parallel, and the parallel sub-memory includes a plurality of first parallel sub-memory. The plurality of first parallel sub-memory respectively and in parallel receive and store the first input weight parameters. The plurality of parallel sub-processors are respectively electrically connected with the decoding module and the parallel sub-memory, and the parallel sub-processors include a plurality of first parallel sub-processors. The plurality of first parallel sub-processors are respectively connected with the first parallel sub-processors The row sub-memory is electrically connected, and receives the first input weight parameter according to the operation command to output the first output weight parameter.

The convolutional neural network processor according to the embodiment described in the preceding paragraph, wherein the first output weight parameter comprises a complex 3×3 weight parameter. The operation module includes 3×3 operation sub-modules. The 3×3 operation sub-module is electrically connected to the first parallel sub-processor, and performs operations with the input data according to the first output weight parameter to generate 3×3 post-processing operation data. The 3×3 operation sub-module includes complex numbers 3×3 convolution distributor group, complex 3×3 local convolution operation unit and complex 3×3 post-processing operation unit. Each 3×3 convolutional distributor group is electrically connected with a first parallel sub-processor, and the 3×3 convolutional distributor group is used for receiving and distributing the 3×3 weight parameters of the first output weight parameter. The complex 3×3 local convolution operation units are respectively electrically connected with a 3×3 convolution distributor group, and each 3×3 local convolution operation unit includes a 3×3 local register group and a 3×3 local filter operation unit. The 3×3 local register group is electrically connected with a 3×3 convolution distributor group, and the 3×3 local register group of the 3×3 local convolution operation unit receives and stores the 3×3 first output weight parameters A weight parameter, and according to the 3×3 weight parameter of the first output weight parameter, a complex 3×3 operation parameter is output. The 3×3 local filter operation unit is electrically connected to the 3×3 local register group, and the 3×3 local filter operation unit of the 3×3 local convolution operation unit performs operations according to the 3×3 operation parameters and the input data to generate complex numbers 3×3 operation data. The complex 3×3 post-processing operation unit is electrically connected to the 3×3 local convolution operation unit, and performs 3×3 post-processing operation according to the 3×3 operation data to generate 3×3 post-processing operation data, wherein the output data is 3×3 post-processing operation data.

The convolutional neural network processor according to the embodiment described in the preceding paragraph, wherein each 3×3 local register set includes two sub-3×3 local register sets. The two sub 3×3 local register groups alternately store a 3×3 weight parameter or output a 3×3 operation parameter to the 3×3 local filtering operation unit.

According to the convolutional neural network processor of the embodiment described in the preceding paragraph, the input weight parameter further includes a bias input weight parameter, and the output weight parameter further includes a bias output weight parameter. The parallel sub-memory further includes a biased parallel sub-memory. The bias parallel sub-memory stores the bias input weight parameters in parallel. The parallel sub-processor further includes a biased parallel sub-processor. The bias parallel sub-processor is electrically connected to the bias parallel sub-memory, receives the bias input weight parameter according to the operation command, and outputs the bias output weight parameter.

The convolutional neural network processor according to the embodiment described in the preceding paragraph, wherein the bias output weight parameter comprises a complex bias weight parameter. The computing module further includes a bias distributor. The bias divider is electrically connected with the bias parallel sub-processor and the 3×3 operation sub-module. The bias divider outputs the weight parameter according to the bias voltage to generate a complex 3×3 bias weight parameter, and the 3×3 bias The pressure weight parameters are output to the 3×3 post-processing operation unit.

According to the convolutional neural network processor of the embodiment described in the preceding paragraph, the input weight parameter further includes at least one second input weight parameter, and the output weight parameter further includes at least one second output weight parameter. The parallel sub-memory further includes at least one second parallel sub-memory. The second parallel sub-memory respectively and in parallel receive and store at least one second input weight parameter. The parallel sub-processor further includes at least one second parallel sub-processor. The at least one second parallel sub-processor is electrically connected to the at least one second parallel sub-memory respectively, and the root At least one second input weight parameter is received according to the operation command to output at least one second output weight parameter.

According to the convolutional neural network processor of the embodiment described in the preceding paragraph, the operation module includes a 3×3 operation sub-module and a 1×1 operation sub-module. The 3×3 operation sub-module is electrically connected to the first parallel sub-processor, and performs operation on the input data according to the first output weight parameter to generate 3×3 post-processing operation data. The 1×1 operation sub-module is electrically connected with at least one second parallel sub-processor and the 3×3 operation sub-module, and performs operations according to at least one second output weight parameter and 3×3 post-processing operation data to generate 1×1 post-processing operation data, wherein the output data can be 1×1 post-processing operation data.

According to the convolutional neural network processor of the embodiment described in the preceding paragraph, the at least one second output weight parameter includes a complex number of 1×1 weight parameters. The 1×1 operation sub-module includes at least one 1×1 convolution distributor group, a complex 1×1 local convolution operation unit and a complex 1×1 post-processing operation unit. The 1×1 convolutional distributor group is electrically connected to the at least one second parallel sub-processor for receiving and distributing the 1×1 weight parameter of the at least one second output weight parameter. The complex 1×1 local convolution operation unit is electrically connected with at least one 1×1 convolution distributor, and each 1×1 local convolution operation unit includes a 1×1 local register group and a 1×1 local filter operation unit. The 1×1 local register group is electrically connected to at least one 1×1 convolution distributor group, and the 1×1 local register group of the 1×1 local convolution operation unit receives and stores 1× of the second output weight parameter 1 weight parameter, and according to the 1×1 weight parameter of the at least one second output weight parameter, a complex 1×1 operation parameter is output. The 1×1 local filter operation unit is electrically connected to the 1×1 local register group, and the 1×1 local filter operation unit of the 1×1 local convolution operation unit is based on the The 1×1 operation parameters are operated on with the 3×3 post-processing operation data to generate complex 1×1 operation data. The complex 1×1 post-processing operation unit is electrically connected to the 1×1 local convolution operation unit, and performs 1×1 post-processing operation according to the 1×1 operation data to generate 1×1 post-processing operation data.

The convolutional neural network processor according to the embodiment described in the preceding paragraph, wherein each 1×1 local register group includes two sub-1×1 local register groups. The two sub 1×1 local register groups alternately store a 1×1 weight parameter or output a 1×1 operation parameter to a 1×1 operation unit.

According to the convolutional neural network processor of the embodiment described in the preceding paragraph, the input weight parameter further includes a bias input weight parameter, and the output weight parameter further includes a bias output weight parameter. The parallel sub-memory further includes a biased parallel sub-memory. The bias parallel sub-memory stores the bias input weight parameters in parallel. The parallel sub-processor further includes a biased parallel sub-processor. The bias parallel sub-processor is electrically connected to the bias parallel sub-memory, receives the bias input weight parameter according to the operation command, and outputs the bias output weight parameter.

The convolutional neural network processor according to the embodiment described in the preceding paragraph, wherein the bias output weight parameter comprises a complex bias weight parameter. The computing module further includes a bias distributor. The bias divider is electrically connected to the bias parallel sub-processor, the 3×3 operation sub-module and the 1×1 operation sub-module, and the bias divider generates a complex 3×3 bias weight according to the bias output weight parameter parameter and complex 1×1 bias weight parameter. The bias distributor outputs the 3×3 bias weight parameters to the 3×3 post-processing arithmetic unit. The bias distributor outputs the 1×1 bias weight parameters to the 1×1 post-processing arithmetic unit.

According to an embodiment of the present invention, a data processing method of a convolutional neural network processor is provided, which includes a receiving step, an instruction decoding step, a parallel processing step, and an operation step. The receiving step drives the message decoding unit to receive the input program and the complex input weight parameter, wherein the message decoding unit includes a decoding module and a parallel processing module. The instruction decoding step drives the decoding module to receive the input program and generate operation instructions according to the input program. The parallel processing step drives the parallel processing module to receive the input weight parameters, and process the input weight parameters in parallel according to the operation instruction to generate complex output weight parameters. The operation step drives the operation module to receive the input data and the output weight parameter, and operate the input data and the output weight parameter according to the operation instruction to generate the output data.

Thereby, the data processing method of the convolutional neural network processor can drive the decoding module and parallel processing module of the information decoding unit, and the convolution judgment unit through the receiving step, the instruction decoding step, the parallel processing step and the operation step. The computing module performs highly parallel computing, thereby providing high-performance and low-power computing.

According to the data processing method of a convolutional neural network processor according to the above-mentioned embodiment, the decoding module includes a program memory and an instruction decoder, and the instruction decoding step includes a program storage sub-step and a program decoding sub-step. Program storage sub-step driver memory stores input programs. The program decoding sub-step drives the instruction decoder to decode the input program to generate operation instructions.

The data processing method of a convolutional neural network processor according to the embodiment described in the preceding paragraph, wherein the parallel processing module includes a complex parallel sub-memory Volume and complex parallel sub-processors, and the parallel processing step includes a weight parameter storage sub-step and a weight parameter processing sub-step. The weight parameter storage sub-step drives the parallel sub-memory to store the input weight parameters in parallel. The weight parameter processing sub-step drives the parallel sub-processor, and the parallel sub-processor reads the input weight parameter in parallel according to the operation instruction and performs operation processing to generate the output weight parameter.

According to the data processing method of the convolutional neural network processor according to the embodiment described in the preceding paragraph, when the input weight parameter is a complex number of uncompressed input weight parameters, the operation process is to store the uncompressed input weight parameters. When the input weight parameter is a complex compressed input weight parameter, the operation process is to store and decompress the compressed input weight parameter.

According to the data processing method for a convolutional neural network processor according to the embodiment described in the preceding paragraph, the output weight parameter includes a complex number of first output weight parameters. The operation module includes 3×3 operation sub-modules. The operation step includes a first operation sub-step. The first operation sub-step drives the 3×3 operation sub-module to receive the input data and the first output weight parameter to generate 3×3 post-processing operation data.

According to the data processing method for a convolutional neural network processor according to the embodiment described in the preceding paragraph, each of the first output weight parameters includes a complex number of 3×3 weight parameters. The 3×3 operation sub-module includes a complex 3×3 convolution distributor group, a complex 3×3 local convolution operation unit and a complex 3×3 post-processing operation unit. The first operation sub-step includes a 3×3 parameter allocation procedure, a 3×3 operation parameter generation procedure, a 3×3 convolution operation procedure and a 3×3 post-processing operation procedure. The 3×3 parameter assignment program drives the 3×3 convolutional assigner group to receive 3×3 of the first output weight parameters weight parameter, and assign the 3×3 weight parameter of the first output weight parameter to the 3×3 local convolution operation unit. Each 3×3 local convolution operation unit includes a 3×3 local register group and a 3×3 local filter operation unit. The 3×3 operation parameter generation program drives the 3×3 local register group of the 3×3 local convolution operation unit to receive the 3×3 weight parameter of the first output weight parameter, and according to the 3×3 weight of the first output weight parameter The parameter yields a complex 3x3 arithmetic parameter. The 3×3 convolution operation program drives the 3×3 local filter operation unit of the 3×3 local convolution operation unit to perform a 3×3 convolution operation on the 3×3 operation parameters and input data to generate complex 3×3 operation data . The 3×3 post-processing operation program drives the 3×3 post-processing operation unit to perform 3×3 post-processing operations on the 3×3 operation data to generate 3×3 post-processing operation data, wherein the output data is the 3×3 post-processing operation material.

According to the data processing method for a convolutional neural network processor according to the embodiment described in the preceding paragraph, the output weight parameter further includes a bias output weight parameter. The computing module further includes a bias distributor. The operation step further includes a bias voltage operation sub-step. The bias operation sub-step drives the bias distributor to output weight parameters according to the bias voltage to generate complex 3×3 bias weight parameters, and the bias distributor provides the 3×3 bias weight parameters to the 3×3 operation sub-module.

According to the data processing method of the convolutional neural network processor according to the embodiment described in the preceding paragraph, the output weight parameter further includes at least one second output weight parameter. The operation module contains 1×1 operation sub-modules. The operation step further includes a second operation sub-step. The second operation sub-step drives the 1×1 operation sub-module to receive 3×3 post-processing operation data and at least one second output weight parameter to generate 1×1 post-processing operation data.

According to the data processing method for a convolutional neural network processor according to the embodiment described in the preceding paragraph, the at least one second output weight parameter includes a complex 1×1 weight parameter. The 1×1 operation sub-module includes a complex 1×1 convolution distributor group, a complex 1×1 local convolution operation unit and a complex 1×1 post-processing operation unit. The second operation sub-step includes a 1×1 parameter allocation procedure, a 1×1 operation parameter generation procedure, a 1×1 convolution operation procedure, and a 1×1 post-processing operation procedure. The 1×1 parameter assignment program drives the 1×1 convolutional assigner group to receive the 1×1 weight parameter of the at least one second output weight parameter, and assign the 1×1 weight parameter of the at least one second output weight parameter to the 1×1 A local convolution operation unit, wherein each 1×1 local convolution operation unit includes a 1×1 local register group and a 1×1 local filter operation unit. The 1×1 operation parameter generation program drives the 1×1 local register group of the 1×1 local convolution operation unit to receive the 1×1 weight parameter of the at least one second output weight parameter, and according to the at least one second output weight parameter The 1×1 weight parameter produces a complex 1×1 operation parameter. The 1×1 convolution operation program drives the 1×1 local filter operation unit of the 1×1 local convolution operation unit to perform 1×1 convolution operation on the 1×1 operation parameters and 3×3 post-processing operation data to generate complex numbers 1×1 operation data. The 1×1 post-processing operation program drives the 1×1 post-processing operation unit to perform 1×1 post-processing operations on the 1×1 operation data to generate 1×1 post-processing operation data, wherein the output data is the 1×1 post-processing operation material.

According to the data processing method for a convolutional neural network processor according to the embodiment described in the preceding paragraph, the output weight parameter further includes a bias output weight parameter. The computing module further includes a bias distributor. The operation step further includes a bias voltage operation sub-step. The bias operation sub-step drives the bias distributor to output the weight parameter according to the bias to generate a complex 3×3 bias weight parameter and a complex 1×1 bias Weight parameters, wherein the bias distributor provides 3×3 bias weight parameters to the 3×3 arithmetic sub-modules, and provides 1×1 bias weight parameters to the 1×1 arithmetic sub-modules.

100‧‧‧Convolutional Neural Network Processor

102‧‧‧Input Program

104‧‧‧Input Weight Parameters

106‧‧‧Enter data

1062‧‧‧3×3 post-processing operation data

1064‧‧‧1×1 post-processing operation data

108‧‧‧Output data

110‧‧‧Message Decoding Unit

111‧‧‧Decoding Module

1111‧‧‧Program memory

1112‧‧‧Instruction Decoder

112‧‧‧Parallel processing module

1121‧‧‧Parallel Processing Submodule

1121a‧‧‧Parallel sub-memory

1121aa‧‧‧First parallel sub-memory

1121ab‧‧‧biased parallel sub-memory

1121ac‧‧‧Second parallel sub-memory

1213d‧‧‧1×1 local filtering operation unit

1213e‧‧‧1×1 post-processing operation unit

1213f‧‧‧First 1×1 convolution distributor

1213g‧‧‧Second 1×1 convolution distributor

122‧‧‧Controller

s200‧‧‧Data processing method of convolutional neural network processor

s210‧‧‧Receive steps

s220‧‧‧ instruction decoding steps

s221‧‧‧Program storage sub-step

s222‧‧‧Program decoding sub-step

s230‧‧‧Parallel processing steps

s231‧‧‧Weight parameter storage sub step

1121b‧‧‧Parallel Subprocessor

1121ba‧‧‧First parallel sub-processor

1121bb‧‧‧biased parallel sub-processor

1121bc‧‧‧Second Parallel Subprocessor

120‧‧‧Convolution judgment unit

121‧‧‧Computing Modules

1211‧‧‧3×3 operation sub-module

1211a‧‧‧3×3 arithmetic circuit

1211b‧‧‧3×3 local convolution operation unit

1211c‧‧‧3×3 local register bank

1211ca, 1211cb‧‧‧Sub 3×3 local register bank

1211d‧‧‧3×3 local filtering operation unit

1211e‧‧‧3×3 post-processing operation unit

1211f‧‧‧First 3×3 convolutional divider

1211g‧‧‧Second 3×3 convolution distributor

1212‧‧‧bias distributor

1213‧‧‧1×1 arithmetic sub-module

1213a‧‧‧1×1 arithmetic circuit

1213b‧‧‧1×1 local convolution operation unit

1213c‧‧‧1×1 local register bank

1213ca, 1213cb‧‧‧Sub 1×1 local register bank

s232‧‧‧ weight parameter processing sub-step

s240‧‧‧Operation steps

s241‧‧‧First operator substep

s2411‧‧‧3×3 parameter assignment program

s2412‧‧‧3×3 operation parameter generation program

s2413‧‧‧3×3 convolution operation program

s2414‧‧‧3×3 post-processing operation program

s242‧‧‧Bias operation sub-step

s243‧‧‧Second operator substep

s2431‧‧‧1×1 parameter assignment program

s2432‧‧‧1×1 operation parameter generation program

s2433‧‧‧1×1 convolution operation program

s2434‧‧‧1×1 post-processing operation program

FIG. 1 shows a block diagram of a convolutional neural network processor according to an embodiment of one structural aspect of the present invention; FIG. 2 shows a convolutional neural network according to an embodiment of another structural aspect of the present invention. The block diagram of the processor; FIG. 3 shows the block diagram of the 3×3 operation sub-module of the convolutional neural network processor according to the implementation of the structural aspect of FIG. 2; FIG. 4 shows the block diagram according to the third FIG. 5 is a schematic diagram of a 3×3 local convolution operation unit of a 3×3 operation sub-module of a convolutional neural network processor according to an embodiment of a structural aspect; FIG. 5 shows an implementation according to another structural aspect of the present invention. A block diagram of a convolutional neural network processor in accordance with FIG. 5; FIG. 6 shows a block diagram of a 1×1 operation sub-module of a convolutional neural network processor according to an embodiment of the structural aspect of FIG. 5; FIG. 7 Fig. 6 shows a schematic diagram of a 1 × 1 local convolution operation unit of a 1 × 1 operation sub-module of a convolutional neural network processor according to the embodiment of the structural aspect of Fig. 6; Fig. 8 shows a method according to the present invention A block diagram of the steps of the data processing method of the convolutional neural network processor according to an embodiment of the aspect; FIG. 9 illustrates the data processing of the convolutional neural network processor according to the embodiment of the method aspect of FIG. 8 Step block of the instruction decoding step of the method Fig. 10 shows a step block diagram of parallel processing steps of the data processing method of the convolutional neural network processor according to the embodiment of the method aspect of Fig. 8; Fig. 11 shows the method according to Fig. 8 A step block diagram of the operation steps of the data processing method of the convolutional neural network processor of an embodiment of the aspect; and FIG. 12 illustrates a convolutional neural network according to another embodiment of the method aspect of FIG. 8 A step block diagram of the operation steps of the data processing method of the processor.

Several embodiments of the present invention will be described below with reference to the drawings. For the sake of clarity, many practical details are set forth in the following description. It should be understood, however, that these practical details should not be used to limit the invention. That is, in some embodiments of the present invention, these practical details are unnecessary. In addition, for the purpose of simplifying the drawings, some well-known and conventional structures and elements will be shown in a simplified and schematic manner in the drawings; and repeated elements may be denoted by the same reference numerals.

FIG. 1 shows a block diagram of a convolutional neural network processor 100 in accordance with one embodiment of a structural aspect of the present invention. As can be seen from FIG. 1 , the convolutional neural network processor 100 includes a message decoding unit 110 and a convolution determination unit 120 . The convolution determination unit 120 is electrically connected to the message decoding unit 110 .

The message decoding unit 110 receives the input program 102 and the complex input weight parameter 104 . The message decoding unit 110 includes a decoding module 111 and a parallel processing module 112 . The decoding module 111 receives the input program 102 and outputs an operation command according to the input program 102 . The parallel processing module 112 is electrically connected to the decoding module 111 , and the parallel processing module 112 receives the input weight parameter 104 and the operation command. The parallel processing module 112 includes a complex parallel processing sub-module 1121 , and the parallel processing sub-module 1121 generates complex output weight parameters according to the operation command and the input weight parameters 104 . The convolution determination unit 120 includes an operation module 121 . The operation module 121 is electrically connected to the parallel processing module 112 , and the operation module 121 generates the output data 108 according to the operation of the input data 106 and the output weight parameter. Specifically, after receiving the input program 102 and the input weight parameter 104 , the message decoding unit 110 of the convolutional neural network processor 100 uses the decoding module 111 to generate an operation command to process the input weight parameter 104 . Each of the parallel processing sub-modules 1121 of the parallel processing module 112 can be electrically connected to the decoding module 111 respectively to generate output weight parameters according to the operation command. The operation module 121 may perform operations according to the input data 106 and the output weight parameters generated by the parallel processing module 112 to generate the output data 108 . Input data 106 may be data stored in a block buffer bank or data from external sources. In addition, the CNN processor 100 can use block buffers instead of input buffers and output buffers to save bandwidth of external memory. Thus, the convolutional neural network processor 100 can perform highly parallel operations through the information decoding unit 110 and the convolution determination unit 120 to provide high-performance operations.

The decoding module 111 may include a program memory 1111 and an instruction decoder 1112 . The program memory 1111 can store the input program 102 . The instruction decoder 1112 is electrically connected to the program memory 1111 . The command decoder 1112 decodes the input program 102 to output operational commands. That is to say, after receiving the input program 102, the decoding module 111 stores the input program 102 in the program memory 1111 and decodes it through the command decoder 1112 to generate an operation command, and then drives each parallel processing sub-module through the operation command 1121 processes the input weight parameters 104 to generate output weight parameters.

When the input weight parameter 104 is an uncompressed input weight parameter, the complex parallel processing sub-module 1121 includes a complex parallel sub-memory 1121a and a complex parallel sub-processor 1121b. The parallel sub-memory 1121a stores the uncompressed input weight parameters in parallel. The parallel sub-processor 1121b is electrically connected to the decoding module 111 and a parallel sub-memory 1121a, respectively. The parallel sub-processor 1121b receives the uncompressed input weight parameter in parallel according to the operation instruction to generate the output weight parameter. Specifically, each parallel processing sub-module 1121 may include a parallel sub-memory 1121a and a parallel sub-processor 1121b, respectively. After receiving the input weight parameters 104 , the parallel processing module 112 stores the input weight parameters 104 in the parallel sub-memory 1121 a of each parallel processing sub-module 1121 respectively and in parallel. Since each parallel processing sub-module 1121 is electrically connected to the decoding module 111, respectively, each parallel sub-processor 1121b can receive the uncompressed input weight parameter from the parallel sub-memory 1121a in parallel according to the operation command to generate the output weight parameter. . Thus, the parallel processing module 112 can process the input weight parameters 104 in parallel to generate the output weight parameters.

When the input weight parameter 104 is a complex compression input weight parameter, the complex parallel processing sub-module 1121 includes a complex parallel sub-memory 1121a and a complex parallel sub-processor 1121b. The parallel sub-memory 1121a stores the compression input weight parameters in parallel. The parallel sub-processor 1121b is electrically connected to the decoding module 111 and a parallel sub-memory 1121a, respectively. The parallel sub-processor 1121b receives and decompresses the compressed input weight parameters in parallel according to the operation instructions to generate output weight parameters. Specifically, each parallel processing sub-module 1121 may include a parallel sub-memory 1121a and a parallel sub-processor 1121b, respectively. After receiving the input weight parameters 104 , the parallel processing module 112 stores the input weight parameters 104 in the parallel sub-memory 1121 a of each parallel processing sub-module 1121 respectively and in parallel. Since each parallel processing sub-module 1121 is electrically connected to the decoding module 111, respectively, each parallel sub-processor 1121b can receive the compression input weight parameter from the parallel sub-memory 1121a in parallel according to the operation command, and apply the compression input weight parameters are decoded to produce output weight parameters. Thus, the parallel processing module 112 can process the input weight parameters 104 in parallel to generate the output weight parameters.

Please refer to Figure 1, Figure 2, Figure 3 and Figure 4 together. FIG. 2 shows a block diagram of a convolutional neural network processor 100 according to an embodiment of another structural aspect of the present invention. FIG. 3 shows a block diagram of the 3×3 operation sub-module 1211 of the convolutional neural network processor 100 according to the implementation of the structural aspect of FIG. 2 . FIG. 4 is a schematic diagram of the 3×3 local convolution operation unit 1211b of the 3×3 operation sub-module 1211 of the convolutional neural network processor 100 according to the implementation of the structural aspect of FIG. 3 . In the embodiments shown in FIGS. 2 to 4, the input weight parameters 104 may include a plurality of first input weight parameters and bias input weights parameter. The output weight parameters include a complex number of first output weight parameters and a bias output weight parameter. The parallel processing sub-module 1121 includes a plurality of parallel sub-memory 1121a and a plurality of parallel sub-processors 1121b. The parallel sub-memory 1121a stores the input weight parameters 104 in parallel, and includes a plurality of first parallel sub-memory 1121aa and bias parallel sub-memory 1121ab. The first parallel sub-memory 1121aa respectively and in parallel receives and stores the first input weight parameters. The bias parallel sub-memory 1121ab stores the bias input weight parameters in parallel. The parallel sub-processors 1121b are electrically connected to the decoding module 111 and the parallel sub-memory 1121a, respectively, and include a plurality of first parallel sub-processors 1121ba and bias parallel sub-processors 1121bb. The first parallel sub-processors 1121ba are respectively electrically connected to one of the first parallel sub-memory 1121aa, and receive the first input weight parameter according to the operation command to output the first output weight parameter. The bias parallel sub-processor 1121bb is electrically connected to the bias parallel sub-memory 1121ab, and receives the bias input weight parameter according to the operation command to output the bias output weight parameter. In the embodiment shown in FIG. 2, the numbers of the first parallel sub-memory 1121aa and the first parallel sub-processors 1121ba are both 9. However, in other embodiments, the number of the first parallel sub-memory 1121aa and the first parallel sub-processors The quantity of 1121ba can be a multiple of 9, which is not limited in this case. The numbers of the biased parallel sub-memory 1121ab and the biased parallel sub-processor 1121bb are both 1, but the invention is not limited to this. Specifically, after receiving the input weight parameter 104, the parallel processing module 112 stores the first input weight parameter in the input weight parameter 104 in the first parallel sub-memory 1121aa, and stores the bias input weight parameter in the bias Flatten into parallel memory 1121ab. The first parallel sub-processor 1121ba operates according to The command reads the first input weight parameter from the first parallel sub-memory 1121aa, and performs processing to generate the first output weight parameter. The bias parallel sub-processor 1121bb reads the bias input weight parameter from the bias parallel sub-memory 1121ab according to the operation command, and processes it to generate the bias output weight parameter.

Each first output weight parameter includes a complex 3×3 weight parameter. The arithmetic module 121 may include a 3×3 arithmetic sub-module 1211 and a bias distributor 1212. The 3×3 arithmetic sub-module 1211 is electrically connected to the first parallel sub-processor 1121ba, and is connected to the input according to the first output weight parameter. Data 106 operates to generate 3x3 post-processing operation data 1062. 3x3 operation sub-module 1211 includes 3x3 convolution divider group, 3x3 local convolution operation unit 1211b, and 3x3 post-processing operation unit 1211e. Each 3×3 convolution distributor group is electrically connected to a first parallel sub-processor 1121ba, and the 3×3 convolution distributor group is used for receiving and distributing the 3×3 weight parameters of the first output weight parameter. The 3×3 local convolution operation units 1211b are respectively electrically connected to a 3×3 convolution distributor group, and include a 3×3 local register group 1211c and a 3×3 local filter operation unit 1211d. The 3×3 local register group 1211c is electrically connected to a 3×3 convolution distributor group, and the 3×3 local register group 1211c of the 3×3 local convolution operation unit 1211b receives and stores the first output weight parameter. some 3×3 weight parameters, and output complex 3×3 operation parameters according to the 3×3 weight parameters of the first output weight parameter. The 3×3 local filter operation unit 1211d is electrically connected to the 3×3 local register group 1211c, and the 3×3 local filter operation unit 1211d of the 3×3 local convolution operation unit 1211b is based on the 3×3 operation parameters and the input data 106 An operation is performed to generate complex 3x3 operation data. In detail, 3 × 3 copies The ground filter operation unit 1211d can perform a 3×3 convolution operation. When the number of the first parallel sub-processors 1121ba is 9, the spatial filter positions of the 3×3 local filter operation units 1211d can respectively correspond to the first parallel sub-processors 1121ba. The parallel sub-processors 1121ba; when the number of the first parallel sub-processors 1121ba is 18, the spatial filtering positions of the 3×3 local filtering operation units 1211d can correspond to the two first parallel sub-processors 1121ba, and so on. Repeat. The 3×3 post-processing operation unit 1211e is electrically connected to the 3×3 local convolution operation unit 1211b, and performs a 3×3 post-processing operation according to the 3×3 operation data to generate the 3×3 post-processing operation data 1062 . The output data 108 of the convolutional neural network processor 100 may be 3×3 post-processing operation data 1062 . The bias distributor 1212 is electrically connected to the bias parallel sub-processor 1121bb and the 3×3 operation sub-module 1211 . The bias distributor 1212 outputs the weight parameter according to the bias to generate a complex 3×3 bias weight parameter, and outputs the 3×3 bias weight parameter to the 3×3 post-processing operation unit 1211e.

In FIG. 3 , the 3×3 arithmetic sub-module 1211 includes complex 3×3 arithmetic circuits 1211 a , and the number of the 3×3 arithmetic circuits 1211 a may be 32. Each 3×3 operation circuit 1211a is composed of a complex 3×3 local convolution operation unit 1211b and a 3×3 post-processing operation unit 1211e , and the number of the 3×3 local convolution operation units 1211b may be 32. That is to say, the number of 3×3 local convolution operation units 1211b in the 3×3 operation sub-module 1211 is 1024, and the number of 3×3 post-processing operation units 1211e is 32.

Please refer to FIG. 3 and FIG. 4 together. After receiving the 3×3 weight parameter of the first output weight parameter, the 3×3 operation sub-module 1211 can distribute the 3×3 weight parameter through the 3×3 convolutional distributor group. to 3×3 local convolution operations unit 1211b. In Figure 4, the configuration of the 3×3 convolutional distributor group adopts the two-stage allocation method, and the 3×3 convolutional distributor group includes the first 3×3 convolutional distributor 1211f and the complex second 3×3 convolutional distributor 1211g . The first 3×3 convolutional distributor 1211f is electrically connected to the first parallel sub-processor 1121ba to receive and distribute the 3×3 weighting parameters of the first output weighting parameters to the second 3×3 convolutional distributor 1211g, and the second 3×3 convolutional distributor 1211g 3. After receiving the 3×3 weight parameters, the convolution distributor 1211g distributes the 3×3 weight parameters to the 3×3 local convolution operation unit 1211b. Although the present invention uses a two-stage distribution method, the distribution method is not This is limited. The 3×3 local register bank 1211c may include two sub-3×3 local register banks 1211ca and 1211cb. The two sub-3×3 local register groups 1211ca and 1211cb combined with a multiplexer can alternately store 3×3 weight parameters or output 3×3 operation parameters to the 3×3 local filtering operation unit 1211d. That is to say, when the sub 3×3 local register bank 1211ca is used to store the 3×3 weight parameters, the sub 3×3 local register bank 1211cb outputs the 3×3 operation parameters to the 3×3 local filter operation unit 1211d ; When the sub 3×3 local register group 1211cb is used to store the 3×3 weight parameters, the sub 3×3 local register group 1211ca outputs the 3×3 operation parameters to the 3×3 local filter operation unit 1211d, that is, this case The 3×3 local register group 1211c performs storage of 3×3 weight parameters and output of 3×3 operation parameters in a ping-pong manner.

The 3×3 local filtering operation unit 1211d can perform a 3×3 convolution operation according to the 3×3 operation parameters and the input data 106 to generate 3×3 operation data. For example, the block size of the input data 106 may be 6×4, and the 3×3 local filtering operation unit 1211d may perform a 3×3 convolution operation with the input data 106 according to the 3×3 operation parameters. In order to achieve highly parallel operations, convolutional neural networks The path processor 100 can deploy a plurality of multipliers in the 3×3 operation sub-module 1211, and the number of multipliers in the 3×3 local filtering operation unit 1211d can be 73728. The 3×3 post-processing operation unit 1211e is in After receiving the 3×3 operation data generated by the 3×3 local filtering operation unit 1211d and the 3×3 bias weight parameters generated by the bias distributor, the processing can be performed according to the 3×3 operation data and the 3×3 bias weight parameters. 3x3 postprocessing operation to generate 3x3 postprocessing operation data 1062. In the embodiments shown in FIGS. 3 and 4 , the 3×3 post-processing operation data 1062 is the output data 108 of the convolutional neural network processor 100 .

In FIG. 2 , the convolution determination unit 120 further includes a controller 122 . The controller 122 is electrically connected to the message decoding unit 110 . Specifically, the controller 122 is electrically connected to the command decoder 1112 to receive the operation command, and control the 3×3 operation sub-module 1211 and the bias voltage distributor 1212 of the operation module 121 according to the operation command.

FIG. 5 shows a block diagram of a convolutional neural network processor 100 according to an embodiment of another structural aspect of the present invention. FIG. 6 shows a block diagram of the 1×1 operation sub-module 1213 of the convolutional neural network processor 100 according to the implementation of the structural aspect of FIG. 5 . FIG. 7 is a schematic diagram of the 1×1 local convolution operation unit 1213b of the 1×1 operation sub-module 1213 of the convolutional neural network processor 100 according to the embodiment of the structural aspect of FIG. 6 . The difference between the convolutional neural network processor 100 of FIG. 5 and the convolutional neural network processor 100 of FIG. 2 is that the parallel sub-memory 1121a of the convolutional neural network processor 100 of FIG. 5 further includes at least A second parallel sub-memory 1121ac, the parallel sub-processor 1121b further includes at least one second parallel sub-processor 1121bc, and the operation module 121 further includes a 1×1 operation sub-module 1213 . In addition, the input weight parameter 104 further includes At least one second input weight parameter. The output weight parameter further includes at least one second output weight parameter. At least one second parallel sub-memory 1121ac receives and stores at least one second input weight parameter respectively and in parallel. The at least one second parallel sub-processor 1121bc is respectively electrically connected to the at least one second parallel sub-memory 1121ac, and receives at least one second input weight parameter according to the operation command to output at least one second output weight parameter. The configuration of the 3×3 operation sub-module 1211 is the same as that of the 3×3 operation sub-module 1211 in the convolutional neural network processor 100 in FIG. 2 , and details are not described here. In the embodiment shown in FIG. 5, the numbers of the first parallel sub-memory 1121aa and the first parallel sub-processors 1121ba are both 9, and the numbers of the second parallel sub-memory 1121ac and the second parallel sub-processors 1121bc are both 1. However, in other embodiments, when the number of the first parallel sub-memory 1121aa and the first parallel sub-processor 1121ba is 18, the number of the second parallel sub-memory 1121ac and the second parallel sub-processor 1121bc is both 2, By analogy, this case is not limited to this. The numbers of the biased parallel sub-memory 1121ab and the biased parallel sub-processor 1121bb are both 1, but the invention is not limited to this.

Specifically, after receiving the input weight parameters 104 , the parallel processing module 112 stores the first input weight parameters in the input weight parameters 104 in the first parallel sub-memory 1121aa, and stores the second input weight parameters in the input weight parameters 104 in the first parallel sub-memory 1121aa. The weight parameter is stored in the second parallel sub-memory 1121ac, and the bias input weight parameter is stored in the bias parallel sub-memory 1121ab. The operation modes of the first parallel sub-processor 1121ba and the bias parallel sub-processor 1121bb in FIG. 5 are the same as those of the first parallel sub-processor 1121ba and the bias parallel sub-processor 1121bb in FIG. 2, and will not be repeated here. second level The row sub-processor 1121bc reads the second input weight parameter from the second parallel sub-memory 1121ac according to the operation instruction, and performs processing to generate the second output weight parameter.

The 1×1 arithmetic sub-module 1213 is electrically connected to at least one second parallel sub-processor 1121bc and the 3×3 arithmetic sub-module 1211 , and performs processing with the 3×3 post-processing arithmetic data 1062 according to at least one second output weight parameter Operates to generate 1×1 post-processing operation data 1064. The 1×1 operation sub-module 1213 includes at least one 1×1 convolution distributor group, a complex 1×1 local convolution operation unit, and a complex 1×1 post-processing operation unit 1213e . The at least one 1×1 convolutional distributor group is electrically connected to the at least one second parallel sub-processor 1121bc for receiving and distributing the 1×1 weight parameter of the at least one second output weight parameter. The 1×1 local convolution operation unit 1213b is electrically connected with at least one 1×1 convolution distributor. Each 1×1 local convolution operation unit 1213b includes a 1×1 local register group 1213c and a 1×1 local filter operation unit 1213d. The 1×1 local register group 1213c is electrically connected to at least one 1×1 convolutional distributor group. The 1×1 local register group 1213c of the 1×1 local convolution operation unit 1213b receives and stores the 1×1 weight parameter of the at least one second output weight parameter, and according to the 1×1 weight of the at least one second output weight parameter parameter to output a 1×1 operation parameter. The 1×1 local filter operation unit 1213d is electrically connected to the 1×1 local register group 1213c. The 1×1 local filtering operation unit 1213d of the 1×1 local convolution operation unit 1213b performs operations according to the 1×1 operation parameters and the 3×3 post-processing operation data 1062 to generate complex 1×1 operation data. Specifically, the 1×1 local filtering operation unit 1213d can perform a 1×1 convolution operation. When the number of the second parallel sub-processors 1121bc is 1, the spatial filtering bits of the 1×1 local filtering operation unit 1213d can correspond to the second parallel sub-processors 1121bc; when the number of the second parallel sub-processors 1121bc is 2, the spatial filtering position of the 1×1 local filtering operation unit 1213d can correspond to the two second parallel sub-processors 1121bc, so that By analogy, this case will not be repeated. The 1×1 post-processing operation unit 1213e is electrically connected to the 1×1 local convolution operation unit 1213b, and performs a 1×1 post-processing operation according to the 1×1 operation data to generate the 1×1 post-processing operation data 1064 . The output data 108 of the convolutional neural network processor 100 is 1×1 post-processing operation data 1064 . The biased parallel sub-memory 1121ab and the biased parallel sub-processor 1121bb in FIG. 5 are the same as the biased parallel sub-memory 1121ab and the biased parallel sub-processor 1121bb in FIG. 2, and will not be described here. The configuration relationship between the bias distributor 1212 and the 3×3 operation sub-module 1211 in FIG. 5 is the same as the configuration relationship between the bias distributor 1212 and the 3×3 operation sub-module 1211 in FIG. 2 , and will not be repeated here. .

Specifically, the bias distributor 1212 in FIG. 5 is electrically connected to the bias parallel sub-processor 1121bb, the 3×3 arithmetic sub-module 1211 and the 1×1 arithmetic sub-module 1213 . The bias distributor 1212 outputs the weight parameter according to the bias voltage to generate a complex 3×3 bias weight parameter and a complex 1×1 bias weight parameter. The bias distributor 1212 outputs the 3×3 bias weight parameter to the 3×3 post-processing arithmetic unit 1211e. The bias distributor 1212 outputs the 1×1 bias weight parameter to the 1×1 post-processing arithmetic unit 1213e.

In FIG. 6 , the 1×1 operation sub-module 1213 includes a complex number of 1×1 operation circuits 1213 a , and the number of the 1×1 operation circuits 1213 a may be 32. Each 1×1 operation circuit 1213a is composed of a complex 1×1 local convolution operation unit 1213b and a 1×1 post-processing operation unit 1213e. The number of product operation units 1213b may be 32. That is to say, the number of 1×1 local convolution operation units 1213b in the 1×1 operation sub-module 1213 is 1024, and the number of 1×1 post-processing operation units 1213e is 32.

Please refer to FIG. 6 and FIG. 7 together. After receiving the 1×1 weight parameter of the second output weight parameter, the 1×1 operation sub-module 1213 can allocate the 1×1 weight parameter through the 1×1 convolutional distributor group. to the 1×1 local convolution operation unit 1213b. In Fig. 7, the configuration of the 1×1 convolution distributor group adopts a two-stage distribution method, and includes a first 1×1 convolution distributor 1213f and a complex number of second 1×1 convolution distributors 1213g. The operation mode is the same as that of 3 The ×3 convolutional allocator group is the same, and will not be repeated here. The 1×1 local register bank 1213c may include two sub-1×1 local register banks 1213ca and 1213cb. The two sub 1×1 local register groups 1213ca and 1213cb combined with a multiplexer can alternately store 1×1 weight parameters or output 1×1 operation parameters to the 1×1 local filtering operation unit 1213d . The operation mode of the 1×1 local register group 1213c is the same as that of the 3×3 local register group 1211c, and will not be described here. That is to say, both the 3×3 local register bank 1211c and the 1×1 local register bank 1213c in this case operate in a ping-pong manner. Therefore, the 1×1 local filtering operation unit 1213d can perform a 1×1 post-processing operation according to the 1×1 operation parameters and the 3×3 post-processing operation data 1062 to generate 1×1 operation data. In the embodiments shown in FIGS. 5 to 7 , the 1×1 post-processing operation data 1064 is the output data 108 of the convolutional neural network processor 100 .

In order to achieve highly parallel operations, the convolutional neural network processor 100 may deploy multiple multipliers in the 3×3 operation sub-module 1211 and the 1×1 operation sub-module 1213 , for example, 3×3 local Filter operation unit The number of multipliers in 1211d may be 73728, and the number of multipliers in 1×1 local filtering operation unit 1213d may be 8192. In addition, the controller 122 in FIG. 5 is the same as the controller 122 in FIG. 2, and details are not described here.

FIG. 8 is a block diagram showing the steps of the data processing method s200 of the convolutional neural network processor according to an embodiment of a method aspect of the present invention. In FIG. 8, the data processing method s200 of the convolutional neural network processor includes a receiving step s210, an instruction decoding step s220, a parallel processing step s230, and an operation step s240.

Please refer to FIG. 1 , in detail, the receiving step s210 drives the message decoding unit 110 to receive the input program 102 and the complex input weight parameter 104 . The message decoding unit 110 includes a decoding module 111 and a parallel processing module 112 . The instruction decoding step s220 drives the decoding module 111 to receive the input program 102 and generate an operation instruction according to the input program 102 . The parallel processing step s230 drives the parallel processing module 112 to receive the input weight parameters 104 and process the input weight parameters 104 in parallel according to the operation instruction to generate complex output weight parameters. The operation step s240 drives the operation module 121 to receive the input data 106 and the output weight parameter, and operates the input data 106 and the output weight parameter to generate the output data 108 according to the operation instruction. That is, the message decoding unit 110 of the convolutional neural network processor 100 can receive the input program 102 and the input weight parameter 104 through the receiving step s210 to execute the instruction decoding step s220 and the parallel processing step s230. Since the parallel processing module 112 is electrically connected to the decoding module 111 , the parallel processing module 112 can generate the output weight parameter according to the operation command generated by the decoding module 111 in the command decoding step s220 , that is, the parallel processing step s230 . In addition, shipping The arithmetic module 121 is electrically connected to the parallel processing module 112. Therefore, in the operation step s240, the arithmetic module 121 can perform operations according to the input data 106 and the output weight parameters after receiving the input data 106 and the output weight parameters to generate Output data 108. Thereby, the data processing method s200 of the convolutional neural network processor can drive the decoding module 111 and the parallel processing module of the message decoding unit 110 through the receiving step s210, the instruction decoding step s220, the parallel processing step s230 and the operation step s240 112, and the operation module 121 of the convolution determination unit 120 performs highly parallel operations, thereby providing high performance and low power consumption operations.

For example, in FIG. 8 , the input program 102 and the input weight parameter 104 received in the receiving step s210 of the data processing method s200 of the convolutional neural network processor may include related commands and parameters corresponding to the complex input data 106 . When the instruction decoding step s220 and the parallel processing step s230 are executed, the relevant instructions and parameters corresponding to the complex input data 106 are stored in the program memory 1111 and the parallel sub-memory 1121a. During the execution of the instruction decoding step s220 and the parallel processing step s230, the relevant instructions and parameters related to one of the input data 106 may be processed. During the execution of the operation step s240, the data processing method s200 of the convolutional neural network processor can process the related instructions and parameters of the other input data 106, that is, perform the instruction decoding step for the other input data 106. s220 and parallel processing step s230. In other words, the data processing method s200 of the convolutional neural network processor first stores all relevant instructions and parameters of the input data 106 in the program memory 1111 and the parallel subroutines In the memory 1121a, the instruction decoding step s220, the parallel processing step s230 and the operation step s240 corresponding to each input data 106 are then executed. In addition, when the operation step s240 is operating on one of the input data 106 , the instruction decoding step s220 and the parallel processing step s230 can process related commands and parameters of the other input data 106 . Therefore, the data processing method s200 of the convolutional neural network processor can perform operations on the complex input data 106 after the receiving step s210 is performed.

FIG. 9 shows a block diagram of the steps of the instruction decoding step s220 of the data processing method s200 of the convolutional neural network processor according to the embodiment of the method aspect of FIG. 8 . The decoding module 111 may include a program memory 1111 and an instruction decoder 1112 . The instruction decoding step s220 may include a program storage sub-step s221 and a program decoding sub-step s222. The program storage sub-step s221 stores the input program 102 in the driver memory 1111 . The program decoding sub-step s222 drives the instruction decoder 1112 to decode the input program 102 to generate operation instructions. That is to say, the convolutional neural network processor 100 can drive the decoding module 111 to receive the input program 102 through the program storage sub-step s221 and the program decoding sub-step s222, store the input program 102 in the program memory 1111, and then borrow the input program 102. The input program 102 stored in the program memory 1111 is decoded by the instruction decoder 1112 to generate operation instructions.

FIG. 10 shows a block diagram of the steps of the parallel processing step s230 of the data processing method s200 of the convolutional neural network processor according to the embodiment of the method aspect of FIG. 8 . The parallel processing module 112 may include a plurality of parallel sub-memory 1121a and a plurality of parallel sub-processors 1121b. Parallel processing steps s230 includes a weight parameter storage sub-step s231 and a weight parameter processing sub-step s232. The weight parameter storage sub-step s231 drives the parallel sub-memory 1121a to store the input weight parameters 104 in parallel. The weight parameter processing sub-step s232 drives the parallel sub-processor 1121b. The parallel sub-processor 1121b reads the input weight parameter 104 in parallel according to the operation instruction and performs operation processing to generate the output weight parameter. That is to say, the convolutional neural network processor 100 can drive the parallel processing module 112 to receive the input weight parameters 104 through the weight parameter storage sub-step s231 and the weight parameter processing sub-step s232, and store the input weight parameters 104 in the parallel sub-memory In the body 1121a, the parallel sub-processor 1121b further processes the input weight parameters 104 stored in the parallel sub-memory 1121a according to the operation instruction to generate the output weight parameters. When the input weight parameter 104 is an uncompressed input weight parameter, the operation process may be to store the uncompressed input weight parameter. When the input weight parameters 104 are compressed input weight parameters, the operational process may be to store and decompress the compressed input weight parameters.

FIG. 11 shows a block diagram of the steps of the operation step s240 of the data processing method s200 of the convolutional neural network processor according to the embodiment of the method aspect of FIG. 8 . Please refer to Figure 2 to Figure 4 together. The output weight parameters may include a plurality of first output weight parameters and a bias output weight parameter. The first output weight parameter includes a complex 3×3 weight parameter. The operation module 121 may include a 3×3 operation sub-module 1211 and a bias distributor 1212. The 3×3 operation sub-module 1211 includes a complex 3×3 convolution distributor group, a complex 3×3 local convolution operation unit 1211b and A complex number 3×3 post-processing operation unit 1211e. The operation step s240 may include a first operation sub-step s241 and a bias operation sub-step s242. The first operation sub-step s241 includes a 3×3 parameter assignment program s2411, a 3×3 operation parameter generation program s2412, a 3×3 convolution operation program s2413 and a 3×3 post-processing operation program s2414. The 3×3 parameter assignment program s2411 drives the The 3×3 convolution distributor group receives the 3×3 weight parameters of the first output weight parameter, and distributes the 3×3 weight parameters of the first output weight parameter to the 3×3 local convolution operation unit 1211b, wherein each 3×3 weight parameter The local convolution operation unit 1211b includes a 3×3 local register group 1211c and a 3×3 local filter operation unit 1211d. The 3×3 operation parameter generation program s2412 drives the 3×3 local register group 1211c of the 3×3 local convolution operation unit 1211b to receive the 3×3 weight parameters of the first output weight parameter, and according to the first output weight parameter 3 The ×3 weight parameter produces a complex 3×3 operation parameter. The 3×3 convolution operation program s2413 drives the 3×3 local filter operation unit 1211d of the 3×3 local convolution operation unit 1211b to perform a 3×3 convolution operation on the 3×3 operation parameters and the input data 106 to generate a complex number 3× 3 Operation data. The 3×3 post-processing operation program s2414 drives the 3×3 post-processing operation unit 1211e to perform a 3×3 post-processing operation on the 3×3 operation data to generate the 3×3 post-processing operation data 1062 . The bias operation sub-step s242 drives the bias distributor 1212 to output the weight parameter according to the bias to generate a complex 3×3 bias weight parameter. The bias distributor 1212 provides the 3×3 bias weight parameters to the 3×3 arithmetic sub-module 1211 . That is, the convolutional neural network processor 100 can generate 3×3 post-processing operation data 1062 through the first operation sub-step s241 and the bias operation sub-step s242 . In detail, the 3×3 operation sub-module 1211 can be used to execute the first operation sub-step s241 , and the 3×3 convolution allocator group of the 3×3 operation sub-module 1211 can execute the 3×3 parameter assignment program s2411 to ×3 weight parameters are assigned to different 3×3 local convolution operation units The 3×3 local register bank 1211c in the element 1211b is used for the 3×3 local register bank 1211c to execute the 3×3 operation parameter generation program s2412. The 3×3 local register bank 1211c can include two sub-3×3 Local register banks 1211ca, 1211cb. The two sub-3×3 local register groups 1211ca and 1211cb act in a ping-pong manner, and further receive the 3×3 weight parameters and output the 3×3 operation parameters to the 3×3 local filtering operation unit 1211d. The 3×3 local filtering operation unit 1211d performs a 3×3 convolution operation according to the 3×3 operation parameters and the input data 106 in the 3×3 convolution operation program s2413 to generate 3×3 operation data. In the 3×3 post-processing operation program s2414, the 3×3 post-processing operation unit 1211e executes the operation according to the 3×3 bias weight parameters and the 3×3 operation data output by the bias distributor 1212 in the bias operation sub-step s242 3x3 post-processing operation to generate 3x3 post-processing operation data 1062. In the embodiments of FIGS. 2-4 and 11 , the 3×3 post-processing operation data 1062 may be the output data 108 of the convolutional neural network processor 100 .

FIG. 12 is a block diagram illustrating the steps of the operation step s240 of the data processing method s200 of the convolutional neural network processor according to another embodiment of the method aspect of FIG. 8 . Please refer to Figure 5 to Figure 7 together. The output weight parameters may include a plurality of first output weight parameters, at least one second output weight parameter, and a bias output weight parameter. The first output weight parameter includes a complex 3×3 weight parameter. The at least one second output weight parameter includes a complex 1×1 weight parameter. The arithmetic module 121 may include a 3×3 arithmetic sub-module 1211 , a 1×1 arithmetic sub-module 1213 and a bias distributor 1212 . The 3×3 arithmetic sub-module 1211 includes a complex 3×3 convolutional distributor group, a complex 3 A ×3 local convolution operation unit 1211b and a complex 3×3 post-processing operation unit 1211e. 1×1 arithmetic submodule It includes a complex 1×1 convolution distributor group, a complex 1×1 local convolution operation unit 1213b and a complex 1×1 post-processing operation unit 1213e. The operation step s240 may include a first operation sub-step s241, a second operation sub-step s243 and a bias operation sub-step s242. The first operation sub-step s241 in FIG. 12 is the same as the first operation sub-step s241 in FIG. 11 , and details are not described here. The second operation sub-step s243 drives the 1×1 operation sub-module 1213 to receive the 3×3 post-processing operation data 1062 and at least one second output weight parameter to generate the 1×1 post-processing operation data 1064 . The second operation sub-step s243 includes a 1×1 parameter assignment program s2431, a 1×1 operation parameter generation program s2432, a 1×1 convolution operation program s2433 and a 1×1 post-processing operation program s2434. The 1×1 parameter assignment program s2431 drives the at least one 1×1 convolution distributor group to receive the 1×1 weight parameter of the at least one second output weight parameter, and distribute the 1×1 weight parameter of the at least one second output weight parameter to the 1×1 local convolution operation unit s1213b, wherein each 1×1 local convolution operation unit s1213b includes a 1×1 local register group s1213c and a 1×1 local filter operation unit s1213d. The 1×1 operation parameter generation program s2432 drives the 1×1 local register group 1213c of the 1×1 local convolution operation unit 1213b to receive the 1×1 weight parameter of the at least one second output weight parameter, and according to the at least one second output The 1×1 weight parameter of the weight parameter generates a complex 1×1 operation parameter. The 1×1 convolution operation program s2433 drives the 1×1 local filter operation unit 1213d of the 1×1 local convolution operation unit 1213b to perform a 1×1 convolution operation on the 1×1 operation parameters and the 3×3 post-processing operation data 1062 to generate complex 1×1 arithmetic data. The 1×1 post-processing operation program s2434 drives the 1×1 post-processing operation unit 1213e to perform a 1×1 post-processing operation on the 1×1 operation data to generate the 1×1 post-processing operation data 1064 . That is, the volume The product neural network processor 100 can generate 1×1 post-processing operation data 1064 through the first operation sub-step s241 , the second operation sub-step s243 and the bias operation sub-step s242 . Specifically, the 1×1 operation sub-module 1213 can be used to execute the second operation sub-step s243 , and the 1×1 convolution allocator group of the 1×1 operation sub-module 1213 can execute the 1×1 parameter assignment program s2431 to convert 1 The ×1 weight parameter is allocated to the 1×1 local register group 1213c in the different 1×1 local convolution operation units 1213b so that the 1×1 local register group 1213c can execute the 1×1 operation parameter generation program s2432. 1 The ×1 local register bank 1213c may include two sub-1×1 local register banks 1213ca and 1213cb. The two sub-1×1 local register groups 1213ca and 1213cb act in a ping-pong manner, and further receive the 1×1 weight parameter and output the 1×1 operation parameter to the 1×1 local filtering operation unit 1213d. The 1×1 local filtering operation unit 1213d performs a 1×1 convolution operation according to the 1×1 operation parameters and the 3×3 post-processing operation data 1062 in the 1×1 convolution operation program s2433 to generate 1×1 operation data. In the 1×1 post-processing operation program s2434, the 1×1 post-processing operation unit 1213e executes the operation according to the 1×1 bias weight parameters and the 1×1 operation data output by the bias distributor 1212 in the bias operation sub-step s242 1x1 postprocessing operation to generate 1x1 postprocessing operation data 1064. In the embodiments of FIGS. 5-7 and 12 , the 1×1 post-processing operation data 1064 may be the output data 108 of the convolutional neural network processor 100 .

Please refer to Figure 5 to Figure 10 and Figure 12 together. Specifically, the convolutional neural network processor 100 can execute the data processing method s200 of the convolutional neural network processor, and the convolutional neural network processor 100 includes a message decoding unit 110 and a convolution determination unit 120 . The message decoding unit 110 may The receiving step s210, the instruction decoding step s220 and the parallel processing step s230 are performed. After receiving the input program 102 in the receiving step s210, the decoding module 111 stores the input program 102 through the program memory 1111, that is, the program storage sub-step s221, and then stores the input program 102 in the program memory through the instruction decoder 1112 in the program decoding sub-step s222 The input program 102 in the body 1111 is decoded to output operation instructions to the parallel processing module 112 and the controller 122 of the convolution determination unit 120 , wherein the input program 102 may include relevant instructions corresponding to the complex input data 106 . To put it simply, in the program decoding sub-step s222, the command decoder 1112 decodes the relevant command corresponding to one of the input data 106 to output the operation command. After receiving the operation command, the controller 122 can control the computing module 121 according to the operation command. The parallel processing module 112 receives the input weight parameter 104 in the receiving step s210, and executes the parallel processing step s230. The input weight parameter 104 includes a first input weight parameter, a second input weight parameter, and a bias input weight parameter. The number of the first input weight parameter may be a multiple of 9216, the number of the second input weight parameter may be a multiple of 1024, and the bias The number of input weight parameters can be a multiple of 64. In other words, the input weight parameters 104 include relevant parameters corresponding to the complex input data 106 . In the weight parameter storage sub-step s231, the first parallel sub-memory 1121aa, the second parallel sub-memory 1121ac and the bias parallel sub-memory 1121ab store the first input weight parameter, the second input weight parameter and the bias input weight respectively The parameters, wherein the first parallel sub-memory 1121aa is 9, the second parallel sub-memory 1121ac and the bias parallel sub-memory 1121ab are 1 respectively. In addition, the number of the first parallel sub-processors 1121ba in the parallel processing module 112 is 9, the number of the second parallel sub-processors 1121bc and the bias parallel The number of sub-processors 1121bb is all one. In the weight parameter processing sub-step s232 , the numbers of the first input weight parameters and the second input weight parameters that can be processed by the first parallel sub-processor 1121ba and the second parallel sub-processor 1121bc per cycle are both four. The first parallel sub-processor 1121ba and the second parallel sub-processor 1121bc respectively need to use 256 cycles to process the first input.weight parameter and the second input weight parameter corresponding to the one of the input data 106 to output the first input weight parameter and the second input weight parameter respectively. With an output weight parameter and a second output weight parameter, the bias parallel sub-processor 1121bb uses 64 cycles to process the bias input weight parameter corresponding to one of the input data 106 to output the bias output weight parameter. Thus, the convolutional neural network processor 100 can process the input weight parameters 104 in parallel by executing the receiving step s210, the instruction decoding step s220 and the parallel processing step s230.

The operation module 121 of the convolution determination unit 120 can execute the operation step s240 , and the operation module 121 includes a 3×3 operation sub-module 1211 , a bias distributor 1212 and a 1×1 operation sub-module 1213 . The bias voltage distributor 1212 may perform the bias voltage operation sub-step s242. In the bias operation sub-step s242, the bias distributor 1212 receives the 3×3 bias weight parameter and the 1×1 bias weight parameter, and distributes the 3×3 bias weight parameter to the 3×3 operation sub-module 1211 Among them, the 3×3 post-processing arithmetic unit 1211e is used to facilitate the 3×3 post-processing arithmetic unit 1211e to execute the 3×3 post-processing arithmetic program s2414, and to assign the 1×1 bias weight parameter to the 1×1 arithmetic sub-module 1213 Among them, the 1×1 post-processing operation unit 1213e enables the 1×1 post-processing operation unit 1213e to execute the 1×1 post-processing operation program s2434.

The 3×3 operation sub-module 1211 can execute the first operation sub-step s241, and the 3×3 operation sub-module 1211 includes a complex 3×3 convolution allocator group, a complex 3×3 local convolution operation unit 1211b, a complex 3× 3. Post-processing operation unit 1211e. The 3×3 convolution assigner group is electrically connected to the first parallel sub-processor 1121ba, and in the 3×3 parameter assignment program s2411, receives and assigns the 3×3 weight parameters to the 3×3 local convolution operation unit 1211b, so as to The 3×3 local convolution operation unit 1211b executes the 3×3 operation parameter generation program s2412 and the 3×3 convolution operation program s2413. Each 3×3 local convolution operation unit 1211b includes a 3×3 local register group 1211c and a 3×3 local filter operation unit 1211d. The 3×3 local register bank 1211c can execute the 3×3 operation parameter generation program s2412. The 3×3 local register bank 1211c includes two sub-3×3 local register banks 1211ca and 1211cb, and executes the 3×3 local register bank 1211cb in a ping-pong manner. The ×3 operation parameter generating program s2412 outputs the 3×3 operation parameters to the 3×3 local filtering operation unit 1211d. In the 3×3 convolution operation program s2413, the 3×3 local filter operation unit 1211d performs a 3×3 convolution operation according to the 3×3 operation parameters and the input data 106 to generate 3×3 operation data, wherein the 3×3 convolution The spatial filtering positions of the operations may respectively correspond to one of the first parallel sub-processors 1121ba. In the 3×3 post-processing operation program s2414, the 3×3 post-processing operation unit 1211e performs the 3×3 post-processing operation according to the 3×3 operation data and the 3×3 bias weight parameter to output the 3×3 post-processing operation data 1062 .

The 1×1 operation sub-module 1213 can execute the second operation sub-step s243, and the 1×1 operation sub-module 1213 includes at least one 1×1 convolution allocator group, a complex 1×1 local convolution operation unit 1213b, a complex 1 ×1 post-processing arithmetic unit 1213e. 1×1 convolutional divider group with at least one second parallel sub-place The processor 1121bc is electrically connected, and in the 1×1 parameter assignment program s2431, the 1×1 weight parameters are received and assigned to the 1×1 local convolution operation unit 1213b, so that the 1×1 local convolution operation unit 1213b can perform 1 The ×1 operation parameter generation program s2432 and the 1×1 convolution operation program s2433. Each 1×1 local convolution operation unit 1213b includes a 1×1 local register group 1213c and a 1×1 local filter operation unit 1213d. The 1×1 local register bank 1213c can execute the 1×1 operation parameter generation program s2432. The 1×1 local register bank 1213c includes two sub-1×1 local register banks 1213ca and 1213cb, and executes 1 in a ping-pong manner. The ×1 operation parameter generating program s2432 outputs the 1×1 operation parameter to the 1×1 local filtering operation unit 1213d. In the 1×1 convolution operation program s2433, the 1×1 local filter operation unit 1213d performs the 1×1 operation according to the 1×1 operation parameters and the 3×3 post-processing operation data 1062 generated in the 3×3 post-processing operation program s2414. 1 convolution operation to generate 1×1 operation data, wherein the spatial filtering positions of the 1×1 convolution operation can respectively correspond to at least one second parallel sub-processor 1121bc. In the 1×1 post-processing operation program s2434, the 1×1 post-processing operation unit 1213e performs a 1×1 post-processing operation according to the 1×1 operation data and the 1×1 bias weight parameter to output the 1×1 post-processing operation data 1064 . The 1×1 post-processing operation data 1064 output in the 1×1 post-processing operation program s2434 is the output data 108 generated by the convolutional neural network processor 100 executing the data processing method s200 of the convolutional neural network processor .

To sum up, the convolutional neural network processor 100 can perform highly parallel operations by executing the data processing method s200 of the convolutional neural network processor to provide high performance and low power consumption operations.

Although the present invention has been disclosed in the above embodiments, it is not intended to limit the present invention. Anyone skilled in the art can make various changes and modifications without departing from the spirit and scope of the present invention. Therefore, the protection of the present invention The scope shall be determined by the scope of the appended patent application.

100‧‧‧Convolutional Neural Network Processor

102‧‧‧Input Program

104‧‧‧Input Weight Parameters

106‧‧‧Enter data

112‧‧‧Parallel processing module

1121‧‧‧Parallel Processing Submodule

1121a‧‧‧Parallel sub-memory

1121b‧‧‧Parallel Subprocessor

108‧‧‧Output data

110‧‧‧Message Decoding Unit

111‧‧‧Decoding Module

1111‧‧‧Program memory

1112‧‧‧Instruction Decoder

120‧‧‧Convolution judgment unit

121‧‧‧Computing Modules

Claims (24)

A convolutional neural network processor for computing an input data, the convolutional neural network processor comprising: a message decoding unit for receiving an input program and a complex input weight parameter, and comprising: a decoding module , receiving the input program, and outputting an operation command according to the input program; and a parallel processing module, electrically connected with the decoding module, and receiving the input weight parameters, and the parallel processing module includes complex parallel processing sub-modules, the parallel processing sub-modules generate complex output weight parameters according to the operation command and the input weight parameters, wherein the parallel processing sub-modules include: a plurality of parallel sub-memory; and a plurality of parallel sub-processors, respectively electrically connected to the decoding module and a parallel sub-memory; and a convolution determination unit electrically connected to the message decoding unit, and comprising: an operation module electrically connected to the parallel processing module , the operation module generates an output data according to the operation of the input data and the output weight parameters, When the input weight parameters are complex uncompressed input weight parameters, the parallel sub-memory stores the uncompressed input weight parameters in parallel, and the parallel sub-processors receive the uncompressed input in parallel according to the operation command Weight parameters, which generate these output weight parameters. The convolutional neural network processor as described in claim 1, wherein the decoding module comprises: a program memory, which stores the input program; and an instruction decoder, which is electrically connected to the program memory, The command decoder decodes the input program to output an operation command. The convolutional neural network processor described in claim 1, wherein when the input weight parameters are complex compressed input weight parameters, the parallel sub-memory stores the compressed input weight parameters in parallel, the The parallel sub-processors receive and decompress the compressed input weight parameters in parallel according to the operation command to generate the output weight parameters. The convolutional neural network processor as described in claim 1, wherein the input weight parameters comprise a plurality of first input weight parameters The output weight parameters include a plurality of first output weight parameters; the parallel sub-memory stores the input weight parameters in parallel, and the parallel sub-memory includes: a plurality of first parallel sub-memory, respectively and in parallel receiving and storing a first input weight parameter; and the parallel sub-processors include: a plurality of first parallel sub-processors, respectively electrically connected with a first parallel sub-memory, and receiving a first parallel sub-processor according to the operation command an input weight parameter to output the first output weight parameter. The convolutional neural network processor as described in claim 4, wherein each of the first output weight parameters includes a complex number of 3×3 weight parameters; and the operation module includes: a 3×3 operation sub-module , which is electrically connected to the first parallel sub-processors, and performs operations on the input data according to the first output weight parameters to generate a 3×3 post-processing operation Data, the 3×3 operation sub-module includes: a plurality of 3×3 convolution divider groups, each of the 3×3 convolution divider groups is electrically connected with a first parallel sub-processor, the 3×3 convolution dividers The distributor group is used for receiving and distributing the 3×3 weight parameters of the first output weight parameters; the complex 3×3 local convolution operation units are respectively electrically connected with one of the 3×3 convolution distributor groups, each of the The 3×3 local convolution operation unit includes: a 3×3 local register group, the 3×3 local register group is electrically connected with the 3×3 convolution allocator group, the 3×3 local volume The 3×3 local register groups of the product operation unit receive and store the 3×3 weight parameters of the first output weight parameters, and according to the 3×3 weight parameters of the first output weight parameters , to output complex 3×3 operation parameters; and a 3×3 local filtering operation unit, which is electrically connected to the 3×3 local register group, and these 3×3 local convolution operation units of these 3×3 The local filtering operation unit is based on the 3×3 operation parameters and the input input data for operation to generate complex 3×3 operation data; and a complex 3×3 post-processing operation unit, electrically connected with the 3×3 local convolution operation units, and perform a 3×3 operation according to the 3×3 operation data ×3 post-processing operation to generate the 3×3 post-processing operation data; wherein, the output data is the 3×3 post-processing operation data. The convolutional neural network processor as described in claim 5, wherein each of the 3×3 local register sets includes: two sub-3×3 local register sets, alternately storing one of the 3×3 weights parameter or output the 3×3 operation parameter to the 3×3 local filtering operation unit. The convolutional neural network processor described in claim 5, wherein the input weight parameters further comprise a bias input weight parameter; the output weight parameters further comprise a bias output weight parameter; and the Some parallel sub-memory also include: A bias parallel sub-memory stores the bias input weight parameter in parallel; the parallel sub-processors further include: a bias parallel sub-processor electrically connected to the bias parallel sub-memory, according to the operation The instruction receives the bias input weight parameter to output the bias output weight parameter. The convolutional neural network processor described in claim 7, wherein the bias output weight parameter comprises a complex bias weight parameter; and the operation module further comprises: a bias distributor, which is connected to the bias The parallel sub-processor and the 3×3 operation sub-module are electrically connected, and the bias divider outputs the weight parameter according to the bias voltage to generate a plurality of 3×3 bias voltage weight parameters, and distributes the 3×3 bias voltage The weight parameters are output to the 3×3 post-processing operation units. The convolutional neural network processor described in claim 4, wherein the input weight parameters further include at least one second input weight weight parameters; the output weight parameters further include at least one second output weight parameter; and the parallel sub-memory further includes: at least one second parallel sub-memory, respectively and in parallel receiving and storing the at least one second input weight parameters; the parallel sub-processors further include: at least one second parallel sub-processor, respectively electrically connected to the at least one second parallel sub-memory, receiving the at least one second input weight parameter according to the operation command, to output the at least one second output weight parameter. The convolutional neural network processor as described in claim 9, wherein the operation module comprises: a 3×3 operation sub-module electrically connected with the first parallel sub-processors, and according to the some first output weight parameters are operated with the input data to generate a 3×3 post-processing operation data; and a 1×1 operation sub-module with the at least one second parallel sub-processor and the 3×3 operation The sub-module is electrically connected, and performs operation on the 3×3 post-processing operation data according to the at least one second output weight parameter to generate a 1×1 post-processing operation data; wherein, the output data is the 1×1 Post-processing operation data. The convolutional neural network processor described in claim 10, wherein the at least one second output weight parameter comprises a complex 1×1 weight parameter; the 1×1 operation sub-module comprises: at least one 1×1 A convolution distributor group, electrically connected to the at least one second parallel sub-processor, for receiving and distributing the 1×1 weight parameters of the at least one second output weight parameter; a complex 1×1 local convolution operation unit , electrically connected with the at least one 1×1 convolution distributor, each of the 1×1 local convolution operation units includes: a 1×1 local register group, the 1×1 local register group and the at least one A 1×1 convolution distributor group is electrically connected, and the 1×1 local register group of the 1×1 local convolution operation units receives and stores the 1×1 weight parameters of the at least one second output weight parameter , and according to the 1×1 weight parameters of the at least one second output weight parameter, to output complex 1×1 operation parameters; and a 1×1 local filtering operation unit, which is electrically connected to the 1×1 local register bank connection, the 1×1 local filtering operation unit of the 1×1 local convolution operation units is based on the some 1×1 operation parameters are operated with the 3×3 post-processing operation data to generate complex 1×1 operation data; and a complex 1×1 post-processing operation unit is electrically connected to the 1×1 local convolution operation units , and perform a 1×1 post-processing operation according to the 1×1 operation data to generate the 1×1 post-processing operation data. The convolutional neural network processor of claim 11, wherein each of the 1×1 local register sets includes: two sub-1×1 local register sets, alternately storing one of the 1×1 weights Parameter or output the 1×1 operation parameter to the 1×1 operation unit. The convolutional neural network processor of claim 11, wherein the input weight parameters further include a bias input weight parameter; the output weight parameters further include a bias output weight parameter; and the The parallel sub-memory further includes: a biased parallel sub-memory storing the bias input weight parameter in parallel; and The parallel sub-processors further include: a bias parallel sub-processor electrically connected to the bias parallel sub-memory, receiving the bias input weight parameter according to the operation command, and outputting the bias output weight parameter. The convolutional neural network processor described in claim 13, wherein the bias output weight parameter comprises a complex bias weight parameter; the operation module further comprises: a bias distributor parallel to the bias The sub-processor, the 3×3 arithmetic sub-module and the 1×1 arithmetic sub-module are electrically connected, and the bias divider outputs the weight parameter according to the bias to generate a complex 3×3 bias weight parameter and a complex 1 ×1 bias weight parameters; wherein, the bias divider outputs the 3×3 bias weight parameters to the 3×3 post-processing operation units; wherein the bias divider outputs the 1×1 biases The compression weight parameters are output to the 1×1 post-processing operation units. A data processing method for a convolutional neural network processor, comprising: a receiving step, driving a message decoding unit to receive an input program and complex input weight parameters, wherein the message decoding unit includes a decoding module and a parallel processing module; an instruction decoding step, driving the decoding module to receive the input program , and generate an operation command according to the input program; a parallel processing step drives the parallel processing module to receive the input weight parameters, and process the input weight parameters in parallel according to the operation command to generate a complex output weight parameters; and an operation step for driving an operation module to receive an input data and the output weight parameters, and to operate the input data and the output weight parameters according to the operation instruction to generate an output data, wherein When the input weight parameters are complex uncompressed input weight parameters, the operation process is to store the uncompressed input weight parameters. The data processing method of a convolutional neural network processor as described in claim 15, wherein the decoding module includes a program memory and an instruction decoder, and the instruction decoding step includes: a program storage sub-step , which drives the program memory to store the an input program; and a program decoding sub-step, driving the instruction decoder to decode the input program to generate the operation instruction. The data processing method of a convolutional neural network processor as described in claim 15, wherein the parallel processing module comprises a plurality of parallel sub-memory and a plurality of parallel sub-processors, and the parallel processing step comprises: a weight A parameter storage sub-step, driving the parallel sub-memory to store the input weight parameters in parallel; and a weight parameter processing sub-step, driving the parallel sub-processors, the parallel sub-processors, according to the operation command, in parallel The input weight parameters are read and an operation process is performed to generate the output weight parameters. The data processing method of a convolutional neural network processor as described in claim 17, wherein when the input weight parameters are complex compressed input weight parameters, the operation process is to store and decompress the compressed input weight parameters . The data processing method for a convolutional neural network processor as described in claim 15, wherein the output weight parameters include plural first output weight parameters; the operation module includes a 3×3 operation sub-module ; and the operation step includes: a first operation sub-step, driving the 3×3 operation sub-module to receive the input data and the first output weight parameters to generate a 3×3 post-processing operation data. The data processing method for a convolutional neural network processor as described in claim 19, wherein each of the first output weight parameters comprises a complex number of 3×3 weight parameters; the 3×3 operation sub-module comprises a complex number of 3 ×3 convolution allocator group, complex 3×3 local convolution arithmetic unit and complex 3×3 post-processing arithmetic unit; and the first operation sub-step includes: a 3×3 parameter assignment procedure, driving the 3×3 volumes The integration divider group receives the 3×3 weight parameters of the first output weight parameters, and assigns the 3×3 weight parameters of the first output weight parameters 3×3 weight parameters are allocated to the 3×3 local convolution operation units, wherein each of the 3×3 local convolution operation units includes a 3×3 local register group and a 3×3 local filter operation unit; a The 3×3 operation parameter generation program drives the 3×3 local register groups of the 3×3 local convolution operation units to receive the 3×3 weight parameters of the first output weight parameters, and according to the The 3×3 weight parameters of the first output weight parameters generate complex 3×3 operation parameters; a 3×3 convolution operation program drives the 3×3 local filters of the 3×3 local convolution operation units The operation unit performs a 3×3 convolution operation on the 3×3 operation parameters and the input data to generate complex 3×3 operation data; and a 3×3 post-processing operation program to drive the 3×3 post-processing The processing operation unit performs a 3×3 post-processing operation on the 3×3 operation data to generate the 3×3 post-processing operation data, wherein the output data is the 3×3 post-processing operation data. The data processing method for a convolutional neural network processor as described in claim 19, wherein the output weight parameters further include a bias output weight parameter; The operation module further includes a bias voltage distributor; and the operation step further includes: a bias voltage operation sub-step, driving the bias voltage distributor to output weight parameters according to the bias voltages to generate complex 3×3 bias voltage weight parameters , the bias distributor provides the 3×3 bias weight parameters to the 3×3 operation sub-module. The data processing method for a convolutional neural network processor as described in claim 19, wherein the output weight parameters further include at least one second output weight parameter; the operation module includes a 1×1 operator module; and the operation step further comprises: a second operation sub-step, driving the 1×1 operation sub-module to receive the 3×3 post-processing operation data and the at least one second output weight parameter to generate a 1×1 1 Post-processing operation data. The data processing method for a convolutional neural network processor as described in claim 22, wherein the at least one second output weight parameter comprises a complex 1×1 weight parameter; The 1×1 operation sub-module includes a complex 1×1 convolution distributor group, a complex 1×1 local convolution operation unit and a complex 1×1 post-processing operation unit; and the second operation sub-step includes: a 1×1 A parameter assignment program, driving the at least one 1×1 convolutional distributor group to receive the 1×1 weight parameters of the at least one second output weight parameter, and assign the 1×1 weight parameters of the at least one second output weight parameter The weight parameters are allocated to the 1×1 local convolution operation units, wherein each of the 1×1 local convolution operation units includes a 1×1 local register group and a 1×1 local filter operation unit; a 1×1 an operation parameter generating program, which drives the 1×1 local register groups of the 1×1 local convolution operation units to receive the 1×1 weight parameters of the at least one second output weight parameter, and according to the at least one The 1×1 weight parameters of the second output weight parameters generate complex 1×1 operation parameters; a 1×1 convolution operation program drives the 1×1 local filter operation unit of the 1×1 local convolution operation units A 1×1 convolution operation is performed on the 1×1 operation parameters and the 3×3 post-processing operation data to generate complex 1×1 operation data; and a 1×1 post-processing operation program is used to drive the 1 ×1 back The logical operation unit performs a 1×1 post-processing operation on the 1×1 operation data to generate the 1×1 post-processing operation data, wherein the output data is the 1×1 post-processing operation data. The data processing method for a convolutional neural network processor as described in claim 22, wherein the output weight parameters further comprise a bias output weight parameter; the operation module further comprises a bias distributor; And the operation step further includes: a bias operation sub-step, driving the bias distributor to output the weight parameter according to the bias to generate a complex 3×3 bias weight parameter and a complex 1×1 bias weight parameter, wherein the The bias distributor provides the 3×3 bias weight parameters to the 3×3 arithmetic sub-module, and provides the 1×1 bias weight parameters to the 1×1 arithmetic sub-module.

Images