WO2018058427A1 - Neural network computation apparatus and method - Google Patents

Abstract

A neural network computation apparatus and method. The neural network computation apparatus comprises: a control unit (100), a storage unit (200), a sparse selection unit (300), and a neural network computation unit (400); the control unit (100) is used for producing micro-commands respectively corresponding to each unit, and sending the micro-commands to each corresponding unit; the sparse selection unit (300) is used for selecting neural network data corresponding to an effective weighted value for computation from the neural network data stored in the storage unit (200) on the basis of the micro-command corresponding to the sparse selection unit (300) issued by the control unit (100) and according to position information indicated by the sparse data therein; and the neural network computation unit (400) is used for executing neural network computation on the neural network data selected by the sparse selection unit (300) on the basis of the micro-command corresponding to the neural network computation unit (400) issued by the control unit (100), in order to obtain computation results. The present apparatus and method improve the ability of the neural network computation apparatus to process different types of data, increasing the speed of neural network computation whilst reducing power consumption.

Description

Neural network computing device and method Technical field

The present invention relates to the field of information technology, and in particular, to a neural network operation device and method compatible with general neural network data, sparse neural network data, and discrete neural network data.

Background technique

Artificial neural networks (ANNs), referred to as neural networks (NNs), are mathematical models of algorithms that mimic the behavioral characteristics of animal neural networks and perform distributed parallel information processing. This kind of network relies on the complexity of the system to adjust the relationship between a large number of internal nodes to achieve the purpose of processing information. At present, neural networks have made great progress in many fields such as intelligent control and machine learning. With the continuous development of deep learning technology, the current model of neural network is getting larger and larger, and the computing performance and memory bandwidth requirements are getting higher and higher. The existing neural network computing platform (CPU, GPU, traditional neural network accelerator) The user's needs have not been met.

In order to improve the computational efficiency of the neural network computing platform, sparse neural network data and discrete neural network data are developed based on the general neural network data. However, the current neural network computing platform needs to set up a separate processing module for each type of neural network data to process, resulting in tight computing resources, and associated problems such as insufficient memory bandwidth and high power consumption.

Summary of the invention

(1) Technical problems to be solved

In view of the above technical problems, the present invention provides a neural network computing device and method for improving the degree of multiplexing of neural network data processing and saving computing resources.

(2) Technical plan

According to an aspect of the invention, a neural network computing device is provided. The neural network computing device includes: a control unit, a storage unit, a sparse selection unit, and a neural network operation unit; wherein: a storage unit is configured to store neural network data; and a control unit is configured to generate a corresponding sparse selection unit and a neural network operation unit respectively And storing the microinstruction of the unit, and sending the microinstruction to the corresponding unit; the sparse selection unit is configured to store in the storage unit according to the microinstruction corresponding to the sparse selection unit delivered by the control unit according to the location information represented by the sparse data therein In the neural network data, the neural network data corresponding to the effective weight is selected to participate in the operation; and the neural network The network operation unit is configured to perform a neural network operation on the neural network data selected by the sparse selection unit according to the micro-command corresponding to the neural network operation unit delivered by the control unit, to obtain an operation result.

According to another aspect of the present invention, there is also provided a neural network data processing method using the above neural network computing device. The neural network data processing method comprises: Step D, the discrete neural network data splitting unit splits the neural network model of the discrete neural network data into N sparsely represented sub-networks, each sub-network contains only one real number, and the remaining weights The values are all 0; step E, the sparse selection unit and the neural network operation unit process each sub-network according to the sparse neural network data to obtain the operation results respectively; and in step F, the neural network operation unit calculates the operation results of the N sub-networks And, the neural network operation result of the discrete neural network data is obtained, and the neural network data processing ends.

According to another aspect of the present invention, there is also provided a neural network data processing method using the above neural network computing device. The neural network data processing method includes: Step A, the data type judging unit judges the type of the neural network data, if the neural network data is sparse neural network data, step B is performed, and if the neural network data is discrete neural network data, step D is performed; If the neural network data is general neural network data, step G is performed; step B, the sparse selection unit selects neural network data corresponding to the effective weight in the storage unit according to the position information represented by the sparse data; step C, neural network operation The unit performs neural network operations on the neural network data acquired by the sparse selection unit, and obtains the operation result of the sparse neural network data, and the neural network data processing ends; in step D, the discrete neural network data splitting unit decomposes the neural network model of the discrete neural network data. Divided into N sparse representation sub-networks, each sub-network contains only one real number, and the remaining weights are 0; step E, sparse selection unit and neural network operation unit process each sub-network according to sparse neural network data , respectively, get the operation result; And in step F, the neural network operation unit sums the operation results of the N sub-networks to obtain the neural network operation result of the discrete neural network data, and the neural network data processing ends; in step G, the neural network operation unit executes the neural network on the general neural network data. The operation is performed, and the operation result is obtained, and the neural network data processing ends.

(3) Beneficial effects

It can be seen from the above technical solutions that the neural network computing device and method of the present invention have at least one of the following beneficial effects:

(1) By multiplexing sparse selection units while efficiently supporting sparse neural networks and discrete The neural network operation represented by the data realizes reducing the amount of data required for the operation and increasing the data multiplexing in the operation process, thereby solving the problems of insufficient performance performance, insufficient memory access bandwidth, and excessive power consumption in the prior art;

(2) Through the dependency processing unit, the device can determine whether the data has an interdependence relationship. For example, the input data used in the next calculation is the output result after the end of the previous calculation, so that the data dependency module is not calculated, and the next calculation is performed. Starting the calculation without waiting for the end of the previous calculation will cause the calculation result to be incorrect. The dependency processing unit determines the data dependency relationship, so that the control device waits for the data to perform the next calculation, thereby ensuring the correctness and high efficiency of the device operation.

DRAWINGS

1 is a schematic structural diagram of a neural network computing device according to a first embodiment of the present invention;

2 is a schematic diagram of sparse neural network weight model data;

3 is a schematic diagram of splitting discrete neural network data of N=4 into two sub-networks;

4 is a schematic diagram of splitting discrete neural network data of N=2 into two sub-networks;

FIG. 5 is a schematic structural diagram of a neural network computing device according to a second embodiment of the present invention; FIG.

6 is a schematic structural diagram of a neural network computing device according to a third embodiment of the present invention;

7 is a flowchart of a neural network data processing method according to a fourth embodiment of the present invention;

8 is a flowchart of a neural network data processing method according to a fifth embodiment of the present invention;

FIG. 9 is a flowchart of a neural network data processing method according to a sixth embodiment of the present invention; FIG.

FIG. 10 is a flowchart of a neural network data processing method according to a seventh embodiment of the present invention.

detailed description

Prior to the introduction of the present invention, three types of neural network data - general neural network data, sparse neural network data, and discrete neural network data are first described.

In the present invention, general-purpose neural network data refers to general-purpose computer data, that is, data types commonly used in computers, such as 32-bit floating-point data, 16-bit floating-point data, 32-bit fixed-point data, and the like.

In the present invention, the discrete neural network data is expressed as: part of the data or all of the data is computer data represented by discrete data. Different from the data representation of 32-bit floating point and 16-bit floating point in general neural network data, the discrete neural network data refers to all the data involved in the operation is only a set of discrete real numbers. The data in the neural network includes input data and Neural network model data. It includes the following types:

(1) The input data and the neural network model data are all composed of these real numbers called all discrete data representations;

(2) The data in the neural network only the neural network model data (all neural network layers or some neural network layers) are composed of these real numbers, and the input data is represented by the general neural network data called model discrete data;

(3) Data in the neural network Only the input data consists of these real numbers, and the neural network model data is represented by the input data of the original general neural network called input discrete data.

The discrete data representation in the present invention refers to the three discrete data representations described above. For example, the input data is the original universal neural network data, which may be an RGB image data, and the neural network model data is represented by discrete data, and the weight data of a certain layer has only two values of -1/+1, which is Neural network represented by discrete neural network data.

In the present invention, the sparse neural network data is: data that is discontinuous in position, specifically including data and data location information. For example, the model data of a neural network is sparse. First, we reflect the data location information by a 01-bit string of the same length as the entire model data. Specifically, 01 reflects whether the model data at the corresponding position is valid, 0 indicates that the position data is invalid and is sparse, and 1 indicates that the position data is valid. Finally, when storing, we only store the data at the valid location to become our data information. The data information stored in this way and the position information stored in the 01-bit string together constitute our sparse neural network data, which is also called sparse data representation in the present invention.

The neural network computing device and method provided by the present invention support sparse neural network data and neural network operations of discrete neural network data by multiplexing sparse selection units.

The invention can be applied to the following (including but not limited to) scenarios: data processing, robots, computers, printers, scanners, telephones, tablets, smart terminals, mobile phones, driving recorders, navigators, sensors, cameras, cloud servers , cameras, camcorders, projectors, watches, earphones, mobile storage, wearable devices and other electronic products; aircraft, ships, vehicles and other types of transportation; televisions, air conditioners, microwave ovens, refrigerators, rice cookers, humidifiers, washing machines, Electric lights, gas stoves, range hoods and other household appliances; and including nuclear magnetic resonance instruments, B-ultrasound, electrocardiograph and other medical equipment.

In order to make the objects, technical solutions and advantages of the present invention more clear, the following combinations are specific. The invention will be further described in detail with reference to the accompanying drawings.

First, the first embodiment

In a first exemplary embodiment of the present invention, a neural network computing device is provided. Referring to FIG. 1, the neural network operation device of this embodiment includes: a control unit 100, a storage unit 200, a sparse selection unit 300, and a neural network operation unit 400. The storage unit 200 is configured to store neural network data. The control unit 100 is configured to generate microinstructions respectively corresponding to the sparse selection unit and the neural network operation unit, and send the microinstructions to the corresponding units. The sparse selection unit 300 is configured to select, according to the microinstruction corresponding to the sparse selection unit delivered by the control unit, the neural network data corresponding to the effective weight in the neural network data stored by the storage unit according to the location information represented by the sparse data therein Participate in the operation. The neural network operation unit 400 is configured to perform a neural network operation on the neural network data selected by the sparse selection unit according to the micro-instruction of the corresponding neural network operation unit delivered by the control unit, to obtain an operation result.

The respective components of the neural network computing device of the present embodiment will be described in detail below.

The storage unit 200 is configured to store three types of neural network data - general neural network data, sparse neural network data, and discrete neural network data. In an embodiment, the storage unit may be a scratch pad memory and can support different sizes. The data size of the present invention is such that the necessary calculation data is temporarily stored in the scratch pad memory (Scratchpad Memory), so that the computing device can more flexibly and efficiently support data of different sizes in the process of performing neural network operations. The memory cells can be implemented by a variety of different memory devices (SRAM, eDRAM, DRAM, memristor, 3D-DRAM or non-volatile memory, etc.).

The control unit 100 is configured to generate microinstructions respectively corresponding to the sparse selection unit and the neural network operation unit, and send the microinstructions to the corresponding units. Wherein, in the present invention, the control unit can support a plurality of different types of neural network algorithms, including but not limited to CNN/DNN/DBN/MLP/RNN/LSTM/SOM/RCNN/FastRCNN/Faster-RCNN.

The sparse selection unit 300 is configured to select a neuron participating operation corresponding to the effective weight according to the location information of the sparse neural network data. When dealing with discrete neural network data, we also process the corresponding discrete data through the sparse selection unit.

The neural network operation unit 400 is configured to acquire input data from the storage unit according to the microinstruction generated by the control unit, and execute a general neural network or a sparse neural network or a discrete data representation. Through the network operation, the operation result is obtained, and the operation result is stored in the storage unit.

Based on the above description, the following focuses on the sparse selection unit in this embodiment. Referring to FIG. 1 , the sparse selection unit may process the sparse data representation and the discrete data representation. Specifically, the sparse selection unit selects a neural network corresponding to the location according to the location information of the sparse data and the 01 bit string. The input data of one layer is sent to the neural network operation unit. In the 01 bit string, each bit corresponds to one weight data in the neural network model, and 0 indicates that the corresponding weight data is invalid and does not exist. 1 indicates that the corresponding weight data is valid and exists. The data portion of the sparse data representation stores only valid data. For example, we have the sparse neural network weight model data shown in Figure 2. The sparse selection module directly sends the effective weight portion of the sparse data representation to the neural network operation unit, and then selects the data of the input neuron corresponding to the position of one of the valid bits according to the 01 string to be sent to the neural network operation unit. In Figure 2, the sparse selection module will be associated with the weight position 1/2/3/7/9/11/15 (this number corresponds to the left-to-right position of the position information number 1 in Figure 2, which corresponds to the array number). The corresponding input neuron is sent to the neural network operation unit.

A major feature of the present invention is the reuse of the sparse selection module 300 in discrete neural network data. Specifically, for discrete neural network data, several real values are used as a few sparse neural network data to perform operations. When preparing neural network model data, the neural network model is split into N sub-networks. The subnetwork is the same size as the original discrete neural network data. Each subnetwork contains only one real number, and the remaining weights are all 0, so each subnetwork is similar to the sparse representation described above. The only difference from the above sparse data is that after the neural network operation unit is calculated, the sub-network needs an external command to control the neural network operation unit to sum the sub-network calculation results to obtain the final result.

Referring to FIG. 1 , in this embodiment, the neural network computing device further includes: a discrete neural network data splitting unit 500. The discrete neural network data splitting unit 500 is configured to:

(1) determining the number N of real values in the discrete neural network data;

(2) The neural network model of the discrete neural network model data is divided into N sub-networks, each sub-network contains only one real number, and the remaining weights are all 0;

The sparse selection unit 300 and the neural network operation unit 400 process each sub-network according to the sparse neural network data, and respectively obtain the operation result. The neural network operation unit is further configured to sum the operation results of the N sub-networks to obtain the discrete neural network data. The result of the neural network operation.

As shown in FIG. 3, the weight data of a certain layer in the neural network model data is represented by discrete data. Our discrete data consists of 4 real values (N=4), which have only four weights of -1/1/2/-2, and are divided into four sparsely represented sub-networks according to these four weights. During the specific implementation process, the four sub-networks are prepared externally, and the sparse selection module reads four sub-networks in turn, and the subsequent processing method is the same as the sparse neural network data, and the input data corresponding to the weight position information is selected and sent to the operation. unit. The only difference is that after the calculation of the arithmetic unit is completed, an external command is needed to control the neural network operation unit to sum the operation results of the four sub-networks.

When the number of quantized values N=2, two sparse representations can be combined into one (0 and 1 in the sparse representation respectively represent two different quantized values, no longer indicating valid and inactive). As shown in FIG. 4, especially when N=2, we can combine the position information 2 string of two sparsely represented sub-networks into one string of bits, and the bit string refers to a sequence of numbers 01. The 0/1 at this time does not indicate whether or not the position information is sparse, and the position information of the 1/-1 weight is respectively indicated. The sparse selection module selects the input data twice, and the input data corresponding to the position information of the weight 1 and the weight-1 respectively is input to the operation unit. Similarly, an external command control unit is required to sum the output of the two sub-networks.

Second, the second embodiment

In a second exemplary embodiment of the present invention, a neural network computing device is provided. As shown in FIG. 5, the neural network computing device of the present embodiment differs from the first embodiment in that the data type determining unit 600 is added to determine the type of the neural network data.

The neural network data type of the present invention is specified in the instruction. The control unit controls the operation mode of the sparse selection unit and the operation unit by the output result of the data type judgment unit:

(a) for sparse neural network data, the sparse selection module selects corresponding input data according to the location information and sends it to the neural network operation unit;

Specifically, when the neural network data is sparse neural network data, the sparse selection unit selects, according to the location information represented by the sparse data, a neuron corresponding to the effective weight to participate in the operation in the storage unit; The neural network operation unit performs a neural network operation on the neural network data acquired by the sparse selection unit, and obtains an operation result.

(b) for the discrete neural network data, the sparse selection module selects the corresponding input data according to the position information and sends it to the operation unit, and the operation unit sums the calculation result according to the external operation instruction;

Specifically, when the neural network data is discrete neural network data, the discrete neural network data splitting unit is operated, and the neural network model of the discrete neural network data is split into N sub-networks; The unit and the neural network operation unit work, and each sub-network is processed according to the sparse neural network data to obtain the operation result respectively; the neural network operation unit is operated, and the operation results of the N sub-networks are summed to obtain the discrete neural network. Neural network operation results of network data.

(c) For general neural network data, the sparse selection module does not work and is not selected based on location information.

Specifically, when the neural network data is general neural network data, the sparse selection unit is disabled, and the neural network operation unit performs a neural network operation on the general neural network data to obtain an operation result.

Third, the third embodiment

In a third exemplary embodiment of the present invention, a neural network computing device is provided. Compared with the second embodiment, the neural network computing device of the present embodiment differs in that a dependency processing function is added to the control unit.

Referring to FIG. 6, according to an embodiment of the present invention, the control unit 100 includes: an instruction cache module 110, configured to store a neural network instruction to be executed, where the neural network instruction includes address information of the neural network data to be processed; The module 120 is configured to acquire a neural network instruction from the instruction cache module, and the decoding module 130 is configured to decode the neural network instruction to obtain micro instructions corresponding to the storage unit, the sparse selection unit, and the neural network operation unit respectively. The microinstruction includes address information of the corresponding neural network data; the instruction queue 140 is configured to store the decoded microinstruction; the scalar register file 150 is configured to store address information of the to-be-processed neural network data; The dependency processing module 160 is configured to determine whether the microinstruction in the instruction queue and the previous microinstruction access the same data, and if so, store the microinstruction in a storage queue, and store the microinstruction after the execution of the previous microinstruction. The microinstruction in the queue is transmitted to the corresponding unit; otherwise, the microinstruction is directly transmitted to the corresponding unit.

The instruction cache module is configured to store a neural network instruction to be executed. During the execution of the instruction, it is also cached in the instruction cache module. When an instruction is executed, if the instruction is also the earliest instruction in the instruction cache module that is not committed, the instruction will be submitted once submitted. The operation of this instruction will not be able to cancel the change of the device status. In one embodiment, the instruction cache module can be a reordering cache.

In addition, the neural network computing device of this embodiment further includes: an input and output unit for storing data in the storage unit, or acquiring a neural network operation result from the storage unit. Among them, the direct storage unit is responsible for reading data from or writing data to the memory.

Fourth, the fourth embodiment

According to the neural network computing device of the third embodiment, the present invention also provides a general neural network data (general neural network refers to a neural network in which data is not represented by discrete data representation or sparse representation), and is used for performing a general operation according to an operation instruction. Neural network operations. As shown in FIG. 7, the processing method of the general neural network data in this embodiment includes:

Step S701, the instruction fetch module extracts the neural network instruction from the instruction cache module, and sends the neural network instruction to the decoding module.

Step S702, the decoding module decodes the neural network instruction, and obtains micro-instructions corresponding to the storage unit, the sparse selection unit, and the neural network operation unit respectively, and sends each micro-instruction to the instruction queue;

Step S703, obtaining a neural network operation operation code of the micro-instruction and a neural network operation operand from the scalar register file, and then the micro-instruction is sent to the dependency processing unit;

Step S704, the dependency processing unit analyzes whether the microinstruction has a dependency on the data with the microinstruction that has not been executed before, and if so, the microinstruction needs to wait in the storage queue until it is not executed before. After the microinstruction no longer has a dependency on the data, the microinstruction is sent to the neural network operation unit and the storage unit;

Step S705, the neural network operation unit extracts the required data (including input data, neural network model data, etc.) from the scratchpad memory according to the address and size of the required data.

Step S706, then completing the neural network operation corresponding to the operation instruction in the neural network operation unit, and writing the result obtained by the neural network operation back to the storage unit.

So far, the data processing method for the sparse neural network of the fourth embodiment of the present invention has been completed.

Fifth, the fifth embodiment

Based on the neural network computing device of the third embodiment, the present invention also provides a sparse neural network data processing method for performing a sparse neural network operation according to an operation instruction. As shown in FIG. 8, the processing method of the sparse neural network data in this embodiment includes:

Step S801, the instruction module extracts the neural network instruction from the instruction cache module, and the The neural network command is sent to the decoding module;

Step S802, the decoding module decodes the neural network instruction, obtains micro-instructions respectively corresponding to the storage unit, the sparse selection unit, and the neural network operation unit, and sends each micro-instruction to the instruction queue;

Step S803, obtaining a neural network operation operation code of the micro-instruction and a neural network operation operand from the scalar register file, and then the micro-instruction is sent to the dependency processing unit;

Step S804, the dependency processing unit analyzes whether the microinstruction has a dependency on the data with the microinstruction that has not been executed before, and if so, the microinstruction needs to wait in the storage queue until it is not executed before. After the microinstruction no longer has a dependency on the data, the microinstruction is sent to the neural network operation unit and the storage unit;

Step S805, the operation unit extracts the required data (including input data, neural network model data, neural network sparse representation data) from the scratchpad memory according to the address and size of the required data, and then the sparse selection module selects according to the sparse representation. Input data corresponding to effective neural network weight data;

For example, input data is represented by general data, and neural network model data is represented by sparse representation. The sparse selection module selects input data corresponding to the weight according to the 01-bit string of the neural network model data, and the length of the 01-bit string is equal to the length of the neural network model data, as shown in FIG. 2, where the bit string number is 1 The input data input device corresponding to the position weight does not input the input data corresponding to the weight. Thus, we select the input data corresponding to the sparse weight position based on the position information 01 bit string of the weight of the input sparse representation.

Step S806, the neural network operation corresponding to the operation instruction is completed in the operation unit (because we have selected the input data corresponding thereto according to the sparse weight data in S5. Therefore, the calculation process and the step S106 in FIG. 3 The same is the case: the process of adding the offset and the final excitation after the input and the weight are multiplied, and writing the result of the neural network operation back to the storage unit.

So far, the data processing method of the sparse neural network of the fifth embodiment of the present invention has been completed.

Sixth embodiment

Based on the neural network computing device of the third embodiment, the present invention also provides a discrete neural network data processing method for performing a neural network operation of discrete data representation according to an operation instruction.

As shown in FIG. 9, the data processing method for the discrete neural network in this embodiment includes:

Step S901, the fetching module extracts the neural network instruction from the instruction cache module, and the The neural network command is sent to the decoding module;

Step S902, the decoding module decodes the neural network instruction, obtains micro-instructions corresponding to the storage unit, the sparse selection unit, and the neural network operation unit, respectively, and sends each micro-instruction to the instruction queue;

Step S903, acquiring a neural network operation operation code of the micro-instruction and a neural network operation operand from the scalar register file, and then the micro-instruction is sent to the dependency processing unit;

Step S904, the dependency processing unit analyzes whether the microinstruction has a dependency on the data with the microinstruction that has not been executed before, and if so, the microinstruction needs to wait in the storage queue until it is not executed before. After the microinstruction no longer has a dependency on the data, the microinstruction is sent to the neural network operation unit and the storage unit;

Step S905, the operation unit extracts the required data (including the input data, such as the model data of the plurality of subnetworks described above) from the cache memory according to the address and size of the required data, and each subnetwork includes only one discrete representation. The weight, and the sparse representation of each sub-network, and then the sparse selection module selects the input data corresponding to the effective weight data of the sub-network according to the sparse representation of each sub-network. The storage method of the discrete data is as shown in FIG. 3 and FIG. 4, for example. The sparse selection module is similar to the above operation, and according to the position information represented by the sparsely represented 01-bit string, the corresponding input data is selected and retrieved from the scratchpad memory into the device)

Step S906, and then completing the operation of the sub-neural network corresponding to the operation instruction in the operation unit (this process is also similar to the calculation process in the above, the only difference is that, for example, the difference like the difference between FIG. 2 and FIG. 3, the sparse representation method The model data has only one sparse representation, while the discrete data representation may generate multiple sub-models from one model data. In the operation process, the operation results of all sub-models need to be accumulated and the operation of each sub-network is performed. The results are added and the final result of the operation is written back to the storage unit.

So far, the sixth embodiment of the present invention has been introduced to the sparse neural network data processing method.

Seventh embodiment

Based on the neural network computing device of the third embodiment, the present invention also provides a neural network data processing method. Referring to FIG. 10, the neural network data processing method of this embodiment includes:

Step A, the data type judging unit judges the type of the neural network data, if the neural network data is sparse neural network data, step B is performed, if the neural network data is discrete neural network data, step D is performed; if the neural network data is a general neural network Data, execution steps G;

Step B: The sparse selection unit selects neural network data corresponding to the effective weight in the storage unit according to the location information represented by the sparse data;

Step C: The neural network operation unit performs a neural network operation on the neural network data acquired by the sparse selection unit, and obtains an operation result of the sparse neural network data, and the neural network data processing ends;

Step D, the discrete neural network data splitting unit splits the neural network model of the discrete neural network data into N sparsely represented sub-networks, each sub-network contains only one real number, and the remaining weights are all 0;

Step E, the sparse selection unit and the neural network operation unit process each sub-network according to the sparse neural network data, and respectively obtain the operation result;

Step F: the neural network operation unit sums the operation results of the N sub-networks to obtain a neural network operation result of the discrete neural network data, and the neural network data processing ends;

In step G, the neural network operation unit performs a neural network operation on the general neural network data, and obtains the operation result, and the neural network data processing ends.

So far, the seventh embodiment of the present invention has been introduced to the sparse neural network data processing method.

It should be noted that the implementations that are not shown or described in the drawings or the text of the specification are all known to those of ordinary skill in the art and are not described in detail. In addition, the above definitions of the various elements and methods are not limited to the specific structures, shapes or manners mentioned in the embodiments, and those skilled in the art can simply modify or replace them, for example: discrete data, sparse data The general data of the common neural network data is mixed. The input data is general neural network data. In the neural network model data, some layers of data use discrete data, and some layers of data use sparse data. Since the basic flow in the apparatus of the present invention is exemplified by a layer of neural network operation, the neural network actually used is often multi-layered, so it is common for each layer to adopt different data types in real use.

It should also be noted that the order of the above steps is not limited to the above, and may be varied or rearranged depending on the desired design, unless specifically described or necessarily occurring in sequence. The above embodiments may be used in combination with other embodiments or based on design and reliability considerations, that is, the technical features in different embodiments may be freely combined to form more embodiments.

In summary, the present invention solves the problem of reducing the amount of data required for the operation and increasing the data multiplexing in the operation process by multiplexing the sparse selection unit and efficiently supporting the sparse neural network and the neural network operation of the discrete data representation. In the prior art, there are problems such as insufficient computing performance, insufficient memory access bandwidth, and excessive power consumption. At the same time, through the dependency processing module, the effect of ensuring the correct operation of the neural network and improving the running efficiency and shortening the running time is achieved. It has wide application in many fields, and has strong application prospect and great economic value.

The specific embodiments of the present invention have been described in detail, and are not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and scope of the present invention are intended to be included within the scope of the present invention.

Claims (10)

A neural network computing device, comprising: a control unit, a storage unit, a sparse selection unit, and a neural network operation unit; wherein: a storage unit for storing neural network data; The control unit is configured to generate microinstructions respectively corresponding to the sparse selection unit and the neural network operation unit and the storage unit, and send the microinstruction to the corresponding unit; a sparse selection unit, configured to select, according to the microinstruction corresponding to the sparse selection unit delivered by the control unit, the neural network data corresponding to the effective weight in the neural network data stored in the storage unit according to the position information represented by the sparse data therein Participate in the operation; The neural network operation unit is configured to perform a neural network operation on the neural network data selected by the sparse selection unit according to the micro-instruction of the corresponding neural network operation unit delivered by the control unit, to obtain an operation result. The neural network computing device according to claim 1, wherein the neural network data is discrete neural network data; The neural network computing device further includes: a discrete neural network data splitting unit, configured to determine the number N of real values in the discrete neural network data, and split the neural network model of the discrete neural network data into N sparse representations. Network, each subnet contains only one real number, and the remaining weights are 0; The sparse selection unit and the neural network operation unit process each sub-network according to the sparse neural network data, and respectively obtain the operation result; The neural network operation unit is further configured to sum the operation results of the N sub-networks to obtain a neural network operation result of the discrete neural network data. The neural network computing device according to claim 2, wherein said N = 2 or 4. The neural network computing device according to claim 2, further comprising: a data type determining unit, configured to determine a type of the neural network data; The control unit is used to: (a) when the neural network data is sparse neural network data, causing the sparse selection unit to select neural network data corresponding to the effective weight in the storage unit according to the location information represented by the sparse data; Neural network acquired by the arithmetic unit for sparse selection unit The data performs a neural network operation to obtain an operation result; (b) when the neural network data is discrete neural network data, causing the discrete neural network data splitting unit to work, splitting the neural network model of the discrete neural network data into N sub-networks; and causing the sparse selection unit Working with the neural network computing unit, processing each sub-network according to the sparse neural network data, respectively obtaining the operation result; causing the neural network operation unit to work, summing the operation results of the N sub-networks, and obtaining the discrete neural network The result of the neural network operation of the data. The neural network computing device according to claim 4, wherein the control unit is further configured to: (c) when the neural network data is general neural network data, causing the sparse selection unit not to operate, and causing the neural network operation unit to perform a neural network operation on the general neural network data to obtain an operation result. The neural network computing device according to claim 1, wherein the control unit comprises: An instruction cache module, configured to store a neural network instruction to be executed, where the neural network instruction includes address information of the neural network data to be processed; An instruction fetch module, configured to acquire a neural network instruction from the instruction cache module; a decoding module, configured to decode the neural network instruction, to obtain micro-instructions corresponding to the storage unit, the sparse selection unit, and the neural network operation unit, where the micro-instruction includes address information of the corresponding neural network data; An instruction queue for storing the decoded microinstructions; a scalar register file for storing address information of the to-be-processed neural network data; The dependency processing module is configured to determine whether the microinstruction in the instruction queue and the previous microinstruction access the same data, and if so, store the microinstruction in a storage queue, and store the queue after the execution of the previous microinstruction The microinstruction in is transmitted to the corresponding unit; otherwise, the microinstruction is directly transmitted to the corresponding unit. The neural network computing device according to any one of claims 1 to 6, which is applied to the following scenarios: an electronic product, a vehicle, a home appliance or a medical device, wherein: The electronic product is one of the following groups: data processing, robot, computer, printing Machine, scanner, telephone, tablet, smart terminal, mobile phone, driving recorder, navigator, sensor, camera, cloud server, camera, camera, projector, watch, earphone, mobile storage, wearable device; The vehicle is one of the following groups: an airplane, a ship, a vehicle; The household appliance is one of the following groups: a television, an air conditioner, a microwave oven, a refrigerator, a rice cooker, a humidifier, a washing machine, an electric lamp, a gas stove, a range hood; The medical device is one of the following groups: a nuclear magnetic resonance instrument, a B-mode ultrasound, and an electrocardiograph. A neural network data processing method using the neural network computing device of claim 2, comprising: Step D, the discrete neural network data splitting unit splits the neural network model of the discrete neural network data into N sparsely represented sub-networks, each sub-network contains only one real number, and the remaining weights are all 0; Step E, the sparse selection unit and the neural network operation unit process each sub-network according to the sparse neural network data, and respectively obtain the operation result; In step F, the neural network operation unit sums the operation results of the N sub-networks to obtain a neural network operation result of the discrete neural network data, and the neural network data processing ends. A neural network data processing method using the neural network computing device of claim 5, comprising: Step A, the data type determining unit determines the type of the neural network data, if the neural network data is sparse neural network data, performing step B, if the neural network data is discrete neural network data, performing step D; if the neural network data is universal Neural network data, performing step G; Step B, the sparse selection unit selects neural network data corresponding to the effective weight in the storage unit according to the location information represented by the sparse data; Step C: The neural network operation unit performs a neural network operation on the neural network data acquired by the sparse selection unit, and obtains an operation result of the sparse neural network data, and the neural network data processing ends; Step D, the discrete neural network data splitting unit splits the neural network model of the discrete neural network data into N sparsely represented sub-networks, each sub-network contains only one real number, and the remaining weights are all 0; Step E, the sparse selection unit and the neural network operation unit process each sub-network according to the sparse neural network data, and respectively obtain the operation result; Step F, the neural network operation unit sums the operation results of the N sub-networks to obtain a neural network operation result of the discrete neural network data, and the neural network data processing ends; In step G, the neural network operation unit performs a neural network operation on the general neural network data to obtain an operation result, and the neural network data processing ends. The neural network processing method according to claim 8 or 9, wherein the control unit comprises: an instruction cache module, an instruction fetch module, a decoding module, an instruction queue, a scalar register file, and a dependency processing module; Before the step A, the method further includes: The instruction fetch module extracts the neural network instruction from the instruction cache module, and sends the neural network instruction to the decoding module; Decoding module decodes the neural network instruction, and obtains micro-instructions corresponding to the storage unit, the sparse selection unit, and the neural network operation unit, respectively, and sends each micro-instruction to the instruction queue; The microinstruction then acquires the neural network operation opcode of the microinstruction and the neural network operation operand from the scalar register file, and the subsequent microinstruction is sent to the dependency processing unit; The dependency processing unit analyzes whether the microinstruction has a dependency on the data with the microinstruction that has not been executed before, and if so, the microinstruction needs to wait in the storage queue until it is not executed with the microinstruction that has not been executed before. The microinstruction is sent to the neural network operation unit, the sparse selection unit, and the storage unit after the dependency is no longer present in the data.

Images