WO2020199476A1 - Neural network acceleration method and apparatus based on pulsation array, and computer device and storage medium - Google Patents

Abstract

A neural network acceleration method and apparatus based on a pulsation array, and a computer device and a storage medium. The method comprises: acquiring a convolution parameter of a convolution filter; segmenting a plurality of sub-filters from the convolution filter according to a preset filter segmentation rule; segmenting, according to a preset feature map segmentation rule, a plurality of feature sub-maps from a feature map to be convoluted; on the basis of the pulsation array, performing convolution calculation on the corresponding feature sub-maps according to the sub-filters; and superposing convolution calculation results corresponding to the sub-filters.

Description

Neural network acceleration method, device, computer equipment and storage medium based on systolic array

This application claims the priority of a Chinese patent application filed with the Chinese Patent Office on April 4, 2019, the application number is 201910268881.8, and the invention title is "Neural network acceleration method, device, computer equipment and storage medium based on systolic array". The entire content is incorporated into this application by reference.

Technical field

This application relates to the technical field of neural networks, and in particular to a neural network acceleration method, device, computer equipment, and storage medium based on a systolic array.

Background technique

The most important part of commonly used neural networks is the calculation of convolution. Convolution calculations often encounter situations where the convolution filter is not equal to 1. In this case, some mainstream neural network computing libraries, such as CUDNN (NVIDIA The deep network calculation library) will be significantly slower when calculating this convolution. Some deep learning accelerators, such as Field-Programmable Gate Array (FPGA), network process units (NPU), etc., are usually implemented with a systolic array structure in the convolution part. The situation where the convolution filter of the product filter is not equal to 1 is very unfriendly.

The prior art generally calculates the convolution result when the convolution step length is 1, and then down-samples and discards the unnecessary convolution result to obtain the feature map of a specific convolution step length. This obviously wastes calculation and scheduling resources. It will also slow down the convolution calculation.

Summary of the invention

The embodiments of the application provide a systolic array-based neural network acceleration method, device, computer equipment, and storage medium, which can better solve the problem of systolic array calculation and scheduling resources being wasted in convolution calculation with a step size of not one.

In the first aspect, this application provides a neural network acceleration method based on a systolic array, the method including:

Acquiring a convolution parameter of a convolution filter, where the convolution parameter includes a convolution step size and the size of the convolution filter;

If the convolution step length is not 1 and the size of the convolution filter is greater than 1×1, a number of sub-filters are segmented from the convolution filter according to a preset filter segmentation rule, and each sub-filter is The size of the filter is smaller than the size of the convolution filter;

Acquiring a feature map to be convolved, and segmenting a number of feature submaps from the feature map to be convolved according to a preset feature map segmentation rule, the plurality of feature submaps corresponding to the plurality of subfilters one-to-one;

Based on the systolic array, perform convolution calculation on the corresponding feature submap according to each of the subfilters, and the step size of the convolution calculation is 1;

The convolution calculation results corresponding to each of the subfilters are superimposed, and the superimposed result is output as the convolution calculation result of the feature map to be convolved by the convolution filter.

In the second aspect, the present application provides a neural network acceleration device based on a systolic array, the device including:

A convolution parameter acquisition module, configured to acquire convolution parameters of a convolution filter, where the convolution parameters include a convolution step size and a size of the convolution filter;

The filter segmentation module is configured to: if the convolution step size is not 1 and the size of the convolution filter is greater than 1×1, segment the convolution filter according to a preset filter segmentation rule. Filter, the size of each of the sub-filters is smaller than the size of the convolution filter;

The feature map segmentation module is used to obtain the feature map to be convolved and segment the feature map to be convolved according to a preset feature map segmentation rule into several feature sub-images, the several feature sub-images and the several sub-filters One-to-one correspondence

The convolution module is configured to perform convolution calculation on the corresponding feature sub-maps according to each of the sub-filters based on the systolic array, and the step size of the convolution calculation is 1;

The superposition module is configured to superimpose the convolution calculation results corresponding to each of the sub-filters, and output the superposition result as the convolution calculation result of the feature map to be convolved by the convolution filter.

In a third aspect, the present application provides a computer device that includes a memory and a processor; the memory is used to store a computer program; the processor is used to execute the computer program and when the computer is executed The program realizes the above-mentioned neural network acceleration method based on systolic array.

In a fourth aspect, the present application provides a computer-readable storage medium that stores a computer program. If the computer program is executed by a processor, the above-mentioned systolic array-based neural network acceleration method is implemented.

This application discloses a neural network acceleration method, device, equipment and storage medium based on a systolic array. When the convolution step length is not 1, a number of sub-filters are segmented from the convolution filter according to a preset filter segmentation rule. According to the preset feature map segmentation rules, several feature submaps can be segmented from the feature map to be convolved. The convolution calculation can be performed with the convolution step length of 1, and the convolution calculation results corresponding to each subfilter are superimposed. The superposition result of is the same as the convolution calculation result of the convolution step size of the convolution feature map to be performed according to the original convolution filter is not 1, that is, the two convolution calculations before and after the segmentation operation are equivalent; but due to the segmentation operation The post-convolution step size is 1, which can make fuller use of the computing power of the systolic array.

Description of the drawings

In order to explain the technical solutions of the embodiments of the present application more clearly, the following will briefly introduce the accompanying drawings used in the description of the embodiments. Obviously, the drawings in the following description are some embodiments of the present application. For those of ordinary skill in the art, without creative work, other drawings can be obtained from these drawings.

1 is a schematic flowchart of a neural network acceleration method based on a systolic array according to an embodiment of the application;

2 is a schematic flowchart of a neural network acceleration method based on a systolic array according to another embodiment of the application;

Fig. 3 is a schematic diagram of a partitioned convolution filter when the convolution step length is 2 and the convolution filter size is 2×2;

4 is a schematic diagram of segmenting the feature map to be convolved when the convolution step size is 2 and the convolution filter size is 2×2;

FIG. 5 is a schematic diagram of a sub-process of an embodiment of dividing the feature map to be convolved in FIG. 1;

6 is a schematic diagram of a feature map to be convolved after segmentation and zero padding;

Figure 7 is a schematic diagram of the structure of a systolic array;

Fig. 8 is a schematic diagram of convolution calculation performed by a systolic array;

FIG. 9 is a schematic diagram of a sub-process of performing convolution calculation based on a systolic array in FIG. 1;

10 is a schematic flowchart of a neural network acceleration method based on a systolic array according to another embodiment of the application;

FIG. 11 is a schematic diagram of a divided convolution filter when the convolution step size is 2 and the convolution filter size is 3×3;

FIG. 12 is a schematic diagram of a sub-process of a partitioned convolution filter when the convolution step size is 2 and the convolution filter size is 3×3;

FIG. 13 is a schematic diagram of segmenting the feature map to be convolved when the convolution step length is 2 and the convolution filter size is 3×3;

14 is a schematic flowchart of a neural network acceleration method based on a systolic array according to another embodiment of the application;

Fig. 15 is a schematic diagram of a partitioned convolution filter when the convolution step length is 3 and the convolution filter size is 3×3;

16 is a schematic diagram of a sub-process of segmenting the feature map to be convolved when the convolution step is 3 and the size of the convolution filter is 3×3;

FIG. 17 is a schematic diagram of segmenting the feature map to be convolved when the convolution step length is 3 and the convolution filter size is 3×3;

FIG. 18 is a schematic diagram of performing equivalent transformation of the topological structure of a deep convolutional neural network downsampling according to a neural network acceleration method;

19 is a schematic structural diagram of a neural network acceleration device based on a systolic array according to an embodiment of the application;

20 is a schematic structural diagram of a neural network acceleration device based on a systolic array according to another embodiment of the application;

FIG. 21 is a schematic structural diagram of a computer device provided by an embodiment of this application.

detailed description

The technical solutions in the embodiments of the present application will be described clearly and completely in conjunction with the accompanying drawings in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, rather than all of them. Based on the embodiments in this application, all other embodiments obtained by those of ordinary skill in the art without creative work shall fall within the protection scope of this application.

The flowchart shown in the drawings is merely an illustration, and does not necessarily include all contents and operations/steps, nor does it have to be executed in the described order. For example, some operations/steps can also be decomposed, combined or partially combined, so the actual execution order may be changed according to actual conditions. In addition, although the functional modules are divided in the device schematic diagram, in some cases, they may be divided into different modules from the device schematic diagram.

The embodiments of the present application provide a neural network acceleration method, device, equipment and storage medium based on a systolic array. Among them, the systolic array-based neural network acceleration method can be applied to a terminal or a server to accelerate the training or inference of the systolic array-based neural network.

Hereinafter, some embodiments of the present application will be described in detail with reference to the accompanying drawings. In the case of no conflict, the following embodiments and features in the embodiments can be combined with each other.

Please refer to FIG. 1. FIG. 1 is a schematic flowchart of a neural network acceleration method based on a systolic array provided by an embodiment of the present application.

As shown in Figure 1, the neural network acceleration method based on systolic array includes the following steps:

Step S110: Obtain convolution parameters of the convolution filter.

Wherein, the convolution parameter includes the convolution step size and the size of the convolution filter.

Filter, also known as the kernel kernel, feature detector, sliding the filter on the input image or feature map and calculating the point product is the convolution operation, and the output matrix of the convolution operation is called the convolution feature ( Convolved Feature), activation map (Activation Map) or feature map (Feature Map).

Exemplarily, before the neurons in the neural network perform the convolution operation, the pre-stored or initialized convolution parameters of the convolution filter are first obtained.

In this embodiment, the convolution parameter includes the convolution step stride and the size of the convolution filter, that is, the height h and the width w of the convolution filter; in other embodiments, the convolution parameter also includes input The number of channels and/or the number of output channels; the number of input channels in depth is determined by the number of channels in the feature map to be convolved, and the number of output channels out depth is equal to the number of convolution filters, which can determine the output after convolution The number of channels of the feature map.

The systolic array-based neural network acceleration method can be used in scenarios where the number of input channels is equal to or greater than 1, and can also be used in scenarios where the number of output channels is equal to or greater than 1.

Step S120: If the convolution step size is not 1 and the size of the convolution filter is greater than 1×1, segment a number of sub-filters from the convolution filter according to a preset filter segmentation rule.

Wherein, the size of each sub-filter is smaller than the size of the convolution filter.

Some deep learning accelerators such as FPGA, dedicated NPU, etc. are usually implemented with a systolic array structure in the convolution part, but this structure is very unfriendly to the case where the convolution step length is not equal to 1; this embodiment will be larger than 1×1 The convolution filter is divided into several sub-filters, so that each sub-filter performs a convolution operation with a convolution step length equal to 1, so as to make full use of the performance of the systolic array structure.

In some embodiments, as shown in FIG. 2 and FIG. 3, in step S120, if the convolution step size is not 1 and the size of the convolution filter is greater than 1×1, according to a preset filter segmentation rule The convolution filter divides into several sub-filters, which specifically include:

Step S121: If the convolution step size is 2 and the size of the convolution filter is 2×2, 4 subfilters are divided from the convolution filter, and the size of each subfilter is 1. ×1.

As shown in Figure 3, the convolution parameter corresponding to a certain convolution operation is [6 6 2 2], that is, the number of input channels in depth is 6, the number of output channels out depth is equal to 6, and the size of the 6 convolution filters KernelTenseor Both are 2×2.

As shown in Figure 3, each 2×2 convolution filter is divided into 4 1×1 sub-filters. Take the four sub-filters divided by the first convolution filter as an example, where the first sub-filter includes the weight w1 of the odd-numbered rows and odd-numbered columns of the convolution filter, and the second sub-filter includes the convolution The weights w2 of the odd rows and even columns of the filter, the third subfilter includes the weights w3 of the even rows and odd columns of the convolution filter, and the fourth subfilter includes the weights of the even rows and even columns of the convolution filter. The value w4.

Specifically, assign the weight of the first row and first column of the convolution filter to the first 1×1 sub-filter, and assign the weight of the first row and second column of the convolution filter to the second 1 ×1 sub-filter, assign the weight of the second row and first column of the convolution filter to the third 1×1 sub-filter, and assign the weight of the second row and second column of the convolution filter to The fourth 1×1 sub-filter.

Step S130: Obtain a feature map to be convolved, and segment a number of feature sub-maps from the feature map to be convolved according to a preset feature map segmentation rule.

Wherein, the plurality of feature sub-images correspond to the plurality of sub-filters one to one.

Exemplarily, the number of channels of the feature map to be convolved may be equal to or greater than 1, and the number of channels of the feature map to be convolved may determine the number of input channels in the convolution parameter of the corresponding convolution filter.

In some embodiments, as shown in FIG. 2 and FIG. 4, if the convolution step size is 2 and the size of the convolution filter is 2×2, step S130 obtains the feature map to be convolved and performs the The feature map segmentation rules of to segment several feature sub-maps from the feature map to be convolved, which specifically include:

Step S1311: Assign the values of the odd rows and odd columns of the feature map to be convolved to the corresponding positions of the first feature submap.

Step S1312. Assign the values of the odd rows and even columns of the feature map to be convolved to the corresponding positions of the second feature submap.

Step S1313: Assign the values of the even rows and odd columns of the feature map to be convolved to the corresponding positions of the third feature submap.

Step S1314: Assign the values of the even rows and even columns of the feature map to be convolved to the corresponding positions of the fourth feature submap.

Exemplarily, the values in the same row in the feature map to be convolved are also located in the same row in each feature submap, and the values in the same column in the feature map to be convolved are also located in the same column in each feature submap.

As shown in Figure 4, the acquired convolution feature map input Tensor is the convolution feature map of [1 6 4 4]. The number of channels of the feature map to be convolved is 6, and the width and height are both 4. According to the preset segmentation feature, 4 feature submaps are segmented from the feature map to be convolved.

In some embodiments, as shown in FIG. 5, step S130 acquiring a feature map to be convolved and segmenting several feature sub-maps from the feature map to be convolved according to a preset feature map segmentation rule specifically includes:

Step S131: Obtain a feature map to be convolved.

Exemplarily, the acquired convolution feature map is shown in FIG. 6.

Step S132: If the length or width of the acquired feature map to be convolved is not an integer multiple of the convolution step length, perform zero padding on the preset position of the feature map to be convolved to make the zero padding to be convolved The length or width of the feature map is an integer multiple of the convolution step length.

In this embodiment, the length and width of the acquired convolution feature map are both 3. The padding is exemplarily added to the right and bottom of the convolution feature map, so that the length of the feature map to be convolved after zero padding is The width is 4.

Step S133: According to a preset feature map segmentation rule, a number of feature sub-maps are segmented from the feature map to be convolved after zero padding.

As shown in FIG. 6, according to the above steps S1311-step S1314, 4 feature submaps are segmented from the feature map to be convolved after zero padding.

In some embodiments, the structure of the feature map to be convolved is NCHW, such as [1 6 4 4], N represents the number, C represents the number of channels, H represents the height, and W represents the width; therefore, the number of instances batchsize=1, the channel When the number of channel=6, the height of the feature map to be convolved H=4, and the width of the feature map to be convolved W=4, that is, the number of tensors to be convolved is 1, there are 6 channels, and each channel is a waiting Convolution feature map. The feature maps to be convolved for different channels under the same number can be divided and convolved first, and then the feature maps to be convolved under the next number and different channels can be divided and convolved.

Step S140, based on the systolic array, perform convolution calculation on the corresponding feature submap according to each of the subfilters, and the step size of the convolution calculation is 1.

The core concept of Systolic Array is to allow data to flow in the array of arithmetic units, reduce the number of memory accesses, and make the structure more regular, the wiring more uniform, and the frequency.

In some embodiments, as shown in FIG. 7, the Systolic Array includes L×L processing units PE, and the systolic array is connected to the weight register filter buffer, the input register in buffer, and the output register out buffer. The left side of each row processing unit PE and the upper side of each column processing unit PE are provided with a first-in first-out register FIFO. The weight of the filter is stored in the first-in first-out register FIFO and transmitted to all processing units PE in the same row. The processing units PE in the first row and the first column receive the data from the feature map to be convolved in the input register, and the first The processing unit PE in the row and the first column both transmit data from the feature map to be convolved to the processing unit PE in the lower right corner of each. This design maximizes data reuse.

Exemplarily, as shown in FIG. 8, the systolic array performs two-dimensional convolution on a 5×5 feature map X according to a 3×3 filter W.

Assume that the filter W and the feature map X have the following forms:

Among them, wi and xj respectively represent a certain row of data of filter W and feature map X, and the three processing units PE in the last row output three rows of convolution results:

Where * represents one-dimensional convolution calculation.

In some embodiments, as shown in FIG. 9, step S140 is based on the systolic array, and performs convolution calculation on the corresponding feature sub-map according to each of the sub-filters, which specifically includes:

Step S141: Load the weight of the sub-filter into the weight register connected to the systolic array.

Exemplarily, the weight of the sub-filter is loaded into the weight register filter buffer, and the weight of the sub-filter is stored in the first-in first-out register FIFO and transmitted to the processing unit PE in the same row.

Step S142: Load the characteristic sub-map corresponding to the sub-filter into the input register connected to the systolic array.

Exemplarily, the feature submap corresponding to the subfilter is loaded into the input register in buffer, and the processing unit PE in the first row and the first column of the systolic array receives data from the feature submap in the input register in buffer.

Step S143: Obtain the output result of the systolic array convolution calculation.

Exemplarily, the processing units PE in the first row and the first column of the systolic array both transmit data from the feature sub-map to the processing unit PE in the lower right corner of each; the processing unit PE in the last row outputs the feature corresponding to the sub-filter pair Convolution calculation is performed on the sub-image, and the convolution step length is 1 convolution result.

As shown in Figure 3, the weight of the first sub-filter is w1, and the convolution calculation is performed on the first feature sub-image corresponding to it in Figure 4; the weight of the second sub-filter is w2, which is Convolution calculation is performed on the three feature sub-images; the weight of the third sub-filter is w3, and the convolution calculation is performed on the third feature sub-image; the weight of the fourth sub-filter is w4, and the weight of the fourth sub-filter is w4. Perform convolution calculations. The results of the convolution calculation corresponding to the first to fourth sub-filters are as follows:

Step S150: Superimpose the convolution calculation results corresponding to each of the sub-filters, and output the superimposed result as the result of the convolution calculation of the feature map to be convolved by the convolution filter.

Exemplarily, the convolution calculation results corresponding to the 4 sub-filters are superimposed to obtain:

If you directly perform convolution calculation on the convolution feature map on the left side of Fig. 4 according to the convolution filter on the left side of Fig. 3 with a convolution step size of 2, the result of the convolution calculation is:

Therefore, the neural network acceleration method based on the systolic array of this embodiment divides a number of sub-filters from the convolution filter according to a preset filter segmentation rule when the convolution step length is not 1, and according to the preset Set the feature map segmentation rule to segment a number of feature sub-maps from the feature map to be convolved. The convolution calculation can be performed with a convolution step length of 1, and the convolution calculation results corresponding to each sub-filter are superimposed. It is the same as the convolution calculation result of the convolution step length not 1 performed on the convolution feature map according to the original convolution filter, that is, the two convolution calculations before and after the segmentation operation are equivalent; therefore, the result of the superposition can be used as the The convolution filter outputs the result of the convolution calculation of the feature map to be convolved for subsequent processing such as reconvolution, pooling, classification, etc.; but because the convolution step length is 1, after the segmentation operation, The computing power of the systolic array can be fully utilized.

Exemplarily, as shown in FIG. 10 and FIG. 11, in step S120, if the convolution step size is not 1 and the size of the convolution filter is greater than 1×1, according to a preset filter segmentation rule from the The convolution filter divides into several sub-filters, including:

Step S122: If the convolution step size is 2 and the size of the convolution filter is 3×3, 4 sub-filters are divided from the convolution filter, and the size of each sub-filter is 2. ×2.

Wherein the first subfilter includes the weights of odd rows and odd columns of the convolution filter, the second subfilter includes the weights of odd rows and even columns of the convolution filter, and the third subfilter includes the weights of the odd rows and even columns of the convolution filter. The weights of the even rows and odd columns of the convolution filter, and the fourth sub-filter includes the weights of the even rows and even columns of the convolution filter.

In some embodiments, the size of the convolution filter kernel (filter) cannot divide the convolution stride, and the zero-filled convolution filter can be made by performing zero padding at the preset position of the convolution filter The length or width of is an integer multiple of the convolution step length. In this embodiment, the size of the convolution filter is 3×3, and the convolution step size is 2. The size of the convolution filter cannot divide the convolution step size. The zero padding operation can be used to make the zero pad convolution The length or width of the filter is an integer multiple of the convolution step length, so that the convolution filter can divide a number of sub-filters according to a preset filter division rule.

Specifically, as shown in FIGS. 11 and 12, if the convolution step size is 2 and the size of the convolution filter is 3×3, 4 sub-filters are divided from the convolution filter , The size of each sub-filter is 2×2, and specifically includes:

Step S11: Assign weights of odd rows and odd columns of the convolution filter to the first subfilter.

Step S12: Assign the weights of the odd rows and even columns of the convolution filter to the first column of the second sub-filter, and fill the second column of the second sub-filter with 0.

Step S13: Assign the weights of the even rows and odd columns of the convolution filter to the first row of the third sub-filter, and fill the second row of the third sub-filter with 0.

Step S14: Assign the weights of the even rows and even columns of the convolution filter to the first row and first column of the fourth subfilter, and fill the remaining positions of the fourth subfilter with 0.

In this embodiment, as shown in FIG. 10 and FIG. 13, if the convolution step size is 2 and the size of the convolution filter is 3×3, step S130 obtains the feature map to be convolved and performs The feature map segmentation rules of to segment several feature sub-maps from the feature map to be convolved, which specifically include:

Step S1321: Allocate the values of odd rows and odd columns of the feature map to be convolved to the corresponding positions of the first feature submap.

Step S1322, assign the values of odd rows and even columns of the feature map to be convolved to the corresponding positions of the second feature submap.

Step S1323: Assign the values of the even rows and odd columns of the feature map to be convolved to the corresponding positions of the third feature submap.

Step S1324: Assign the values of the even rows and even columns of the feature map to be convolved to the corresponding positions of the fourth feature submap.

When the convolution step size is 2 and the size of the convolution filter is 3×3, 4 feature submaps are segmented from the feature map to be convolved according to the preset feature map segmentation rule; if the feature to be convolved The number of channels in the picture is 1, and the number of channels after division is 4.

In some embodiments, as shown in FIG. 14 and FIG. 15, in step S120, if the convolution step size is not 1 and the size of the convolution filter is greater than 1×1, according to the preset filter division rule The convolution filter divides into several sub-filters, which specifically include:

Step S123: If the convolution step size is 3 and the size of the convolution filter is 3×3, 9 sub-filters are divided from the convolution filter, and the size of each sub-filter is 1. ×1 and each includes one of the 9 weights of the convolution filter.

Exemplarily, assign the weight of the first row and first column of the convolution filter to the first 1×1 sub-filter, and assign the weight of the first row and second column of the convolution filter to the second one 1×1 sub-filter, assign the weight of the first row and third column of the convolution filter to the third 1×1 sub-filter, and assign the weight of the second row and first column of the convolution filter For the fourth 1×1 sub-filter, assign the weight of the second row and second column of the convolution filter to the fifth 1×1 sub-filter, and so on.

In this embodiment, as shown in FIG. 16 and FIG. 17, if the convolution step size is 3 and the size of the convolution filter is 3×3, step S130 obtains the feature map to be convolved and performs The feature map segmentation rules of to segment several feature sub-maps from the feature map to be convolved, which specifically include:

Step S1331: Assign the value of the 3n+1th row and 3n+1th column of the feature map to be convolved to the corresponding position of the first feature submap. Where n is a natural number.

Step S1332, assign the value in the 3n+1th row and 3n+2th column of the feature map to be convolved to the corresponding position of the second feature submap.

Step S1333: Assign the values in the 3n+1th row and 3n+3th column of the feature map to be convolved to the corresponding position of the third feature submap.

Step S1334: Assign the value in the 3n+2th row and 3n+1th column of the feature map to be convolved to the corresponding position of the fourth feature submap.

Step S1335: Assign the values in the 3n+2th row and 3n+2th column of the feature map to be convolved to the corresponding position of the fifth feature submap.

Step S1336: Assign the value in the 3n+2th row and 3n+3th column of the feature map to be convolved to the corresponding position of the sixth feature submap.

Step S1337: Assign the values in the 3n+3th row and 3n+1th column of the feature map to be convolved to the corresponding position of the seventh feature submap.

Step S1338: Assign the value in the 3n+3th row and 3n+2th column of the feature map to be convolved to the corresponding position of the eighth feature submap.

Step S1339: Assign the value in the 3n+3th row and 3n+3th column of the feature map to be convolved to the corresponding position of the ninth feature submap.

Exemplarily, as shown in FIG. 17, the length and width of the acquired feature map to be convolved are both 8. If it is not the convolution step size, that is, an integer multiple of 3, the pre-convolution feature map is Set the position to perform zero-padded so that the length or width of the zero-padded feature map to be convolution is an integer multiple of the convolution step; then according to the preset feature map segmentation rule, the zero-padded feature map to be convolved Segment 9 feature sub-maps.

The systolic array-based neural network acceleration method of the present application divides several sub-filters from the convolution filter according to the preset filter segmentation rule when the convolution step length is not 1, and according to the preset feature map segmentation rule Segmenting several feature sub-maps from the feature map to be convolved can achieve convolution calculation with a convolution step length of 1. It can be well adapted to some special dedicated deep network accelerators such as FPGA, NPU and other low-level systolic array (Systolic Array) structure, which can save computing resources, and this segmentation method itself is a special computing logic , Can be integrated into various deep learning frameworks. The segmentation transformation method provided in this application does not affect the forward and backward paths of the deep network itself, and because it saves computing resources, it actually improves the speed of training and inference.

In some embodiments, as shown in FIG. 18 is a schematic diagram of the down-sampling topology of the traditional deep convolutional neural network ResNet50 according to the systolic array-based neural network acceleration method of the present application; the left side of the arrow is the traditional A simplified model of the down-sampled topology structure of the deep convolutional neural network ResNet50. The left side of the arrow is the calculated topology after the equivalent transformation of the segmentation transformation.

Compared with the calculation graph structure of the traditional deep convolutional neural network, the calculation topology after the equivalent transformation of the segmentation transformation has the following advantages: 1. Eliminates the two 1×1 mapping convolutions on the left side of the traditional ResNet50 , Reduce computing resources. 2. The residual component part on the right side of the traditional ResNet50 can be converted into a direct identity mapping (Identity Mapping), which is conducive to the propagation of residuals. The systolic array-based neural network acceleration method of this application can be applied to many network models For example, densNet, or shakeshake network, etc., as long as the network is stored in the down-sampling part, the neural network acceleration method provided in this application can be used for transformation, and then calculation training can be performed.

Please refer to FIG. 19, which is a schematic structural diagram of a systolic array-based neural network acceleration device provided by an embodiment of the present application. The systolic array-based neural network acceleration device can be configured in a server to execute the aforementioned systolic array The neural network acceleration method.

As shown in Figure 19, the systolic array-based neural network acceleration device includes:

The convolution parameter obtaining module 110 is configured to obtain convolution parameters of a convolution filter, where the convolution parameters include a convolution step size and a size of the convolution filter.

The filter segmentation module 120 is configured to: if the convolution step length is not 1 and the size of the convolution filter is greater than 1×1, segment the convolution filter according to a preset filter segmentation rule. Sub-filters, the size of each sub-filter is smaller than the size of the convolution filter.

The feature map segmentation module 130 is configured to obtain a feature map to be convolved and segment a number of feature sub-maps from the feature map to be convolved according to a preset feature map segmentation rule, the number of feature sub-maps and the number of sub-filters One to one correspondence.

The convolution module 140 is configured to perform convolution calculation on the corresponding feature sub-maps according to each of the sub-filters based on the systolic array, and the step size of the convolution calculation is 1.

The superposition module 150 is configured to superimpose the convolution calculation results corresponding to each of the sub-filters, and output the superimposition result as the convolution calculation result of the feature map to be convolved by the convolution filter.

In some embodiments, as shown in FIG. 20, the feature map segmentation module 130 includes:

The feature map acquiring sub-module 131 is used to acquire the feature map to be convolved.

The zero padding sub-module 132 is configured to, if the length or width of the acquired feature map to be convolved is not an integer multiple of the convolution step length, perform zero padding on the preset position of the feature map to be convolved to make zero padding The length or width of the subsequent feature map to be convolved is an integer multiple of the convolution step length.

The feature map segmentation submodule 133 is used to segment a number of feature submaps from the feature map to be convolved after zero padding according to preset feature map segmentation rules.

In some embodiments, as shown in FIG. 20, the convolution module 140 includes:

The weight loading sub-module 141 is used to load the weight of the sub-filter into the weight register connected to the systolic array;

The sub-picture loading sub-module 142 is used to load the characteristic sub-picture corresponding to the sub-filter to the input register connected to the systolic array.

The output sub-module 143 is used to obtain the output result of the systolic array convolution calculation.

In some embodiments, as shown in FIG. 20, the filter segmentation module 120 includes a first filter segmentation sub-module 121, which is used if the convolution step size is 2 and the size of the convolution filter is 2× 2. Divide 4 sub-filters from the convolution filter, each of the sub-filters has a size of 1×1; wherein the first sub-filter includes the weights of odd rows and odd columns of the convolution filter , The second sub-filter includes the weights of the odd-numbered rows and even columns of the convolution filter, the third sub-filter includes the weights of the even-numbered rows and odd columns of the convolution filter, and the fourth sub-filter includes the weights of the convolution filter. The weights of the even rows and even columns of the product filter.

The feature map segmentation module 130 includes a first feature map segmentation sub-module 1301, which is configured to, if the convolution step size is 2 and the size of the convolution filter is 2×2, divide the odd rows of the feature map to be convolution The values of the odd columns are assigned to the corresponding positions of the first feature submap, the values of the odd rows and the even columns of the feature map to be convolved are assigned to the corresponding positions of the second feature submap, and the feature map to be convolved is assigned The values of even rows and odd columns are allocated to the corresponding positions of the third feature submap, and the values of even rows and even columns of the feature map to be convolved are allocated to the corresponding positions of the fourth feature submap.

In some embodiments, as shown in FIG. 20, the filter segmentation module 120 includes a second filter segmentation sub-module 122, which is used if the convolution step size is 2 and the size of the convolution filter is 3× 3. Divide 4 sub-filters from the convolution filter, each of the sub-filters has a size of 2×2; wherein the first sub-filter includes the weights of odd rows and odd columns of the convolution filter , The second sub-filter includes the weights of the odd-numbered rows and even columns of the convolution filter, the third sub-filter includes the weights of the even-numbered rows and odd columns of the convolution filter, and the fourth sub-filter includes the weights of the convolution filter. The weights of the even rows and even columns of the product filter.

The feature map segmentation module 130 includes a second feature map segmentation sub-module 1302, which is used to divide the odd-numbered rows of the feature map to be convolved if the convolution step size is 2 and the size of the convolution filter is 3×3 The values of the odd columns are assigned to the corresponding positions of the first feature submap, the values of the odd rows and the even columns of the feature map to be convolved are assigned to the corresponding positions of the second feature submap, and the feature map to be convolved is assigned The values of even rows and odd columns are allocated to the corresponding positions of the third feature submap, and the values of even rows and even columns of the feature map to be convolved are allocated to the corresponding positions of the fourth feature submap.

In some embodiments, as shown in FIG. 20, the filter segmentation module 120 includes a third filter segmentation sub-module 123, which is used if the convolution step size is 3 and the size of the convolution filter is 3× 3. Separate 9 sub-filters from the convolution filter, each of the sub-filters has a size of 1×1 and each includes one of the 9 weights of the convolution filter.

The feature map segmentation module 130 includes a third feature map segmentation sub-module 1303, which is configured to: if the convolution step size is 3 and the size of the convolution filter is 3×3, the 3nth feature map to be convolved Assign the value in row 3n+1 and column 3n+1 to the corresponding position of the first feature submap, and assign the value in row 3n+1 and column 3n+2 of the feature map to be convolved to the second feature submap Assign the value in row 3n+1, column 3n+3 of the feature map to be convolved to the corresponding position of the third feature submap, and assign the feature map to be convolved to row 3n+2, row 3n+3. The value in column 3n+1 is assigned to the corresponding position of the fourth feature submap, and the value in row 3n+2 and column 3n+2 of the feature map to be convolved is assigned to the corresponding position of the fifth feature submap, Assign the value in row 3n+2, column 3n+3 of the feature map to be convolved to the corresponding position of the sixth feature submap, and assign the feature map to be convolved in row 3n+3, column 3n+1 The value of is assigned to the corresponding position of the seventh feature submap, the value of the 3n+3 row 3n+2 column of the feature map to be convolved is assigned to the corresponding position of the eighth feature submap, and the The value in the 3n+3 row and 3n+3 column of the convolution feature map is assigned to the corresponding position of the ninth feature sub-map, where n is a natural number.

It should be noted that those skilled in the art can clearly understand that for the convenience and conciseness of description, the specific working process of the above-described device and each module and unit can refer to the corresponding process in the foregoing method embodiment. No longer.

The method and device of this application can be used in many general or special computing system environments or configurations. For example: personal computers, server computers, handheld or portable devices, tablet devices, multi-processor systems, microprocessor-based systems, set-top boxes, programmable consumer electronic devices, network PCs, small computers, large computers, including the above Distributed computing environment of any system or device, etc.

Exemplarily, the foregoing method and apparatus may be implemented in the form of a computer program, and the computer program may run on the computer device as shown in FIG. 21.

Please refer to FIG. 21. FIG. 21 is a schematic structural diagram of a computer device according to an embodiment of the present application. The computer equipment can be a server or a terminal.

Referring to FIG. 21, the computer device includes a processor, a memory, and a network interface connected through a system bus, where the memory may include a non-volatile storage medium and an internal memory.

The non-volatile storage medium can store an operating system and a computer program. The computer program includes program instructions. When the program instructions are executed, the processor can execute any neural network acceleration method based on a systolic array.

The processor is used to provide computing and control capabilities and support the operation of the entire computer equipment.

The internal memory provides an environment for the operation of the computer program in the non-volatile storage medium. When the computer program is executed by the processor, the processor can execute any neural network acceleration method based on the systolic array.

The network interface is used for network communication, such as sending assigned tasks. Those skilled in the art can understand that the structure of the computer device is only a block diagram of a part of the structure related to the solution of the application, and does not constitute a limitation on the computer device to which the solution of the application is applied. The specific computer device may include More or fewer components are shown in the figure, or some components are combined, or have different component arrangements.

It should be understood that the processor may be a central processing unit (Central Processing Unit, CPU), the processor may also be other general-purpose processors, digital signal processors (Digital Signal Processor, DSP), and application specific integrated circuits (Application Specific Integrated Circuits). Circuit, ASIC), Field-Programmable Gate Array (FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components, etc. Among them, the general-purpose processor may be a microprocessor or the processor may also be any conventional processor.

Wherein, in one embodiment, the processor is used to run a computer program stored in a memory to implement the steps of the aforementioned method for accelerating a neural network based on a systolic array.

Exemplarily, the processor is configured to run a computer program stored in the memory to implement the following steps:

Acquiring a convolution parameter of a convolution filter, where the convolution parameter includes a convolution step size and the size of the convolution filter;

If the convolution step length is not 1 and the size of the convolution filter is greater than 1×1, a number of sub-filters are segmented from the convolution filter according to a preset filter segmentation rule, and each sub-filter is The size of the filter is smaller than the size of the convolution filter;

Acquiring a feature map to be convolved, and segmenting a number of feature submaps from the feature map to be convolved according to a preset feature map segmentation rule, the plurality of feature submaps corresponding to the plurality of subfilters one-to-one;

Based on the systolic array, perform convolution calculation on the corresponding feature submap according to each of the subfilters, and the step size of the convolution calculation is 1;

The convolution calculation results corresponding to each of the subfilters are superimposed, and the superimposed result is output as the convolution calculation result of the feature map to be convolved by the convolution filter.

From the description of the foregoing implementation manners, it can be understood that those skilled in the art can clearly understand that this application can be implemented by means of software plus a necessary general hardware platform. Based on this understanding, the technical solution of this application essentially or the part that contributes to the existing technology can be embodied in the form of a software product, and the computer software product can be stored in a storage medium, such as ROM/RAM, magnetic disk , CD-ROM, etc., including several instructions to make a computer device (which can be a personal computer, a server, or a network device, etc.) execute the methods described in each embodiment of this application or some parts of the embodiment, such as:

A computer-readable storage medium, the computer-readable storage medium stores a computer program, the computer program includes program instructions, and the processor executes the program instructions to implement any item provided in the embodiments of this application based on Neural network acceleration method of systolic array.

The computer-readable storage medium may be the internal storage unit of the computer device described in the foregoing embodiment, such as the hard disk or memory of the computer device. The computer-readable storage medium may also be an external storage device of the computer device, such as a plug-in hard disk, a smart memory card (SMC), or a secure digital (Secure Digital, SD) equipped on the computer device. ) Card, Flash Card, etc.

The above are only specific implementations of this application, but the protection scope of this application is not limited to this. Anyone familiar with the technical field can easily think of various equivalents within the technical scope disclosed in this application. Modifications or replacements, these modifications or replacements shall be covered within the protection scope of this application. Therefore, the protection scope of this application shall be subject to the protection scope of the claims.

Claims (20)

A neural network acceleration method based on systolic array, including: Acquiring a convolution parameter of a convolution filter, where the convolution parameter includes a convolution step size and the size of the convolution filter; If the convolution step length is not 1 and the size of the convolution filter is greater than 1×1, a number of sub-filters are segmented from the convolution filter according to a preset filter segmentation rule, and each sub-filter is The size of the filter is smaller than the size of the convolution filter; Acquiring a feature map to be convolved, and segmenting a number of feature submaps from the feature map to be convolved according to a preset feature map segmentation rule, the plurality of feature submaps corresponding to the plurality of subfilters one-to-one; Based on the systolic array, perform convolution calculation on the corresponding feature submap according to each of the subfilters, and the step size of the convolution calculation is 1; The convolution calculation results corresponding to each of the subfilters are superimposed, and the superimposed result is output as the convolution calculation result of the feature map to be convolved by the convolution filter. 8. The neural network acceleration method according to claim 1, wherein said obtaining the feature map to be convolved and segmenting a number of feature sub-maps from the feature map to be convolved according to a preset feature map segmentation rule specifically includes: Obtain the feature map to be convolved; If the length or width of the acquired feature map to be convolved is not an integer multiple of the convolution step length, zero padding is performed on the preset position of the feature map to be convolved to make the zero padding feature map to be convolved The length or width is an integer multiple of the convolution step length; According to preset feature map segmentation rules, a number of feature sub-maps are segmented from the zero-filled feature map to be convolved. The neural network acceleration method according to claim 2, wherein, if the convolution step size is not 1 and the size of the convolution filter is greater than 1, according to a preset filter segmentation rule from the convolution The product filter divides into several sub-filters, including: If the convolution step length is 2 and the size of the convolution filter is 2×2, 4 sub-filters are divided from the convolution filter, and the size of each sub-filter is 1×1; Wherein the first subfilter includes the weights of odd rows and odd columns of the convolution filter, the second subfilter includes the weights of odd rows and even columns of the convolution filter, and the third subfilter includes the weights of the odd rows and even columns of the convolution filter. Weights of even rows and odd columns of a convolution filter, and the fourth subfilter includes weights of even rows and even columns of the convolution filter; If the convolution step size is 2 and the size of the convolution filter is 2×2, the acquiring a feature map to be convolved and segmenting it from the feature map to be convolved according to a preset feature map segmentation rule Several feature subgraphs, including: Assign the values of odd rows and odd columns of the feature map to be convolved to the corresponding positions of the first feature submap, and assign the values of odd rows and even columns of the feature map to be convolved to the corresponding positions of the second feature submap Position, assign the values of the even rows and odd columns of the feature map to be convolved to the corresponding positions of the third feature submap, and assign the values of the even rows and even columns of the feature map to be convolved to the fourth feature submap Corresponding position. The neural network acceleration method according to claim 2, wherein, if the convolution step size is not 1 and the size of the convolution filter is greater than 1, according to a preset filter segmentation rule from the convolution The product filter divides into several sub-filters, including: If the convolution step length is 2 and the size of the convolution filter is 3×3, 4 sub-filters are divided from the convolution filter, and the size of each sub-filter is 2×2; Wherein the first subfilter includes the weights of odd rows and odd columns of the convolution filter, the second subfilter includes the weights of odd rows and even columns of the convolution filter, and the third subfilter includes the weights of the odd rows and even columns of the convolution filter. Weights of even rows and odd columns of a convolution filter, and the fourth subfilter includes weights of even rows and even columns of the convolution filter; If the convolution step size is 2 and the size of the convolution filter is 3×3, the acquiring feature map to be convolved and segmenting it from the feature map to be convolved according to a preset feature map segmentation rule Several feature subgraphs, including: Assign the values of odd rows and odd columns of the feature map to be convolved to the corresponding positions of the first feature submap, and assign the values of odd rows and even columns of the feature map to be convolved to the corresponding positions of the second feature submap Position, assign the values of the even rows and odd columns of the feature map to be convolved to the corresponding positions of the third feature submap, and assign the values of the even rows and even columns of the feature map to be convolved to the fourth feature submap Corresponding position. The neural network acceleration method according to claim 4, wherein, if the convolution step size is 2 and the size of the convolution filter is 3×3, 4 sub-convolution filters are divided The filter, the size of each sub-filter is 2×2, and specifically includes: Allocating weights of odd rows and odd columns of the convolution filter to the first subfilter; Assign the weights of the odd rows and even columns of the convolution filter to the first column of the second subfilter, and fill the second column of the second subfilter with 0; Assign the weights of the even rows and odd columns of the convolution filter to the first row of the third subfilter, and fill the second row of the third subfilter with 0; Assign the weights of the even rows and even columns of the convolution filter to the first row and first column of the fourth subfilter, and fill the remaining positions of the fourth subfilter with 0. The neural network acceleration method according to claim 2, wherein, if the convolution step size is not 1 and the size of the convolution filter is greater than 1, according to a preset filter segmentation rule from the convolution The product filter divides into several sub-filters, including: If the convolution step length is 3 and the size of the convolution filter is 3×3, 9 sub-filters are divided from the convolution filter, and the size of each sub-filter is 1×1 and Each includes one of the 9 weights of the convolution filter; If the convolution step length is 3 and the size of the convolution filter is 3×3, the acquiring the feature map to be convolved and segmenting it from the feature map to be convolved according to a preset feature map segmentation rule Several feature subgraphs, including: Assign the value in row 3n+1 and column 3n+1 of the feature map to be convolved to the corresponding position of the first feature submap, and assign the value in row 3n+1 and column 3n+2 of the feature map to be convolved The value of is assigned to the corresponding position of the second feature submap, and the value of row 3n+1 and column 3n+3 of the feature map to be convolved is assigned to the corresponding position of the third feature submap. The value in row 3n+2, column 3n+1 of the convolution feature map is allocated to the corresponding position of the fourth feature submap, and the value in row 3n+2, column 3n+2 of the feature map to be convolved is allocated to Assign the value in row 3n+2 and column 3n+3 of the feature map to be convolved to the corresponding position of the fifth feature submap to the corresponding location of the feature map to be convolved. The value in row 3n+3 and column 3n+1 is allocated to the corresponding position of the seventh feature submap, and the value in row 3n+3 and column 3n+2 of the feature map to be convolved is allocated to the eighth feature For the corresponding position of the sub-image, the value in the 3n+3 row and 3n+3 column of the feature map to be convolved is assigned to the corresponding position of the ninth feature sub-image, where n is a natural number. 7. The neural network acceleration method according to any one of claims 1 to 6, wherein: the systolic array is used to perform convolution calculation on the corresponding feature submap according to each of the subfilters, which specifically includes: Loading the weight of the sub-filter into the weight register connected to the systolic array; Loading the feature sub-map corresponding to the sub-filter into the input register connected to the systolic array; Obtain the output result of the systolic array convolution calculation. A neural network acceleration device based on a systolic array, including: A convolution parameter acquisition module, configured to acquire convolution parameters of a convolution filter, where the convolution parameters include a convolution step size and a size of the convolution filter; The filter segmentation module is configured to: if the convolution step size is not 1 and the size of the convolution filter is greater than 1×1, segment the convolution filter according to a preset filter segmentation rule. Filter, the size of each of the sub-filters is smaller than the size of the convolution filter; The feature map segmentation module is used to obtain the feature map to be convolved and segment the feature map to be convolved according to a preset feature map segmentation rule into several feature sub-images, the several feature sub-images and the several sub-filters One-to-one correspondence The convolution module is configured to perform convolution calculation on the corresponding feature sub-maps according to each of the sub-filters based on the systolic array, and the step size of the convolution calculation is 1; The superposition module is configured to superimpose the convolution calculation results corresponding to each of the sub-filters, and output the superposition result as the convolution calculation result of the feature map to be convolved by the convolution filter. A computer device including a memory and a processor; The memory is used to store computer programs; The processor is configured to execute the computer program and implement the following steps when executing the computer program: Acquiring a convolution parameter of a convolution filter, where the convolution parameter includes a convolution step size and the size of the convolution filter; If the convolution step length is not 1 and the size of the convolution filter is greater than 1×1, a number of sub-filters are segmented from the convolution filter according to a preset filter segmentation rule, and each sub-filter is The size of the filter is smaller than the size of the convolution filter; Acquiring a feature map to be convolved, and segmenting a number of feature submaps from the feature map to be convolved according to a preset feature map segmentation rule, the plurality of feature submaps corresponding to the plurality of subfilters one-to-one; Based on the systolic array, perform convolution calculation on the corresponding feature submap according to each of the subfilters, and the step size of the convolution calculation is 1; The convolution calculation results corresponding to each of the subfilters are superimposed, and the superimposed result is output as the convolution calculation result of the feature map to be convolved by the convolution filter. The computer device according to claim 9, wherein the processor implements the acquisition of the feature map to be convolved and the segmentation of several feature sub-maps from the feature map to be convolved according to a preset feature map segmentation rule , Used to implement the following steps: Obtain the feature map to be convolved; If the length or width of the acquired feature map to be convolved is not an integer multiple of the convolution step length, zero padding is performed on the preset position of the feature map to be convolved to make the zero padding feature map to be convolved The length or width is an integer multiple of the convolution step length; According to preset feature map segmentation rules, a number of feature sub-maps are segmented from the zero-filled feature map to be convolved. The computer device according to claim 10, wherein the processor implements that if the convolution step size is not 1 and the size of the convolution filter is greater than 1, according to a preset filter division rule When several sub-filters are segmented from the convolution filter, it is used to implement the following steps: If the convolution step length is 2 and the size of the convolution filter is 2×2, 4 sub-filters are divided from the convolution filter, and the size of each sub-filter is 1×1; Wherein the first subfilter includes the weights of odd rows and odd columns of the convolution filter, the second subfilter includes the weights of odd rows and even columns of the convolution filter, and the third subfilter includes the weights of the odd rows and even columns of the convolution filter. Weights of even rows and odd columns of a convolution filter, and the fourth subfilter includes weights of even rows and even columns of the convolution filter; If the convolution step size is 2 and the size of the convolution filter is 2×2, the acquiring a feature map to be convolved and segmenting it from the feature map to be convolved according to a preset feature map segmentation rule Several feature subgraphs, including: Assign the values of odd rows and odd columns of the feature map to be convolved to the corresponding positions of the first feature submap, and assign the values of odd rows and even columns of the feature map to be convolved to the corresponding positions of the second feature submap Position, assign the values of the even rows and odd columns of the feature map to be convolved to the corresponding positions of the third feature submap, and assign the values of the even rows and even columns of the feature map to be convolved to the fourth feature submap Corresponding position. The computer device according to claim 10, wherein the processor implements that if the convolution step size is not 1 and the size of the convolution filter is greater than 1, according to a preset filter division rule When several sub-filters are segmented from the convolution filter, it is used to implement the following steps: If the convolution step length is 2 and the size of the convolution filter is 3×3, 4 sub-filters are divided from the convolution filter, and the size of each sub-filter is 2×2; Wherein the first subfilter includes the weights of odd rows and odd columns of the convolution filter, the second subfilter includes the weights of odd rows and even columns of the convolution filter, and the third subfilter includes the weights of the odd rows and even columns of the convolution filter. Weights of even rows and odd columns of a convolution filter, and the fourth subfilter includes weights of even rows and even columns of the convolution filter; If the convolution step size is 2 and the size of the convolution filter is 3×3, the acquiring feature map to be convolved and segmenting it from the feature map to be convolved according to a preset feature map segmentation rule Several feature subgraphs, including: Assign the values of odd rows and odd columns of the feature map to be convolved to the corresponding positions of the first feature submap, and assign the values of odd rows and even columns of the feature map to be convolved to the corresponding positions of the second feature submap Position, assign the values of the even rows and odd columns of the feature map to be convolved to the corresponding positions of the third feature submap, and assign the values of the even rows and even columns of the feature map to be convolved to the fourth feature submap Corresponding position. The computer device according to claim 10, wherein the processor implements that if the convolution step size is not 1 and the size of the convolution filter is greater than 1, according to a preset filter division rule When several sub-filters are segmented from the convolution filter, it is used to implement the following steps: If the convolution step length is 3 and the size of the convolution filter is 3×3, 9 sub-filters are divided from the convolution filter, and the size of each sub-filter is 1×1 and Each includes one of the 9 weights of the convolution filter; If the convolution step length is 3 and the size of the convolution filter is 3×3, the acquiring the feature map to be convolved and segmenting it from the feature map to be convolved according to a preset feature map segmentation rule Several feature subgraphs, including: Assign the value in row 3n+1 and column 3n+1 of the feature map to be convolved to the corresponding position of the first feature submap, and assign the value in row 3n+1 and column 3n+2 of the feature map to be convolved The value of is assigned to the corresponding position of the second feature submap, and the value of row 3n+1 and column 3n+3 of the feature map to be convolved is assigned to the corresponding position of the third feature submap. The value in row 3n+2, column 3n+1 of the convolution feature map is allocated to the corresponding position of the fourth feature submap, and the value in row 3n+2, column 3n+2 of the feature map to be convolved is allocated to Assign the value in row 3n+2 and column 3n+3 of the feature map to be convolved to the corresponding position of the fifth feature submap to the corresponding location of the feature map to be convolved. The value in row 3n+3 and column 3n+1 is allocated to the corresponding position of the seventh feature submap, and the value in row 3n+3 and column 3n+2 of the feature map to be convolved is allocated to the eighth feature For the corresponding position of the sub-image, the value in the 3n+3 row and 3n+3 column of the feature map to be convolved is assigned to the corresponding position of the ninth feature sub-image, where n is a natural number. The computer device according to any one of claims 9-13, wherein: when the processor implements the systolic array based on each of the sub-filters to perform convolution calculations on the corresponding feature sub-maps, Used to implement the following steps: Loading the weight of the sub-filter into the weight register connected to the systolic array; Loading the feature sub-map corresponding to the sub-filter into the input register connected to the systolic array; Obtain the output result of the systolic array convolution calculation. A computer-readable storage medium that stores a computer program, and if the computer program is executed by a processor, the following steps are implemented: Acquiring a convolution parameter of a convolution filter, where the convolution parameter includes a convolution step size and the size of the convolution filter; If the convolution step length is not 1 and the size of the convolution filter is greater than 1×1, a number of sub-filters are segmented from the convolution filter according to a preset filter segmentation rule, and each sub-filter is The size of the filter is smaller than the size of the convolution filter; Acquiring a feature map to be convolved, and segmenting a number of feature submaps from the feature map to be convolved according to a preset feature map segmentation rule, the plurality of feature submaps corresponding to the plurality of subfilters one-to-one; Based on the systolic array, perform convolution calculation on the corresponding feature submap according to each of the subfilters, and the step size of the convolution calculation is 1; The convolution calculation results corresponding to each of the subfilters are superimposed, and the superimposed result is output as the convolution calculation result of the feature map to be convolved by the convolution filter. The storage medium according to claim 15, wherein the processor implements the acquisition of the feature map to be convolved and the segmentation of several feature sub-maps from the feature map to be convolved according to a preset feature map segmentation rule , Used to implement the following steps: Obtain the feature map to be convolved; If the length or width of the acquired feature map to be convolved is not an integer multiple of the convolution step length, zero padding is performed on the preset position of the feature map to be convolved to make the zero padding feature map to be convolved The length or width is an integer multiple of the convolution step length; According to preset feature map segmentation rules, a number of feature sub-maps are segmented from the zero-filled feature map to be convolved. The storage medium according to claim 16, wherein the processor implements that if the convolution step size is not 1 and the size of the convolution filter is greater than 1, according to a preset filter division rule When several sub-filters are segmented from the convolution filter, it is used to implement the following steps: If the convolution step length is 2 and the size of the convolution filter is 2×2, 4 sub-filters are divided from the convolution filter, and the size of each sub-filter is 1×1; Wherein the first subfilter includes the weights of odd rows and odd columns of the convolution filter, the second subfilter includes the weights of odd rows and even columns of the convolution filter, and the third subfilter includes the weights of the odd rows and even columns of the convolution filter. Weights of even rows and odd columns of a convolution filter, and the fourth subfilter includes weights of even rows and even columns of the convolution filter; If the convolution step size is 2 and the size of the convolution filter is 2×2, the acquiring a feature map to be convolved and segmenting it from the feature map to be convolved according to a preset feature map segmentation rule Several feature subgraphs, including: Assign the values of odd rows and odd columns of the feature map to be convolved to the corresponding positions of the first feature submap, and assign the values of odd rows and even columns of the feature map to be convolved to the corresponding positions of the second feature submap Position, assign the values of the even rows and odd columns of the feature map to be convolved to the corresponding positions of the third feature submap, and assign the values of the even rows and even columns of the feature map to be convolved to the fourth feature submap Corresponding position. The storage medium according to claim 16, wherein the processor implements that if the convolution step size is not 1 and the size of the convolution filter is greater than 1, according to a preset filter division rule When several sub-filters are segmented from the convolution filter, it is used to implement the following steps: If the convolution step length is 2 and the size of the convolution filter is 3×3, 4 sub-filters are divided from the convolution filter, and the size of each sub-filter is 2×2; Wherein the first subfilter includes the weights of odd rows and odd columns of the convolution filter, the second subfilter includes the weights of odd rows and even columns of the convolution filter, and the third subfilter includes the weights of the odd rows and even columns of the convolution filter. Weights of even rows and odd columns of a convolution filter, and the fourth subfilter includes weights of even rows and even columns of the convolution filter; If the convolution step size is 2 and the size of the convolution filter is 3×3, the acquiring feature map to be convolved and segmenting it from the feature map to be convolved according to a preset feature map segmentation rule Several feature subgraphs, including: Assign the values of odd rows and odd columns of the feature map to be convolved to the corresponding positions of the first feature submap, and assign the values of odd rows and even columns of the feature map to be convolved to the corresponding positions of the second feature submap Position, assign the values of the even rows and odd columns of the feature map to be convolved to the corresponding positions of the third feature submap, and assign the values of the even rows and even columns of the feature map to be convolved to the fourth feature submap Corresponding position. The storage medium according to claim 16, wherein the processor implements that if the convolution step size is not 1 and the size of the convolution filter is greater than 1, according to a preset filter division rule When several sub-filters are segmented from the convolution filter, it is used to implement the following steps: If the convolution step length is 3 and the size of the convolution filter is 3×3, 9 sub-filters are divided from the convolution filter, and the size of each sub-filter is 1×1 and Each includes one of the 9 weights of the convolution filter; If the convolution step length is 3 and the size of the convolution filter is 3×3, the acquiring the feature map to be convolved and segmenting it from the feature map to be convolved according to a preset feature map segmentation rule Several feature subgraphs, including: Assign the value in row 3n+1 and column 3n+1 of the feature map to be convolved to the corresponding position of the first feature submap, and assign the value in row 3n+1 and column 3n+2 of the feature map to be convolved The value of is assigned to the corresponding position of the second feature submap, and the value of row 3n+1 and column 3n+3 of the feature map to be convolved is assigned to the corresponding position of the third feature submap. The value in row 3n+2, column 3n+1 of the convolution feature map is allocated to the corresponding position of the fourth feature submap, and the value in row 3n+2, column 3n+2 of the feature map to be convolved is allocated to Assign the value in row 3n+2 and column 3n+3 of the feature map to be convolved to the corresponding position of the fifth feature submap to the corresponding location of the feature map to be convolved. The value in row 3n+3 and column 3n+1 is allocated to the corresponding position of the seventh feature submap, and the value in row 3n+3 and column 3n+2 of the feature map to be convolved is allocated to the eighth feature For the corresponding position of the sub-image, the value in the 3n+3 row and 3n+3 column of the feature map to be convolved is assigned to the corresponding position of the ninth feature sub-image, where n is a natural number. 22. The storage medium according to any one of claims 15-19, wherein: when the processor implements the systolic-based array and performs convolution calculations on the corresponding feature submaps according to the subfilters, Used to implement the following steps: Loading the weight of the sub-filter into the weight register connected to the systolic array; Loading the feature sub-map corresponding to the sub-filter into the input register connected to the systolic array; Obtain the output result of the systolic array convolution calculation.

Images