KR102710479B1 - Apparatus and method for accelerating neural network inference based on efficient address translation - Google Patents

Abstract

A neural network inference acceleration device and method having an efficient address conversion function are disclosed. The neural network inference acceleration method having an efficient address conversion function according to one embodiment of the present invention may include: (a) a step of obtaining location information and dimensional parameters associated with operations performed in a predetermined layer forming a neural network; and (b) a step of generating address information for features and weights of the layer based on the location information and the dimensional parameters.

Description

{APPARATUS AND METHOD FOR ACCELERATING NEURAL NETWORK INFERENCE BASED ON EFFICIENT ADDRESS TRANSLATION}

The present invention relates to a neural network inference acceleration device and method having an efficient address translation function. For example, the present invention relates to a low complexity convolutional neural network inference acceleration circuit device having a reconfigurable address translation function and a method for driving the same.

This study was supported by the research grant of the Gyeonggi Regional Research Center (GRRC) under the project titled “3D Spatial Data Processing and Application Technology Research” (Research Period: 2020-07-01 ~ 2021-06-30) (Research Management Agency: Gyeonggi Province, Host Organization: Korea Aerospace University Industry-Academic Cooperation Foundation, Project Identification Number: GRRC-2017-B02).

A convolutional neural network (CNN) is a type of deep learning technology used for image analysis, etc., and has a network structure consisting of an input layer, an output layer, and multiple hidden layers between the input and output layers. Specifically, each layer consists of convolution, pooling, and activation functions, and as these layers are repeated, the features of an image are extracted and classified. Here, the convolution operation is a process of calculating convolution by applying the window of the filter, which is the weight of the convolution, to the input data while moving it at regular intervals.

This convolution (convolution operation) is applied to each layer to perform the operation, and it takes up the longest time in the computation process for image inference due to the repetitive convolution operation, and has the problem of lowering the operating frequency in hardware implementation.

Meanwhile, the convolutional neural network inference acceleration circuit device is configured to perform tasks such as address generation, memory reading, computational process for convolutional neural network inference, and memory writing, and performs image inference for one layer. Here, the address generation process for generating an address for loading input features and weight values from memory and an address for storing inferred output features is performed before the convolution operation for each layer. Accordingly, if the address generation process is implemented in a complex manner, a problem occurs in which the inference speed is significantly reduced each time convolution is performed.

The background technology of this application is disclosed in Korean Patent Publication No. 10-2107077.

The present invention is intended to solve the problems of the above-mentioned prior art, and to provide a neural network inference acceleration device and method equipped with an efficient address conversion function capable of compensating for the decrease in inference speed that occurs during the address generation process among the operations performed by a neural network inference acceleration circuit (accelerator).

However, the technical tasks to be achieved by the embodiments of the present invention are not limited to the technical tasks described above, and other technical tasks may exist.

As a technical means for achieving the above-mentioned technical task, a neural network inference acceleration method having an efficient address conversion function according to one embodiment of the present invention may include (a) a step of obtaining location information and dimensional parameters associated with operations performed in a predetermined layer forming a neural network, and (b) a step of generating address information for features and weights of the layer based on the location information and the dimensional parameters.

In addition, the step (b) may include a step of (b1) determining a layout of the address information based on the dimensional parameters, and a step of (b2) performing a predetermined reference operation on each of a plurality of coordinates included in the location information to determine an address value corresponding to the layout.

Additionally, the above reference operations may include shift operations, AND operations, and OR operations.

In addition, the step (b) can generate first address information for the input feature of the layer, second address information for the weight, and third address information for the output feature of the layer, respectively.

Additionally, each of the first address information and the second address information may be address information for retrieving the input feature and the weight, respectively, from a memory that stores data linked to the neural network.

Additionally, the third address information may be address information for storing the result of an operation using the neural network in the memory.

Additionally, the neural network may be a convolutional neural network (CNN).

Additionally, in the step (a), the dimension parameter for the feature may correspond to three dimensions, and the dimension parameter for the weight may correspond to four dimensions.

Meanwhile, a neural network inference acceleration method having an efficient address conversion function according to one embodiment of the present invention may include: (a) a step of generating address information for input features, weights, and output features of a predetermined layer forming a neural network, respectively, based on location information and dimensional parameters associated with an operation performed in the layer, (b) a step of obtaining the input features and the weights from a memory storing data associated with the neural network based on the address information, (c) a step of performing the operation based on the input features and the weights, and (d) a step of storing a result of performing the operation in the memory based on the address information.

Meanwhile, a neural network inference acceleration device equipped with an efficient address conversion function according to one embodiment of the present invention may include a parameter setting unit that obtains location information and dimensional parameters associated with operations performed in a predetermined layer forming a neural network, and an address conversion unit that generates address information for features and weights of the layer based on the location information and the dimensional parameters.

In addition, the address conversion unit may include a layout generation unit that determines a layout of the address information based on the dimensional parameters, and a conversion performing unit that performs a predetermined reference operation on each of a plurality of coordinates included in the location information to determine an address value corresponding to the layout.

In addition, the address conversion unit can generate first address information for the input feature of the layer, second address information for the weight, and third address information for the output feature of the layer, respectively.

Meanwhile, a neural network accelerator equipped with an efficient address conversion function according to one embodiment of the present invention may include an address generation module which generates address information for input features, weights and output features of a predetermined layer forming a neural network, respectively, based on location information and dimensional parameters associated with an operation performed in the layer; a search module which obtains the input features and the weights from a memory storing data associated with the neural network based on the address information; a calculation module which performs the operation based on the input features and the weights; and a recording module which stores a result of performing the operation in the memory based on the address information.

The above-described problem solving means are merely exemplary and should not be construed as limiting the present invention. In addition to the above-described exemplary embodiments, additional embodiments may exist in the drawings and detailed description of the invention.

According to the above-described means for solving the problem of the present invention, a neural network inference acceleration device and method having an efficient address conversion function capable of compensating for a decrease in inference speed that occurs during an address generation process among operations performed by a neural network inference acceleration circuit (accelerator) can be provided.

According to the solution to the problem of the present invention described above, address conversion can be performed efficiently even in response to multidimensional data by considering that the size of the filter window can change depending on the characteristics of the input data, such as resolution.

According to the above-described means for solving the problem of the present invention, the complexity is reduced by implementing the operation for generating address information based on relatively simple operations such as shift operations and AND operations, thereby effectively reducing the inference time of the neural network.

According to the above-described means for solving the problem of the present invention, an address can be generated flexibly based on the location and dimension parameters of a feature, so that conversion of address information can be easily performed regardless of the dimension of data.

However, the effects that can be obtained from this invention are not limited to the effects described above, and other effects may exist.

FIG. 1 is a schematic diagram of a neural network-based inference system including a neural network inference accelerator equipped with an efficient address conversion function according to one embodiment of the present invention.
FIG. 2 is a schematic diagram of a neural network accelerator including a neural network inference accelerator with an efficient address translation function according to one embodiment of the present invention.
FIG. 3 is a conceptual diagram for explaining a method of converting address information generated by a neural network inference acceleration device equipped with an efficient address conversion function according to one embodiment of the present invention.
FIG. 4 is a schematic diagram of a neural network inference acceleration device having an efficient address conversion function according to one embodiment of the present invention.
FIG. 5 is a flowchart illustrating an operation of a neural network inference acceleration method having an efficient address conversion function according to the first embodiment of the present invention.
FIG. 6 is a flowchart of an operation of a neural network inference acceleration method equipped with an efficient address conversion function according to the second embodiment of the present invention.
Figure 7 is a detailed operation flow diagram of a neural network inference acceleration method equipped with an efficient address conversion function according to the second embodiment of the present invention.

Below, with reference to the attached drawings, embodiments of the present invention are described in detail so that those with ordinary skill in the art can easily practice the present invention. However, the present invention may be implemented in various different forms and is not limited to the embodiments described herein. In addition, in order to clearly describe the present invention in the drawings, parts that are not related to the description are omitted, and similar parts are assigned similar drawing reference numerals throughout the specification.

Throughout this specification, when a part is said to be "connected" to another part, this includes not only the case where it is "directly connected," but also the case where it is "electrically connected" or "indirectly connected" with another element in between.

Throughout this specification, when it is said that an element is located “on,” “above,” “below,” “below,” or “below” another element, this includes not only cases where an element is in contact with another element, but also cases where another element exists between the two elements.

Throughout this specification, whenever a part is said to "include" a component, this does not mean that it excludes other components, but rather that it may include other components, unless otherwise specifically stated.

The present invention relates to a neural network inference acceleration device and method having an efficient address translation function. For example, the present invention relates to a low complexity convolutional neural network inference acceleration circuit device having a reconfigurable address translation function and a method for driving the same.

FIG. 1 is a schematic diagram of a neural network-based inference system including a neural network inference accelerator equipped with an efficient address conversion function according to one embodiment of the present invention.

Referring to FIG. 1, a neural network-based inference system (1000) disclosed in the present invention may include a neural network accelerator (10) according to one embodiment of the present invention and a memory (20) for storing data for neural network operation, output data according to neural network operation (inference) results, etc.

Meanwhile, in the description of the embodiments of the present invention, the neural network may mean a convolutional neural network (CNN) that performs inference using image data and the like as input, but is not limited thereto, and may broadly include various types of artificial neural networks that are already known in the past or may be developed in the future, such as a deep neural network (DNN) and a recurrent neural network (RNN), depending on the implementation example of the present invention.

FIG. 2 is a schematic diagram of a neural network accelerator including a neural network inference accelerator with an efficient address translation function according to one embodiment of the present invention.

Referring to FIG. 2, a neural network accelerator (10) according to one embodiment of the present invention may include an address generation module (100), a search module (200), a calculation module (300), and a recording module (400). For reference, in the description of the embodiment of the present invention, a neural network inference acceleration device (100) equipped with an efficient address conversion function according to one embodiment of the present invention (hereinafter referred to as 'inference acceleration device (100)') may be referred to differently as an address generation module (100) in that it is responsible for the function of generating address information linked to input data and output data for performing an inference process of a neural network. That is, in the description of the embodiment of the present invention, the reference numeral 100 may be used interchangeably for 'inference acceleration device (100)' and 'address generation module (100)'.

More specifically, the inference acceleration device (100) or address generation module (100) disclosed in the present invention can generate address information (such as 'I addr.', 'F addr.', 'O addr.' in FIG. 1) for input features, weights, and output features of a given layer forming a neural network based on location information and dimensional parameters associated with operations performed in the corresponding layer, as described in detail below.

In addition, referring to FIG. 1, the search module (200) can obtain input features ('I' in FIG. 1) and weights ('F' in FIG. 1) from a memory (20) that stores data linked to a neural network based on the generated address information.

In addition, the operation module (300) can perform an operation performed in the corresponding layer based on the input features and weights obtained from the memory (20). In this regard, referring to FIG. 1, the operation performed by the operation module (300) may include, but is not limited to, a convolution operation, an activation operation (activation function), a pooling operation (e.g., Max Pooling), etc.

Hereinafter, with reference to FIG. 3, specific functions and operations including the address information generation process of the inference accelerator (100) will be described in detail.

FIG. 3 is a conceptual diagram for explaining a method of converting address information generated by a neural network inference acceleration device equipped with an efficient address conversion function according to one embodiment of the present invention.

Referring to FIG. 3, the inference accelerator (100) can obtain position information and dimensional parameters associated with operations performed in a given layer forming a neural network.

Specifically, in the description of the embodiments of the present invention, the location information may include coordinate information on input data (e.g., input image, etc.) of input features on which operations are performed in the corresponding layer, and coordinate information of weights. For reference, in the description of the embodiments of the present invention, the weights may be referred to differently as weights, filters, kernels, etc.

In addition, in the description of the embodiments of the present invention, the dimension parameter may mean a tensor dimension associated with a feature including an input feature and an output feature and a weight for performing a convolution operation on the input feature. For example, the dimension parameter for the feature may correspond to three dimensions, and the dimension parameter for the weight may correspond to four dimensions, but is not limited thereto.

Meanwhile, referring to the 'Address' part of Fig. 3, if the tensor is four-dimensional, the layout of address information is generated according to the sizes of the four dimensions, and the address information can be stored in a location corresponding to each dimension. On the other hand, if the tensor is three-dimensional, the layout of address information is generated according to the sizes of the three dimensions with the dotted line part removed in the 'Address' part of Fig. 3, and the address information can be stored in a location corresponding to each dimension.

In this regard, a convolutional neural network must apply filters of different resolutions depending on the size, characteristics, etc. of input data used for inference, and considering this, the inference acceleration device (100) disclosed in the present invention has an advantage in that it can be applied scalably to multidimensional data because it can dynamically generate address information based on the location and dimensional parameter information of the feature. Accordingly, the inference acceleration device (100) can efficiently generate address information even in the case of multidimensional data, and thus can dramatically improve the overall inference speed of the neural network.

In addition, the inference acceleration device (100) can generate address information for features and weights of a predetermined layer forming a neural network based on the acquired (set) location information and dimension parameters. More specifically, the inference acceleration device (100) can generate first address information for input features of the corresponding layer ('I addr.' in FIG. 1), second address information for weights ('F addr.' in FIG. 1), and third address information for output features of the corresponding layer ('O addr.' in FIG. 1), respectively.

In this regard, the first address information and the second address information are address information for respectively retrieving input features ('I' in FIG. 1) and weights ('F' in FIG. 1) from a memory (20) that stores data linked to a neural network, and the third address information may be address information for storing an operation result ('O' in FIG. 1) using a neural network in the memory (20).

In addition, according to one embodiment of the present invention, the inference acceleration device (100) can determine the layout of address information to be generated based on the dimensional parameters as described above, and perform a predetermined reference operation on each of a plurality of coordinates included in the location information to determine each address value corresponding to the generated layout.

Additionally, according to one embodiment of the present invention, the reference operation applied to the coordinates included in the location information to determine the address value may include a shift operation, an AND operation, and an OR operation.

In this regard, in the case of a conventional convolutional neural network inference acceleration circuit device, since the address generation process was mainly performed using an adder and a multiplier in the entire processor processing the convolutional neural network, the address generation process corresponded to a critical path in the entire circuit, which not only limited the operating frequency but also made it difficult to change the address according to the feature dimension, making it difficult to apply various images, unlike this, the inference acceleration device (100) disclosed in the present invention can drastically reduce the complexity by implementing the operation process for generating address information to perform a shift operation, an AND operation, and an OR operation.

Specifically, the inference accelerator (100) determines (defines) shift values and mask values for each coordinate included in the location information based on dimensional parameters, performs a shift operation on each coordinate by the determined shift value, and stores the converted address value in a preset bit using the determined mask value for each coordinate, thereby converting the location of a feature or weight into the address of each coordinate.

For example, the address transformation process for an input feature whose dimension parameter corresponds to three dimensions can be described through Equation 1 below.

[Formula 1]

Here, << represents a shift operator, & represents an AND operator, | represents an OR operator, x , y , and z represent coordinate values of input feature location information, respectively, x_shamt , y_shamt , and z_shamt represent shift values for shifting each coordinate, and x_mask , y_mask , and z_mask are mask values which are constants for storing each coordinate in a specified bit, and the given coordinate values are used to generate address values corresponding to each coordinate in x_trans_addr , y_trans_addr , and z_trans_addr using the shift operator and the AND operator, and the address values of each converted coordinate are finally converted to the first address information, which is the address information of the input feature, using the OR operator.

As another example, the address conversion process for weights corresponding to four dimensions with dimension parameters can be explained through Equation 2 below.

[Formula 2]

Referring to the above equation 2, the weight is also given a shift value and a mask value as constants in the same manner as the input feature, and by using this, the position information of the weight can be converted into the second address information, and while the input feature described above through equation 1 is three-dimensional, the weight has a difference in that the dimension parameter is expressed as four-dimensional, but even if the dimension increases, only the layout of the address information changes, and the address information can be generated using the shift AND and OR operators equivalent to the method of expressing the three-dimensional address.

This is reflected in the form of the address layout shown in the aforementioned Figure 3, and the difference in the address layout when the feature dimension is 3-dimensional and when it is 4-dimensional is that the address layout becomes larger as the dimension increases.

Meanwhile, when the locations of features and weights are converted into addresses by the inference acceleration device (100) as described in detail above, the neural network accelerator (10) can load input features and weights from the memory (20) using the generated address information, and calculate a synthetic convolution in convolution with the loaded data (in other words, the input features and weights) and perform operations including an activation function, pooling, etc. to complete the operation for inference in the corresponding layer, and the result (output) of the operation in this manner stores the output feature value in the memory (20) using the third address information generated by the inference acceleration device (100).

In summary, the neural network accelerator (10) including the inference accelerator (100) disclosed in the present invention first defines the location of the tensor, and generates the addresses of the input features, output features, and weights with the defined location information and the dimension parameter values. Specifically, the address generation module (100) creates the layout of the address through the dimension parameters, and the location information is converted into an address according to the layout of the address. In addition, the neural network accelerator (10) loads the input features and weights from the memory (20) using the generated address information, and then performs convolution with the loaded input features and weights to create a convolution, and performs an activation function and pooling from the convolution operation result. When the output feature of the corresponding layer is generated through this process, the value is written to the memory (20) as the address information (i.e., the third address information) corresponding to the first generated output feature, and the output feature is again used as an input feature for inference in the next layer. The inference process up to this point is a summary of the inference process of one layer, and these layers operate repeatedly to derive inference results for input data (e.g., image data, etc.).

FIG. 4 is a schematic diagram of a neural network inference acceleration device having an efficient address conversion function according to one embodiment of the present invention.

Referring to FIG. 4, the inference acceleration device (100) or address generation module (100) may include a parameter setting unit (110) and an address conversion unit (120). In addition, referring to FIG. 4, the address conversion unit (120) may include a layout generation unit (121) and a conversion performing unit (122).

The parameter setting unit (110) can obtain location information and dimensional parameters associated with operations performed in a given layer forming a neural network.

For example, the parameter setting unit (110) may set (determine) the dimension parameter for a feature (e.g., input feature, etc.) to correspond to three dimensions, and may set (determine) the dimension parameter for a weight to correspond to four dimensions, but is not limited thereto.

The address conversion unit (120) can generate address information for features and weights of the corresponding layer based on the aforementioned location information and dimension parameters.

More specifically, the address conversion unit (120) can generate first address information (I addr.) for an input feature of a given layer of the neural network, second address information (F addr.) for a weight, and third address information (O addr.) for an output feature of the corresponding layer, respectively.

First, the layout generation unit (121) can determine the layout of the address information to be generated based on the above-described dimensional parameters. In addition, the conversion performing unit (122) can perform a predetermined reference operation on each of the plurality of coordinates included in the above-described location information to determine an address value corresponding to the determined layout of the address information.

In this regard, the conversion performing unit (122) can derive each address value to be included in the address information by performing a reference operation including a shift operation, an AND operation, and an OR operation.

Below, we will briefly review the operating flow of this system based on the detailed explanation above.

FIG. 5 is a flowchart illustrating an operation of a neural network inference acceleration method having an efficient address conversion function according to the first embodiment of the present invention.

The neural network inference acceleration method with an efficient address conversion function illustrated in Fig. 5 can be performed by the inference acceleration device (100) described above. Therefore, even if the content is omitted below, the content described for the inference acceleration device (100) can be equally applied to the description of the neural network inference acceleration method with an efficient address conversion function.

Referring to FIG. 5, in step S11, the parameter setting unit (110) can obtain (a) location information and dimensional parameters associated with operations performed in a predetermined layer forming a neural network.

According to one embodiment of the present invention, in step S11, the parameter setting unit (110) can set (determine) the dimension parameter for a feature (e.g., input feature, etc.) to correspond to three dimensions, and set (determine) the dimension parameter for a weight to correspond to four dimensions.

Next, in step S12, the address conversion unit (120) can generate address information for features and weights of the corresponding layer based on the acquired location information and dimension parameters (b).

In this regard, in step S12, the address conversion unit (120) can generate first address information (I addr.) for input features of a given layer of the neural network, second address information (F addr.) for weights, and third address information (O addr.) for output features of the corresponding layer, respectively.

Specifically, in step S12, the layout generation unit (121) can determine the layout of the address information to be generated based on the acquired dimensional parameters (b1).

In addition, in step S12, the conversion performing unit (122) can determine an address value corresponding to the layout of the determined address information by performing a predetermined reference operation on each of the plurality of coordinates included in the (b2) location information.

According to one embodiment of the present invention, in step S12, the conversion performing unit (122) can derive each address value to be included in the address information by performing a reference operation including a shift operation, an AND operation, and an OR operation.

In the above description, steps S11 and S12 may be further divided into additional steps or combined into fewer steps, depending on the implementation example of the present invention.

FIG. 6 is a flowchart of an operation of a neural network inference acceleration method equipped with an efficient address conversion function according to the second embodiment of the present invention.

The neural network inference acceleration method equipped with an efficient address conversion function illustrated in Fig. 6 can be performed by the neural network accelerator (10) described above. Therefore, even if the content is omitted below, the content described for the neural network accelerator (10) can be equally applied to the description of Fig. 6.

Referring to FIG. 6, in step S21, the address generation module (100) can generate address information for input features, weights, and output features of each layer based on (a) location information and dimensional parameters associated with operations performed in a given layer forming a neural network.

Next, in step S22, the search module (200) can obtain input features and weights from the memory (20) storing data associated with the neural network based on the generated address information (b).

Next, in step S23, the operation module (300) can perform an operation including a convolution operation, an activation operation, and a pooling operation based on the acquired input features and weights (c).

Next, in step S24, the recording module (400) can store the result of performing the operation in step S23 in the memory (20) based on the address information generated in step S21.

In the above description, steps S21 to S24 may be further divided into additional steps or combined into fewer steps, depending on the implementation example of the present invention. In addition, some steps may be omitted as needed, and the order between the steps may be changed.

Figure 7 is a detailed operation flow diagram of a neural network inference acceleration method equipped with an efficient address conversion function according to the second embodiment of the present invention.

The detailed process of the neural network inference acceleration method with the efficient address conversion function illustrated in Fig. 7 can be performed by the neural network accelerator (10) described above. Therefore, even if the content is omitted below, the content described for the neural network accelerator (10) can be equally applied to the description of Fig. 7.

Referring to FIG. 7, in step S211, the parameter setting unit (110) can obtain location information and dimension parameters associated with operations performed in a predetermined layer forming a neural network. In other words, in step S211, the parameter setting unit (110) can define the location and dimension of features/weights for generating address information.

Next, in step S212, the transformation performing unit (122) can generate first address information, which is address information for an input feature associated with the corresponding layer, based on the location information and dimension parameters. In this regard, the first address information may mean address information for retrieving an input feature associated with the corresponding layer from a memory (20) that stores data associated with the neural network.

Next, in step S213, the conversion performing unit (122) can generate second address information, which is address information for the weight associated with the corresponding layer, based on the location information and the dimension parameters. In this regard, the second address information may mean address information for loading the weight associated with the corresponding layer from the memory (20) that stores data associated with the neural network.

Next, in step S214, the transformation performing unit (122) can generate third address information, which is address information for an output feature associated with the corresponding layer, based on the location information and dimension parameters. In this regard, the third address information may mean address information for storing the operation result in the corresponding layer using the neural network in the memory (20).

Next, in step S221, the search module (200) can obtain an input feature from the memory (20) based on the first address information among the generated address information.

Next, in step S222, the search module (200) can obtain a weight from the memory (20) based on the second address information among the generated address information.

Next, in step S231, the operation module (300) can perform a convolution operation (convolution) based on the acquired input features and weights.

Next, in step S232, the operation module (300) can perform an activation operation (in other words, apply an activation function) based on the acquired input features and weights.

Next, in step S233, the operation module (300) can perform a pooling operation (e.g., a Max Pooling operation, etc.) based on the acquired input features and weights.

Next, in step S24, the recording module (400) can store the results of the operations performed through steps S231 to S233 in the memory (20) based on the third address information.

In addition, referring to FIG. 7, the above-described steps S211 to S24 may be sequentially and repeatedly performed for a plurality of layers forming a neural network. In other words, when steps S211 to S24 are performed up to the last layer among the plurality of layers forming a neural network as illustrated in FIG. 7, inference for input data (input image) using the corresponding neural network may be completed.

In the above description, steps S211 to S24 may be further divided into additional steps or combined into fewer steps, depending on the implementation example of the present invention. In addition, some steps may be omitted as needed, and the order between the steps may be changed.

A neural network inference acceleration method having an efficient address conversion function according to one embodiment of the present invention may be implemented in the form of program commands that can be executed through various computer means and recorded on a computer-readable medium. The computer-readable medium may include program commands, data files, data structures, etc., alone or in combination. The program commands recorded on the medium may be those specially designed and configured for the present invention or may be those known to and usable by those skilled in the art of computer software. Examples of the computer-readable recording medium include magnetic media such as hard disks, floppy disks, and magnetic tapes, optical media such as CD-ROMs and DVDs, magneto-optical media such as floptical disks, and hardware devices specially configured to store and execute program commands such as ROMs, RAMs, and flash memories. Examples of the program commands include not only machine language codes generated by a compiler, but also high-level language codes that can be executed by a computer using an interpreter, etc. The above hardware devices may be configured to operate as one or more software modules to perform the operations of the present invention, and vice versa.

In addition, the neural network inference acceleration method having the above-described efficient address translation function can also be implemented in the form of a computer program or application executed by a computer stored in a recording medium.

The above description of the present invention is for illustrative purposes only, and those skilled in the art will understand that the present invention can be easily modified into other specific forms without changing the technical idea or essential features of the present invention. Therefore, it should be understood that the embodiments described above are exemplary in all respects and not restrictive. For example, each component described as a single component may be implemented in a distributed manner, and likewise, components described as distributed may be implemented in a combined manner.

The scope of the present application is indicated by the claims described below rather than the detailed description above, and all changes or modifications derived from the meaning and scope of the claims and their equivalent concepts should be interpreted as being included in the scope of the present application.

1000: Neural network-based inference system
10: Neural Network Accelerator
100: Neural network inference accelerator with efficient address conversion function, address generation module
110: Parameter setting section
120: Address conversion section
121: Layout generation section
122: Conversion execution unit
200: Navigation Module
300: Operation Module
400: Record Module
20: Memory

Claims (15)

A neural network inference acceleration method having an efficient address conversion function performed by a neural network inference acceleration device,
(a) a step of obtaining position information and dimensional parameters associated with an operation performed in any one of a plurality of layers including an input layer, an output layer, and a layer between the input layer and the output layer forming a neural network; and
(b) a step of generating address information for features and weights of the layer based on the location information and the dimension parameters;
Including,
Step (b) above,
(b1) a step of determining the layout of the address information based on the above dimension parameters; and
(b2) a step of determining an address value corresponding to the layout by performing a reference operation including a shift operation for each of the plurality of coordinates included in the location information;
Including,
The above step (b2) is,
An inference acceleration method, wherein the shift value and mask value for each coordinate included in the location information are determined based on the dimension parameters, a shift operation is performed for each coordinate by the shift value, and the address value converted to a preset bit is stored using the mask value, thereby converting the location of the feature or the weight into the address of each coordinate. delete In the first paragraph,
The above criteria operation is,
A method for accelerating inference, characterized by further including AND operations and OR operations. In the first paragraph,
Step (b) above,
An inference acceleration method, which generates first address information for input features of the above layer, second address information for the weights, and third address information for output features of the above layer, respectively. In paragraph 4,
The above first address information and the above second address information are address information for respectively loading the input feature and the weight from a memory storing data linked to the neural network.
An inference acceleration method, characterized in that the third address information is address information for storing the result of an operation using the neural network in the memory. In the first paragraph,
An inference acceleration method, characterized in that the above neural network is a convolutional neural network (CNN). In Article 6,
In step (a) above,
A method for accelerating inference, wherein the dimension parameter for the feature corresponds to three dimensions, and the dimension parameter for the weight corresponds to four dimensions. A neural network inference acceleration method having an efficient address translation function performed by a neural network accelerator,
(a) a step of generating address information for input features, weights and output features of a layer based on location information and dimensional parameters associated with operations performed in any one of a plurality of layers including an input layer, an output layer and layers between the input layer and the output layer forming a neural network;
(b) a step of obtaining the input features and the weights from a memory storing data associated with the neural network based on the address information;
(c) performing the operation based on the input features and the weights; and
(d) a step of storing the result of performing the above operation in the memory based on the address information;
Including,
Step (a) above,
A step of determining the layout of the address information based on the above dimension parameters; and
A step of determining an address value corresponding to the layout by performing a reference operation including a shift operation for each of the plurality of coordinates included in the above location information;
Including,
The step of determining the above address value is:
An inference acceleration method, wherein the shift value and mask value for each coordinate included in the location information are determined based on the dimension parameters, a shift operation is performed for each coordinate by the shift value, and the address value converted to a preset bit is stored using the mask value, thereby converting the location of the feature or the weight into the address of each coordinate. In Article 8,
The above neural network is a convolutional neural network (CNN).
A method for accelerating inference, wherein the above operations include convolution operations, activation operations, and pooling operations. In a neural network inference accelerator equipped with an efficient address translation function,
A parameter setting unit for obtaining position information and dimensional parameters associated with an operation performed in any one of a plurality of layers including an input layer, an output layer, and a layer between the input layer and the output layer forming a neural network; and
An address conversion unit that generates address information for features and weights of the layer based on the location information and the dimension parameters;
Including,
The above address conversion section,
A layout generation unit that determines the layout of the address information based on the above dimension parameters; and
A conversion performing unit that determines an address value corresponding to the layout by performing a reference operation including a shift operation for each of the plurality of coordinates included in the above location information;
Including,
The above conversion performing unit,
An inference accelerator device that determines a shift value and a mask value for each coordinate included in the location information based on the dimension parameters, performs a shift operation for each coordinate by the shift value, and stores the converted address value in a preset bit using the mask value, thereby converting the location of the feature or the weight into the address of each coordinate. delete In Article 10,
The above criteria operation is,
An inference accelerator device, characterized by further including AND operations and OR operations. In Article 10,
The above address conversion section,
An inference accelerator device which generates first address information for input features of the above layer, second address information for the weights, and third address information for output features of the above layer, respectively. In Article 10,
An inference accelerator device, characterized in that the above neural network is a convolutional neural network (CNN). As a neural network accelerator with efficient address translation capabilities,
An address generation module which generates address information for input features, weights and output features of a layer based on location information and dimensional parameters associated with operations performed in any one of a plurality of layers including an input layer, an output layer and layers between the input layer and the output layer forming a neural network;
A search module that obtains the input features and the weights from a memory storing data associated with the neural network based on the address information;
A computation module that performs the computation based on the input features and the weights; and
A recording module that stores the result of performing the above operation in the memory based on the address information;
Including,
The above address generation module,
The layout of the address information is determined based on the above dimensional parameters, and a reference operation including a shift operation is performed for each of the plurality of coordinates included in the location information to determine an address value corresponding to the layout.
The above address generation module,
A neural network accelerator that determines a shift value and a mask value for each coordinate included in the location information based on the dimension parameters, performs a shift operation for each coordinate by the shift value, and stores an address value converted to a preset bit using the mask value, thereby converting the location of the feature or the weight into an address of each coordinate.

Images