TWI754970B - Device, method and storage medium for accelerating operation of an activation function - Google Patents

Abstract

The invention provides a device for accelerating operation of an activation function. The device comprises a register for storing a storage table; a matching unit includes a plurality of comparators, a logic unit and a selection unit, and the plurality of comparators are connected with the selection unit through the logic unit, the plurality of comparators are used to match input variable of a activation function with variable interval of the activation function to obtain a comparison output result, and the logic unit performs logic operation according to the comparison output result to obtain the logic output result and determine the calculating variable interval; the selection unit is used to query the storage table according to the calculating variable interval to obtain parameters of a fitting quadratic function; the calculation unit is connected with the matching unit to complete the operation of the input variable according to the parameters. The invention also provides a method and a storage medium for accelerating operation of an activation function.

Description

Apparatus, method and storage medium for accelerating startup function operation

The present invention relates to the technical field of deep learning, and in particular, to a device, method and storage medium for accelerating the operation of a startup function.

For state-of-the-art artificial neural networks, data processing includes convolution, pooling, and priming. Among them, the role of priming is to provide the nonlinear modeling ability of the neural network. The functionality of the existing startup functions is varied and complex. The startup function may include exponential operation and division operation, etc., which introduces complex calculation and time consumption.

In view of the above problems, the present invention provides an apparatus, method and storage medium for accelerating the operation of the activation function, so as to improve the speed of processing the activation function.

A first aspect of the present application provides an apparatus for accelerating the operation of a startup function, the apparatus comprising: The temporary register is used to store a storage table, wherein the storage table describes the variable interval of the startup function and the mapping relationship of the parameters of the fitting quadratic function corresponding to the interval; the matching unit includes a plurality of comparators, a logic unit and a selection unit, the plurality of comparators are connected to the selection unit through the logic unit, and the plurality of comparators are used to match the input variable of the activation function with the variable interval of the activation function to obtain a comparison output result, The logic unit performs a logic operation according to the comparison output result to obtain a logic output result, and outputs the result according to the logic Determine the variable interval to be calculated; the selection unit is used for querying the storage table according to the variable interval to be calculated to obtain the parameters of the fitting quadratic function; the calculation unit is connected to the matching unit and is used for according to the parameter Completion of the operation on the input variable.

Preferably, the logic unit includes an inversion gate, a plurality of XOR gates and a temporary storage unit.

Preferably, the parameters of the fitting quadratic function include quadratic term coefficients, linear term coefficients and constants.

Preferably, the calculation unit includes a multiplier and an adder, wherein the multiplier receives the quadratic term coefficient, the first-order term coefficient and the input variable from the selection unit, and performs a multiplication operation; the adder receives from the selection unit. An addition operation is performed on the output result of the multiplier and the constant from the selection unit.

Preferably, when the input variable is less than the maximum variable in the variable interval, the comparator outputs a low level; When the input variable is greater than or equal to the maximum variable of the variable interval, the comparator outputs a high level.

或f(x)=max(0,x)。 Preferably, the startup function includes a function

or f ( x ) = max (0, x ).

A second aspect of the present application provides a method for accelerating the operation of a startup function, the method comprising: receiving an input variable; comparing the input variable and the variable interval of the startup function to obtain a comparison output result; and performing a logical operation on the comparison output result Obtaining a logical output result; determining a variable interval to be calculated according to the logical output result; querying a storage table according to the variable interval to be calculated to obtain parameters for fitting a quadratic function; and completing an operation for the input variable according to the parameters.

Preferably, the storage table describes the variable interval of the startup function and the mapping relationship of the parameters of the fitting quadratic function corresponding to the interval.

Preferably, the parameters of the fitting quadratic function include quadratic term coefficients, linear term coefficients and constants.

The third party aspect of the present invention provides a computer-readable storage medium on which a computer program is stored, and when the computer program is executed by a processor, the aforementioned data processing method is implemented.

The device, method and storage medium for accelerating the operation of a startup function provided by the present invention, and the device for accelerating the operation of a startup function provided by the present application can match the parameters of the quadratic function corresponding to the variable interval of the startup function by matching the matching unit, and then borrow the parameters of the quadratic function. The computation of the input variables of the activation function is performed by the computing unit according to the parameters. The calculation process of the activation function can be simplified, and the activation function can be fitted with higher precision while improving the calculation efficiency of the activation function processing in the neural network.

10: Computing device

110: Scratchpad

120: Matching unit

130: Computing Unit

121: Comparator

122: Logic Unit

123:Select unit

1220: Reverse gate

1221: XOR gate

1222: Temporary storage unit

200: Computing Systems

201: Receive module

202: Compare Mods

203: Processing modules

204: Determine the module

205: Query Module

1: Electronic equipment

11: Memory

12: Processor

13: Computer-readable storage media

14: Communication bus

FIG. 1 is a schematic diagram of an apparatus for accelerating the operation of an activation function provided by an embodiment of the present invention.

FIG. 2 is a schematic diagram of a matching unit in an apparatus for accelerating the operation of an activation function provided by an embodiment of the present invention.

FIG. 3 is a schematic diagram of a logic unit in an apparatus for accelerating the operation of an activation function provided by an embodiment of the present invention.

FIG. 4 is a schematic diagram of a calculation unit in an apparatus for accelerating the operation of an activation function provided by an embodiment of the present invention.

FIG. 5 is an example diagram of approximating the activation function by a quadratic function and approximating the activation function by a linear function according to an embodiment of the present invention.

FIG. 6 is a flowchart of a method for accelerating the operation of an activation function provided by an embodiment of the present invention.

FIG. 7 is a functional module diagram of an accelerated activation function computing system according to an embodiment of the present invention.

FIG. 8 is a schematic diagram of an electronic device according to an embodiment of the present invention.

In order to more clearly understand the above objects, features and advantages of the present invention, the present invention will be described in detail below with reference to the accompanying drawings and specific embodiments. It should be noted that the embodiments of the present application and the features in the embodiments may be combined with each other in the case of no conflict.

In the following description, many specific details are set forth in order to facilitate a full understanding of the present invention, and the described embodiments are only some, but not all, embodiments of the present invention.

Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention belongs. The terms used herein in the description of the present invention are for the purpose of describing specific embodiments only, and are not intended to limit the present invention.

，f(x)=tanh(x)，ReLU函數f(x)=max(0,x)。非線性的啟動函數在神經網路的啟動處理過程中存在執行速度慢的問題。為瞭解決該問題，本申請中將計算複雜的非線性函數藉由分段擬合二次函數和有限元方法來近似。具體地，藉由將啟動函數劃分為N段，每一段都藉由所述擬合二次函數以對其進行近似。計算每一段對應的擬合二次函數的參數，並存儲至係數記憶體中。 Please refer to FIG. 1 . FIG. 1 is a schematic diagram of an apparatus for accelerating the operation of a startup function (hereinafter referred to as “operation apparatus 10 ” for convenience of description) provided by an embodiment of the present invention. In this embodiment, the activation function of the neural network usually adopts a nonlinear function. For example the sigmoid function

, f ( x )=tanh( x ), the ReLU function f ( x )= max (0, x ). The nonlinear startup function has the problem of slow execution in the startup process of the neural network. In order to solve this problem, the computationally complex nonlinear functions are approximated by piecewise fitting quadratic functions and finite element methods in this application. Specifically, by dividing the activation function into N segments, each segment is approximated by the fitting quadratic function. Calculate the parameters of the quadratic function corresponding to each segment, and store them in the coefficient memory.

i

N。所述參數的資料類型為32位浮點型。 In this embodiment, the computing device 10 includes a temporary register 110 , a matching unit 120 , and a computing unit 130 connected to the matching unit 120 . It should be noted that, although not shown, each unit in the computing device 10 may be driven by a unified clock, so as to ensure the calculation sequence in the processing process. The scratchpad 110 includes a coefficient scratchpad, a segment scratchpad and a configuration scratchpad. The coefficient buffer is used to store the parameters of the fitting quadratic function for fitting the start function. In this embodiment, the parameters include a quadratic term coefficient a, a first-order term coefficient b, and a constant c. Using the parameters, the arithmetic expression of the fitting quadratic function corresponding to a certain variable interval of the activation function can be uniquely determined. For example, if the start-up function is divided into N segments, then N groups of parameters ( a _i , b _i , c _i ) are stored in the coefficient temporary register, where 1

i

N. The data type of the parameter is a 32-bit floating point type.

The segment buffer is used to store the variables of the start function after segment processing. For example, by dividing the variables of the startup function into N parts, N variable intervals can be obtained. In this embodiment, there is a corresponding relationship between the coefficient register and the segment register. The fitting quadratic function constructed by the parameters stored in the coefficient buffer may represent the start function corresponding to the variable interval stored in the segment buffer.

The configuration register is used to configure the fitting quadratic function to be applicable to various types of starting functions. In this implementation manner, the configuration register includes an enabling unit, and when the enable signal of the enabling unit is a high-level signal "1", the start-up after segment processing stored in the segment register is The function selects the corresponding fitting quadratic function; when the enable signal of the enable unit is 0, confirm that the output of the start function is equal to the input.

In one embodiment, the register 110 may further store a variable interval of the activation function and a mapping relationship of the parameters of the fitting quadratic function corresponding to the interval. Preferably, the mapping relationship may be in the form of a storage table. The storage table can be formed in the form of a temporary memory heap, that is, each group of variable interval indexes and corresponding fitting function parameters are stored in the temporary memory, and the temporary memory heap is used to store the complex numbers that can be used to refer to the data table. The outputs are simultaneously connected to a plurality of comparators, thereby improving the parallelism of the computing device. The plurality of outputs include a plurality of variable intervals and corresponding parameters for fitting a quadratic function.

For example, in one embodiment, the storage table constructed based on the sigmoid function is shown in Table 1 below.

In Table 1, N variable intervals are included, each variable interval corresponds to the variable interval range of the startup function, and the quadratic term coefficient, linear term coefficient and constant represent the quadratic term that fits the startup function within the corresponding variable interval range. The quadratic coefficient, linear coefficient, and) constant of the function. For example, the variable interval range of the startup function in Table 1 is divided into a plurality of equally spaced segment intervals [0, 1), [1, 2) and so on. For example, when the variable interval is [0,1), the parameters of the corresponding fitting quadratic function are quadratic term coefficient a ₁ , linear term coefficient b ₁ and constant c ₁ ; when the variable interval is [1,2), The parameters of the corresponding fitting quadratic function are a ₂ , the first-order coefficient b ₂ and the constant c ₂ .

It should be noted that the storage table parameters can be constructed online or dynamically changed during the startup process, and the storage table parameters of a plurality of startup functions involved in the neural network can also be constructed offline, and stored in the temporary register 110 in advance, In order to read dynamically during the startup process, it is preferable to construct the stored table parameters in an offline manner, so as to improve the efficiency of the startup process.

The matching unit 120 is configured to determine which variable interval the input variable of the startup function falls in according to the input variable and variable interval of the startup function, and then match the parameters of the corresponding fitting quadratic function according to the determined variable interval.

Referring to FIG. 2 , the matching unit 120 includes a plurality of comparators 121 , a logic unit 122 and a selection unit 123 . The plurality of comparators 121 are connected to the selection unit 123 through the logic unit 122 . FIG. 2 shows N comparators, which are respectively Comparator 1, Comparator 2 . . . Comparator N-1.

In this embodiment, the comparator 121 is used to match the input variable of the activation function with the variable interval of the activation function, and determine the output value according to the magnitude relationship between them. When the input variable x is less than the maximum variable x _N of the variable interval, the comparator outputs a low level "0"; when the input variable x is greater than or equal to the maximum variable x _N of the variable interval, the comparator outputs a high level level "1".

The logic unit 122 includes an inversion gate 1220 , a plurality of XOR gates 1221 and a temporary storage unit 1222 . For example, as shown in FIG. 3 , the output of the comparator 1 is respectively connected with the flip-gate sum of the logic unit 122 The XOR gate 1 is connected; the output of the comparator 2 is respectively connected with the XOR gate 1 and the XOR gate 2; the output of the comparator 3 is connected with the XOR gate 2 and the XOR gate 3 respectively; and so on, compare The output of the comparator N-2 is respectively connected with the XOR gate N-3 and the XOR gate N-2; the output of the comparator N-1 is respectively connected with the XOR gate N-2 and the temporary storage unit. When the input variable is greater than the maximum value of all variable ranges, the output result of the comparator N-1 is directly input to the temporary storage unit 1222 .

For example, one input of the comparator 2 is used to receive the input variable of the start function, and the other input is used to receive the maximum value x ₂ of the interval variable [1,2), and x ₂ approaches 2. For example, when the input variable is 1.5, since 1.5 is greater than 1, the comparator 1 outputs a high level "1", since 1.5 is less than 2, the comparator 2 outputs a low level "0", and similarly, since 1.5 is less than other interval variables the maximum value, so other comparators also output low level "0". Comparator 1 outputs a low level "0" after being reversed by the logic unit, while the output of comparator 1 and the output of comparator 2 output a high level "1" after being processed by the exclusive OR gate of the logic unit. The output of 2 and the output of comparator 3 output a low level "0" after passing through the XOR gate of the logic unit, and so on, the output of the comparator N-2 and the output of the comparator N-1 pass through the XOR gate of the logic unit Then output low level "0". Therefore, the selection unit selects the parameters ( a ₂ , b ₂ , c ₂ ) of the fitted quadratic function corresponding to the variable interval [1, 2). The parameters selected by the selection unit 123 are then output to the connected calculation unit 130 .

As shown in FIG. 4 , the calculation unit 130 includes a multiplier 130 and an adder 131, wherein the multiplier 130 receives the quadratic term coefficient a, the first-order term coefficient b and the input variable x from the selection unit 123, and performs a multiplication operation, and the adder 131 receives the output result from the multiplier 130 and the constant c from the selection unit 123, and performs an addition operation, so that the calculation unit 130 obtains the function value corresponding to the input variable x, generally expressed as f ( x )= ax ² + bx + c .

It should be noted that, in a preferred embodiment, the number of comparators in each matching unit is equal to the number of variable intervals, as shown in 2, when the startup function includes N variable intervals, each matching The number of comparators in the unit is also set to N. In the above-mentioned embodiments, the data selector, comparator, multiplier, and adder, etc. can be implemented by general-purpose or special-purpose devices.

In this embodiment, by using a fitted quadratic function to approximate the activation function, a smaller error can be brought about compared to approximating the activation function with a linear function. Taking the activation function as f ( x )=tanh( x ) as an example, as shown in Figure 5, the error of approximating f ( x )=tanh( x ) by a quadratic function is reduced by 50%, which is obviously better than that by a linear function. Approximate f ( x )=tanh( x ), which is more suitable for the operation of neural network.

Please refer to FIG. 6 . FIG. 6 is a flowchart of a method for accelerating the operation of an activation function according to an embodiment of the present application. According to different requirements, the order of the steps in the flowchart can be changed, and some steps can be omitted. In this embodiment, the method for accelerating the computation of the startup function is applied to the apparatus 10 for accelerating the computation of the startup function. The method for accelerating the operation of the activation function may include the following steps.

Step S1, receiving input variables.

In this embodiment, the input variable is the input variable of the excitation function.

Step S2, compare the input variable and the variable interval of the activation function to obtain a comparison output result.

In this embodiment, the comparator in the matching unit compares the input variable with the variable range of the activation function to obtain the comparison output result. The comparator is used for matching the input variable of the activation function with the variable interval of the activation function, and determining the output value according to the magnitude relationship between them. When the input variable x is less than the maximum value x _N of the interval variable, the comparison unit outputs a low level "0"; when the input variable x is greater than or equal to the maximum value x _N of the interval variable, the comparison unit outputs a high level level "1".

For example, one input of the comparator 2 is used to receive the input variable of the start function, and the other input is used to receive the maximum value x ₂ of the interval variable [1,2), and x ₂ approaches 2. For example, when the input variable is 1.5, since 1.5 is greater than 1, the comparator 1 outputs a high level "1"; since 1.5 is less than 2, the comparator 2 outputs a low level "0", and similarly, since 1.5 is less than other interval variables the maximum value, so other comparators also output low level "0".

Step S3, performing a logical operation on the comparison output result to obtain a logical output result.

In this embodiment, a logic output result is obtained by performing a logic operation on the comparison output result by the logic unit in the matching unit. The logic unit includes an anti-gate, a plurality of XOR gates and a temporary storage unit. For example, as shown in FIG. 3 , the output of the comparator 1 is respectively connected to the inverse gate and the XOR gate 1 of the logic unit; the output of the comparator 2 is respectively connected to the XOR gate 1 and the XOR gate 2 ; The output of the comparator 3 is connected with the XOR gate 2 and the XOR gate 3 respectively; and so on, the output of the comparator N-2 The output of the comparator N-1 is respectively connected to the XOR gate N-3 and the XOR gate N-2; the output of the comparator N-1 is respectively connected to the XOR gate N-2 and the temporary storage unit.

For example, when the input variable is 1.5, the comparator 1 outputs a low level "0" after the reverse gate of the logic unit, while the output of the comparator 1 and the output of the comparator 2 output high after the exclusive OR gate processing of the logic unit Level "1", the output of the comparator 2 and the output of the comparator 3 output a low level "0" after passing through the exclusive OR gate of the logic unit, and so on, the output of the comparator N-2 and the output of the comparator N-1 The output outputs a low level "0" after passing through the exclusive OR gate of the logic unit.

Step S4: Determine the variable interval to be calculated according to the logical output result.

In this embodiment, when the logical output result is "1", the variable interval where the interval variable of one input of the corresponding comparator is located is the variable interval to be calculated.

For example, when the input variable is 1.5, the comparator 1 outputs a low level "0" after the reverse gate of the logic unit, while the output of the comparator 1 and the output of the comparator 2 output high after the exclusive OR gate processing of the logic unit When the level is "1", the output of the comparator 2 and the output of the comparator 3 output a low level "0" after passing through the exclusive OR gate of the logic unit. Then the variable interval [1, 2) where an input interval variable of the comparator 2 is located is the variable interval to be calculated.

Step S5, query the storage table according to the variable interval to be calculated, and obtain the parameters of the fitting quadratic function.

In this embodiment, the selection unit queries the storage table according to the variable interval to be calculated, so as to obtain the parameters of the fitting quadratic function. For example, the selection unit selects the parameters ( a ₂ , b ₂ , c ₂ ) of the fitted quadratic function corresponding to the variable interval [1,2). The parameters selected by the selection unit are then output to the connected calculation unit.

Step S6, completing the operation on the input variable according to the parameter.

The calculation unit substitutes the parameters ( a ₂ , b ₂ , c ₂ ) into the fitting quadratic function f ( x )= a ₂ x ² + b ₂ x + c ₂ , and completes the calculation for the input variable.

FIG. 7 is a functional module diagram in a preferred embodiment of the accelerated activation function computing system of the present invention.

In some embodiments, the acceleration activation function computing system 200 (hereinafter referred to as “computing system 200 ” for convenience of description) runs in the electronic device 1 . The computing system 200 may include a plurality of functional modules composed of program code segments. The program codes of each program segment in the operating system 200 can be stored in the memory and executed by at least one processor to realize the start-up process.

In this embodiment, the computing system 200 can be divided into a plurality of functional modules according to the functions performed by the computing system 200 . The functional modules may include: a receiving module 201 , a comparing module 202 , a processing module 203 , a determining module 204 and a querying module 205 . The module referred to in the present invention refers to a series of computer program segments that can be executed by at least one processor and can perform fixed functions, and are stored in a memory. In some embodiments, the function of each module will be described in detail in subsequent embodiments.

The receiving module 201 is used for receiving input variables.

In this embodiment, the input variable is the input variable of the excitation function.

The comparison module 202 is used for comparing the input variable and the variable interval of the activation function to obtain a comparison output result.

In this embodiment, the comparator in the matching unit compares the input variable with the variable range of the activation function to obtain the comparison output result. The comparator is used for matching the input variable of the activation function with the variable interval of the activation function, and determining the output value according to the magnitude relationship between them. When the input variable x is less than the maximum value x _N of the interval variable, the comparison unit outputs a low level "0"; when the input variable x is greater than or equal to the maximum value x _N of the interval variable, the comparison unit outputs a high level level "1".

For example, one input of the comparator 2 is used to receive the input variable of the start function, and the other input is used to receive the maximum value x ₂ of the interval variable [1,2), and x ₂ approaches 2. For example, when the input variable is 1.5, since 1.5 is greater than 1, the comparator 1 outputs a high level "1"; since 1.5 is less than 2, the comparator 2 outputs a low level "0", and similarly, since 1.5 is less than other interval variables the maximum value, so other comparators also output low level "0".

The processing module 203 is configured to perform a logical operation on the comparison output result to obtain a logical output result.

In this embodiment, a logic output result is obtained by performing a logic operation on the comparison output result by the logic unit in the matching unit. The logic unit includes an anti-gate, a plurality of XOR gates and a temporary storage unit. For example, as shown in FIG. 3 , the output of the comparator 1 is respectively connected to the inverse gate and the XOR gate 1 of the logic unit; the output of the comparator 2 is respectively connected to the XOR gate 1 and the XOR gate 2 The output of comparator 3 is connected with described XOR gate 2 and XOR gate 3 respectively; By analogy, the output of comparator N-2 is connected with described XOR gate N-3 and XOR gate N-2 respectively; The output of the comparator N-1 is respectively connected with the exclusive OR gate N-2 and the temporary storage unit.

For example, when the input variable is 1.5, the comparator 1 outputs a low level "0" after the reverse gate of the logic unit, while the output of the comparator 1 and the output of the comparator 2 output high after the exclusive OR gate processing of the logic unit Level "1", the output of the comparator 2 and the output of the comparator 3 output a low level "0" after passing through the exclusive OR gate of the logic unit, and so on, the output of the comparator N-2 and the output of the comparator N-1 The output outputs a low level "0" after passing through the exclusive OR gate of the logic unit.

The determining module 204 is used for determining the variable interval to be calculated according to the logical output result.

In this embodiment, when the logical output result is "1", the variable interval where the interval variable of one input of the corresponding comparator is located is the variable interval to be calculated.

For example, when the input variable is 1.5, the comparator 1 outputs a low level "0" after the reverse gate of the logic unit, while the output of the comparator 1 and the output of the comparator 2 output high after the exclusive OR gate processing of the logic unit When the level is "1", the output of the comparator 2 and the output of the comparator 3 output a low level "0" after passing through the exclusive OR gate of the logic unit. Then the variable interval [1, 2) where an input interval variable of the comparator 2 is located is the variable interval to be calculated.

The query module 205 queries the storage table according to the variable interval to be calculated, and obtains the parameters of the fitting quadratic function.

In this embodiment, the selection unit queries the storage table according to the variable interval to be calculated, so as to obtain the parameters of the fitting quadratic function. For example, the selection unit selects the parameters ( a ₂ , b ₂ , c ₂ ) of the fitted quadratic function corresponding to the variable interval [1,2). The parameters selected by the selection unit are then output to the connected calculation unit.

The processing module 203 is further configured to complete the operation on the input variable according to the parameter.

The calculation unit substitutes the parameters ( a ₂ , b ₂ , c ₂ ) into the fitting quadratic function ( x )= a ₂ x ² + b ₂ x + c ₂ , and completes the calculation for the input variable.

The above-mentioned integrated units implemented in the form of software function modules can be stored in a computer-readable storage medium. The above-mentioned software function module is stored in a storage medium, and includes several instructions to enable an electronic device (which may be a personal computer, a dual-screen device, or a network device, etc.) or a processor (processor) to execute various embodiments of the present invention part of the method.

FIG. 8 is a schematic diagram of an electronic device according to Embodiment 3 of the present invention.

The electronic device 1 includes: a memory 11, at least one processor 12, a computer-readable storage medium 13 stored in the memory 11 and executable on the at least one processor 12, and at least one communication bus 14.

When the at least one processor 12 executes the computer-readable storage medium 13, the steps in the foregoing computing method embodiments are implemented.

Exemplarily, the computer-readable storage medium 13 may be divided into one or more modules/units, the one or more modules/units are stored in the memory 11, and are stored in the at least one module/unit. A processor 12 executes to accomplish the present invention. The one or more modules/units may be a series of computer program instruction segments capable of performing specific functions, and the instruction segments are used to describe the execution process of the computer-readable storage medium 13 in the electronic device 1 .

The electronic device 1 may be a mobile phone, a tablet computer, a personal digital assistant (Personal Digital Assistant, PDA) and other devices installed with application programs. Those skilled in the art can understand that the schematic diagram 7 is only an example of the electronic device 1, and does not constitute a limitation on the electronic device 1, and may include more or less components than the one shown, or combine some components, or different For example, the electronic device 1 may also include input and output devices, network access devices, bus bars, and the like.

The at least one processor 12 may be a central processing unit (Central Processing Unit, CPU), and may also be other general-purpose processors, digital signal processors (Digital Signal Processor, DSP), application specific integrated circuits (Application Specific Integrated Circuit, ASIC), off-the-shelf programmable gate array (Field-Programmable Gate Array, FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The processor 12 can be a microprocessor or the processor 12 can also be any conventional processor, etc. The processor 12 is the control center of the electronic device 1, and uses various interfaces and lines to connect the entire electronic device. 1 parts.

The memory 11 can be used to store the computer-readable storage medium 13 and/or modules/units, and the processor 12 can run or execute computer programs and/or modules stored in the memory 11 /unit, and call the data stored in the memory 11 to realize various functions of the electronic device 1 . The memory 11 may mainly include a storage program area and a storage data area, wherein the storage program area may store an operating system, an application program required for at least one function (such as a sound playback function, an image playback function, etc.), etc.; storage data The area can store materials (such as audio materials, etc.) created according to the use of the electronic device 1, and the like. In addition, the memory 11 may include non-volatile memory, such as hard disk, memory, plug-in hard disk, Smart Media Card (SMC), Secure Digital (SD) card, flash memory A memory card (Flash Card), at least one disk memory device, flash memory device, or other non-volatile solid state memory device.

The memory 11 stores program codes, and the at least one processor 12 can call the program codes stored in the memory 11 to execute related functions. For example, each module (receiving module 201, comparing module 202, processing module 203, determining module 204, and querying module 205) described in FIG. 7 is a program code stored in the memory 11, and executed by the at least one processor 12, so as to realize the functions of the various modules to achieve the purpose of starting processing.

If the modules/units integrated in the electronic device 1 are implemented in the form of software functional units and sold or used as independent products, they may be stored in a computer-readable storage medium. Based on this understanding, the present invention implements all or part of the processes in the methods of the above embodiments, and can also be implemented by The computer program instructs the relevant hardware to complete, and the computer program can be stored in a computer-readable storage medium. When the computer program is executed by the processor, the steps of the above-mentioned method embodiments can be implemented. Wherein, the computer program includes computer program code, and the computer program code may be in the form of original code, object code, executable file, or some intermediate form. The computer-readable medium may include: any entity or device capable of carrying the computer program code, a recording medium, a USB flash drive, a removable hard disk, a magnetic disk, an optical disk, a computer memory, a read-only memory (ROM, Read-Only Memory); Only Memory) etc.

In the several embodiments provided by the present invention, it should be understood that the disclosed electronic devices and methods may be implemented in other manners. For example, the electronic device embodiments described above are only illustrative. For example, the division of the units is only a logical function division, and other division methods may be used in actual implementation.

In addition, each functional unit in each embodiment of the present invention may be integrated in the same processing unit, or each unit may exist physically alone, or two or more units may be integrated in the same unit. The above-mentioned integrated units can be implemented in the form of hardware, or can be implemented in the form of hardware plus software function modules.

It will be apparent to those skilled in the art that the present invention is not limited to the details of the above-described exemplary embodiments, but that the present invention may be embodied in other specific forms without departing from the spirit or essential characteristics of the invention. Therefore, the embodiments are to be regarded in all respects as illustrative and not restrictive, and the scope of the present invention is defined by the appended claims rather than the foregoing description, and is therefore intended to fall within the scope of the claims. All changes within the meaning and range of the equivalents of , are included in the present invention. Any reference sign in a claim should not be construed as limiting the claim to which it relates. Furthermore, it is clear that the word "comprising" does not exclude other units or, and the singular does not exclude the plural. A plurality of units or means stated in the system claim can also be implemented by one unit or means by software or hardware. The terms first, second, etc. are used to denote names and do not denote any particular order.

Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and not to limit them. Although the present invention has been described in detail with reference to the preferred embodiments, those of ordinary skill in the art should understand that the technical solutions of the present invention can be Modifications or equivalent substitutions can be made without departing from the technology of the present invention. the spiritual scope of the technical programme.

120: Matching unit

121: Comparator

122: Logic Unit

123:Select unit

Claims (9)

A device for accelerating the operation of a startup function, the device comprising: a temporary memory for storing a storage table, wherein the storage table describes the variable interval of the startup function and the parameters of the fitting quadratic function corresponding to the interval The mapping relationship; the matching unit includes a plurality of comparators, a logic unit and a selection unit, the plurality of comparators are connected to the selection unit through the logic unit, and the plurality of comparators are used for the activation function. The input variable is matched with the variable interval of the startup function to obtain a comparison output result, and the logic unit performs a logic operation according to the comparison output result to obtain a logic output result, and determines the to-be-calculated according to the logic output result. the variable interval; the selection unit is used to query the storage table according to the variable interval to be calculated, and obtain the parameters of the fitting quadratic function; the calculation unit is connected to the matching unit and is used for according to the The parameter completes the operation for the input variable. The device for accelerating the start-up function operation according to claim 1, wherein the logic unit comprises an anti-gate, a plurality of XOR gates and a temporary storage unit. The device for accelerating the operation of a starting function according to claim 1, wherein the parameters for fitting the quadratic function include quadratic term coefficients, linear term coefficients and constants. The apparatus for accelerating the operation of the activation function according to claim 3, wherein the calculation unit includes a multiplier and an adder, wherein the multiplier receives the quadratic term coefficient, the first-order term coefficient and the input from the selection unit The variable is multiplied; the adder receives the output result from the multiplier and the constant from the selection unit, and performs the addition. The device for accelerating the operation of an activation function according to claim 1, wherein the activation function includes a function

or f ( x ) = max (0, x ). A method for accelerating the operation of a startup function, the method comprising: receiving an input variable; comparing the input variable and the variable interval of the startup function to obtain a comparison output result; performing a logical operation on the comparison output result to obtain a logic output result; The variable interval to be calculated is determined according to the logical output result; the storage table is queried according to the variable interval to be calculated to obtain parameters for fitting a quadratic function; and the operation of the input variable is completed according to the parameters. The method for accelerating the operation of the activation function according to claim 6, wherein the storage table describes the variable interval of the activation function and the mapping relationship of the parameters of the fitting quadratic function corresponding to the interval. The method for accelerating the start-up function operation according to claim 7, wherein the parameters of the fitting quadratic function include quadratic term coefficients, linear term coefficients and constants. A computer-readable storage medium on which a computer program is stored, wherein, when the computer program is executed by a processor, the method for accelerating the operation of a startup function according to any one of claim 6 to claim 8 is implemented.

Images