CN111914987A - Data processing method and device based on neural network, equipment and readable medium - Google Patents

Abstract

The present disclosure provides a neural network-based data processing method, device, electronic device, and computer-readable medium, and relates to the field of data processing. The method includes: performing channel splitting processing on input data of the neural network, and splitting the channel The divided input data is subjected to channel shuffling processing; the input data after channel shuffling is subjected to binarization processing to obtain binary input data; the binary input data is subjected to group convolution processing, and the group convolution The weights in the corresponding convolution kernels are binary data; the input data after channel shuffling and the binary data after group convolution processing are accumulated according to the channel order to obtain the output result. The technical solutions provided by the embodiments of the present disclosure can improve the speed of data processing and reduce the loss of data information.

Description

Data processing method and device, device and readable medium based on neural network

technical field

The present disclosure relates to the field of data processing, and in particular, to a neural network-based data processing method and apparatus, electronic device, and computer-readable medium.

Background technique

In related technologies, the neural network in deep learning technology generally uses floating-point calculation to process the input data, but the floating-point calculation requires a large storage space and generates a huge amount of calculation, which seriously hinders the neural network in the mobile applications. Binarized neural network has become a popular research direction of deep learning in recent years due to its potential advantages of high model compression rate and fast calculation speed.

However, in the related art, the binary neural network has a strong information loss problem. Therefore, finding a method that can reduce the information loss of the binary neural network is extremely meaningful for the application of the binary neural network.

It should be noted that the information disclosed in the above Background section is only for enhancement of understanding of the background of the present disclosure, and therefore may contain information that does not form the prior art that is already known to a person of ordinary skill in the art.

SUMMARY OF THE INVENTION

In view of this, the present disclosure provides a neural network-based data processing method and apparatus, electronic device, and computer-readable medium, which can reduce the information loss generated in the binary neural network, and reduce the memory consumption to improve the calculation speed.

Other features and advantages of the present disclosure will become apparent from the following detailed description, or be learned in part by practice of the present disclosure.

According to a first aspect of the embodiments of the present disclosure, a data processing method based on a neural network is proposed, the method includes: performing channel splitting processing on input data of the neural network, and performing channel splitting on the input data after channel splitting. Channel shuffling processing; performing binarization processing on the input data after channel shuffling to obtain binary input data; performing grouping convolution processing on the binary input data, and the grouping convolution processing corresponding convolution kernels The weights in are binary data; the input data after channel shuffling and the binary data after group convolution processing are accumulated according to the channel order to obtain the output result.

In some embodiments, performing channel shuffling processing on the channel-split input data includes: performing random permutation and combination processing on channels of the channel-split input data.

In some embodiments, performing binarization processing on the channel-shuffled input data includes: if the channel-shuffled input data is greater than a preset threshold, converting the channel-shuffled input data into a first value; if the input data after channel shuffling is less than or equal to the preset threshold, convert the input data after channel shuffling into a second value.

In some embodiments, the first value is 1 and the second value is -1.

In some embodiments, performing grouped convolution processing on the binary input data includes: grouping the binary input data and the convolution kernels in the neural network according to channel correspondence; grouping the grouped binary input data The data and its corresponding convolution kernel are processed by binary operation.

In some embodiments, the binary operation processing includes: XOR operation, XOR operation or XOR operation.

In some embodiments, the convolution kernel is a 3×3 convolution kernel.

According to a second aspect of the embodiments of the present disclosure, a data processing apparatus based on a neural network is proposed, the apparatus includes: a channel shuffling module, configured to perform channel splitting processing on input data of the neural network, and split the channel The divided input data is subjected to channel shuffling processing; the binary processing module is configured to perform binary processing on the input data after channel shuffling to obtain binary input data; the grouping convolution module is configured to The value input data is subjected to grouped convolution processing, and the weights in the convolution kernels corresponding to the grouped convolution processing are binary data; the accumulation module is configured to perform channel shuffled input data and grouped convolution processing according to the channel order. The binary data is accumulated to obtain the output result.

According to a third aspect of the embodiments of the present disclosure, an electronic device is provided, the electronic device includes: one or more processors; and a storage device for storing one or more programs, when the one or more programs are stored The one or more processors execute, so that the one or more processors implement the neural network-based data processing method described in any one of the above.

According to a fourth aspect of the embodiments of the present disclosure, a computer-readable medium is provided on which a computer program is stored, characterized in that, when the program is executed by a processor, the neural network-based neural network as described in any of the above is implemented. data processing method.

According to the neural network-based data processing method, apparatus, electronic device, and computer-readable medium provided according to some embodiments of the present disclosure, on the one hand, the input data is processed by performing channel splitting, channel shuffling and other operations on the input data. The grouped convolution reduces the convolution parameters and improves the data processing speed; on the other hand, the input data is compressed by binarizing the input data, which further improves the data processing speed of the neural network; finally, The technical solutions provided by the embodiments of the present disclosure use only one binarization process, which reduces the information loss of the input data and improves the accuracy of the output results.

It is to be understood that the foregoing general description and the following detailed description are exemplary only and do not limit the present disclosure.

Description of drawings

The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the disclosure and together with the description serve to explain the principles of the disclosure. The drawings described below are only some embodiments of the present disclosure, and for those of ordinary skill in the art, other drawings can also be obtained from these drawings without creative efforts.

FIG. 1 shows a schematic diagram of an exemplary system architecture of a neural network-based data processing method or a neural network-based data processing apparatus applied to an embodiment of the present disclosure.

FIG. 2 is a schematic diagram of a ShuffleNet architecture according to the related art.

FIG. 3 is a schematic diagram of a ResNet architecture according to the related art.

FIG. 4 is a schematic diagram of a Bi-RealNet architecture according to the related art.

Fig. 5 is a flow chart of a data processing method based on a neural network according to an exemplary embodiment.

Fig. 6 is a flowchart of another data processing method based on a neural network according to an exemplary embodiment.

Fig. 7 is a flow chart of yet another method for data processing based on a neural network according to an exemplary embodiment.

Fig. 8 is a schematic diagram of a binary neural network according to an exemplary embodiment.

Fig. 9 is a block diagram of a data processing apparatus based on a neural network according to an exemplary embodiment.

Fig. 10 is a block diagram of another neural network-based data processing apparatus according to an exemplary embodiment.

Fig. 11 is a block diagram of yet another neural network-based data processing apparatus according to an exemplary embodiment.

FIG. 12 is a schematic structural diagram of a computer system applied to a neural network-based data processing apparatus according to an exemplary embodiment.

Detailed ways

Example embodiments will now be described more fully with reference to the accompanying drawings. Example embodiments, however, can be embodied in various forms and should not be construed as limited to the embodiments set forth herein; rather, these embodiments are provided so that this disclosure will be thorough and complete, and will fully convey the concept of example embodiments to those skilled in the art. The same reference numerals in the drawings denote the same or similar parts, and thus their repeated descriptions will be omitted.

The described features, structures, or characteristics may be combined in any suitable manner in one or more embodiments. In the following description, numerous specific details are provided in order to give a thorough understanding of the embodiments of the present disclosure. However, those skilled in the art will appreciate that the technical solutions of the present disclosure may be practiced without one or more of the specific details, or other methods, components, devices, steps, etc. may be employed. In other instances, well-known methods, devices, implementations, or operations have not been shown or described in detail to avoid obscuring aspects of the present disclosure.

The accompanying drawings are merely schematic illustrations of the present disclosure, and the same reference numerals in the drawings denote the same or similar parts, and thus their repeated descriptions will be omitted. Some of the block diagrams shown in the figures do not necessarily necessarily correspond to physically or logically separate entities. These functional entities may be implemented in software, or in one or more hardware modules or integrated circuits, or in different networks and/or processor devices and/or microcontroller devices.

The flowcharts shown in the figures are only exemplary illustrations, and do not necessarily include all contents and steps, nor do they have to be performed in the order described. For example, some steps can be decomposed, and some steps can be combined or partially combined, so the actual execution order may be changed according to the actual situation.

In this specification, the terms "a", "an", "the", "the" and "at least one" are used to indicate the presence of one or more elements/components/etc.; the terms "comprising", "including" and "Having" is used to indicate an open-ended inclusive meaning and to mean that there may be additional elements/components/etc. in addition to the listed elements/components/etc.; the terms "first", "secondary" " and "Third" etc. are used only as markers, not as restrictions on the number of their objects.

The exemplary embodiments of the present disclosure will be described in detail below with reference to the accompanying drawings.

FIG. 1 shows a schematic diagram of an exemplary system architecture of a neural network-based data processing method or a neural network-based data processing apparatus that can be applied to an embodiment of the present disclosure.

As shown in FIG. 1 , the system architecture 100 may include terminal devices 101 , 102 , and 103 , a network 104 and a server 105 . The network 104 is a medium used to provide a communication link between the terminal devices 101 , 102 , 103 and the server 105 . The network 104 may include various connection types, such as wired, wireless communication links, or fiber optic cables, among others.

The user can use the terminal devices 101, 102, 103 to interact with the server 105 through the network 104 to receive or send messages and the like. The terminal devices 101, 102, and 103 may be various electronic devices with display screens and supporting web browsing, including but not limited to smart phones, tablet computers, laptop computers, desktop computers, and the like.

The server 105 may be a server that provides various services, such as a background management server that provides support for devices operated by users using the terminal devices 101 , 102 , and 103 . The background management server can analyze and process the received request and other data, and feed back the processing result to the terminal device.

The server 105 may, for example, perform channel splitting processing on the input data of the neural network, and perform channel shuffling processing on the input data after channel splitting; perform binarization processing on the channel-shuffled input data to obtain two value input data; perform grouped convolution processing on the binary input data, and the weight in the convolution kernel corresponding to the grouped convolution processing is binary data; according to the channel order, the input data after channel shuffling is combined with the grouped volume Accumulate the processed binary data to obtain the output result.

It should be understood that the numbers of terminal devices, networks and servers in FIG. 1 are only illustrative, and the server 105 may be an entity server, or may be composed of multiple servers, and may have any number of terminal devices, network and server.

In the related art, floating-point (real-valued) neural networks have problems of high memory consumption and low calculation speed. Due to the large memory consumption of floating-point neural networks, they usually cannot be used on embedded devices or mobile devices with low memory. In order to solve the problem that the floating-point neural network occupies a large amount of memory, a binary neural network is usually used to replace the floating-point neural network to reduce the memory consumption of the neural network and improve the calculation speed of the neural network.

In the related art, the ShuffleNet architecture shown in FIG. 2 and the ResNet architecture shown in FIG. 3 are a kind of CNN network.

FIG. 2 is a schematic diagram of a ShuffleNet architecture according to the related art.

As shown in Figure 2, the ShuffleNet architecture includes a first binarization network 202, a first binary group convolution network 203 (using a 1×1 convolution kernel), a channel shuffling network 204, and a second binarization network 205. A binary depthwise separation convolution network 206 (with a convolution kernel of 3×3 size), a third binarization network 207, and a binary group convolution network 208 (with a convolution kernel of 1×1 size).

FIG. 3 is a schematic diagram of a ResNet architecture according to the related art.

As shown in FIG. 3, the ResNet network includes a fourth binarization network 302, a first binary convolution network 303 (using a convolution kernel of 1×1 size), a fifth binarization network 304, and a second binary group Convolutional network 305 (with a convolution kernel of 3×3 size), a sixth binarization network 307 and a second binary convolutional network 308 (with a convolution kernel of 1×1 size).

As shown in Figure 2 and Figure 3, it can be considered that both the ShuffleNet network and the ResNet network decompose an ordinary convolution into multiple small convolutions, and the data will be binarized once before each convolution, and each binarization will be performed. Processing will make the input data lose a lot of information. In this way, the information included in the data is greatly erased, which greatly reduces the ability of the final fitting.

In the embodiment of the present disclosure, in order to reduce the loss of information, the number of convolutions in the binary neural network can be reduced (ie, the number of times of binarizing the data) can be reduced, so as to reduce the data loss and improve the accuracy of data processing.

In the related art, a binary neural network architecture, that is, a Bi-RealNet (Binary-Real Network) architecture is also proposed.

FIG. 4 is a schematic diagram of a Bi-RealNet architecture according to the related art.

As shown in FIG. 4 , the Bi-RealNet architecture includes a binarization network 402 and a binary convolution network 403 (with a convolution kernel of 3×3 size). The binary convolution network 403 in the Bi-RealNet architecture shown in Figure 4 uses ordinary convolution, that is, the convolution operation is performed on the entire input data. Compared with the grouped convolution, the ordinary convolution Runs slower.

In the embodiment of the present disclosure, in order to overcome the problem of slow running speed of ordinary convolution, grouped convolution can be used to reduce the amount of calculation and improve the operation speed.

Fig. 5 is a flow chart of a data processing method based on a neural network according to an exemplary embodiment. The methods provided by the embodiments of the present disclosure can be processed by any electronic device with computing processing capabilities, such as the server 105 and/or the terminal devices 102 and 103 in the above-mentioned embodiment of FIG. 1 . In the following embodiments, the server 105 is used as the The execution body is taken as an example for illustration, but the present disclosure is not limited to this.

Referring to FIG. 5 , the neural network-based data processing provided by the embodiments of the present disclosure may include the following steps.

Step S501: Perform channel splitting processing on the input data of the neural network, and perform channel shuffling processing on the input data after channel splitting.

In some embodiments, the input data of the neural network may include multiple channels, for example, a piece of RGB color image data may include three channels of R, G, and B.

In some embodiments, multiple channels of the input data can be separated, for example, the RGB color image data can be split according to three channels of R, G, and B.

In some embodiments, in order to enhance the expressive ability of the neural network, channel shuffling processing may be performed on the input data after channel splitting, that is, random arrangement and combination processing may be performed on the channels of the input data after channel splitting. For example, three channels R, G, B of the RGB color image data after channel splitting are randomly arranged and combined to obtain a set of random channel combinations, such as G, R, B.

Step S502 , performing binarization processing on the input data after channel shuffling to obtain binary input data.

In some embodiments, performing binarization processing on the channel-shuffled input data may include: if the channel-shuffled input data is greater than a preset threshold, converting the channel-shuffled input data is a first value; if the input data after channel shuffling is less than or equal to the preset threshold, convert the input data after channel shuffling into a second value.

In some embodiments, the first value is 1 and the second value is -1. Those skilled in the art can understand that the two values of the binary data are not limited to 1 and -1, but may also be 1 and 0, or other binary data. In the embodiments of the present disclosure, the binary data is 1 and -1 as examples for description, but the present disclosure does not limit this.

In some embodiments, the preset threshold may be set to 10. If the input data after channel shuffling is greater than the preset threshold, convert the input data after channel shuffling to 1; if the input data after channel shuffling is less than or equal to the preset threshold, convert The channel shuffled input data is converted to -1.

Step S503 , performing grouped convolution processing on the binary input data, and the weights in the convolution kernels corresponding to the grouped convolution processing are binary data.

In some embodiments, performing grouped convolution processing on the binary input data may include: grouping the binary input data and the convolution kernels in the neural network according to channel correspondence; The input data and its corresponding convolution kernel are processed by binary operation.

Step S504: Accumulate the input data after channel shuffling and the binary data after packet convolution processing according to the channel sequence to obtain an output result.

The technical solutions provided by the embodiments of the present disclosure, on the one hand, realize grouped convolution of the input data by performing channel splitting, channel shuffling and other operations on the input data, which greatly reduces the convolution parameters and improves the data processing speed; On the other hand, by binarizing the input data to realize the compression of the input data, the speed of data processing by the neural network is further improved. The information loss of the input data is reduced, and the accuracy of the output results is improved.

In some embodiments, the binarization processing on the input data after channel shuffling in the embodiment shown in FIG. 5 may include the steps shown in FIG. 6 .

Step S5021, if the input data after channel shuffling is greater than a preset threshold, convert the input data after channel shuffling into a first value.

Step S5022, if the input data after channel shuffling is less than or equal to the preset threshold, convert the input data after channel shuffling into a second value.

In some embodiments, the first value is 1 and the second value is -1. Those skilled in the art can understand that the two values of the binary data are not limited to 1 and -1, but may also be 1 and 0, or other binary data. In the embodiments of the present disclosure, the binary data is 1 and -1 as examples for description, but the present disclosure does not limit this.

In some embodiments, the binary data network is a network that performs binarization processing on weights and activation values (eigenvalues), and the floating-point data network refers to those networks whose weights and activation values are both floating-point data. .

In some embodiments, a floating-point data occupies at least 32 bits (may also be 16 or 64 bits), and a binary data occupies only one bit. The binary data network model is compared with the floating-point data network model. The memory consumption can theoretically be reduced many times, so the binary data network has great advantages in model compression.

In addition, after binarizing the input data, activation function, and weight value of the neural network at the same time, the multiplication and addition of the original 32 floating-point numbers can be solved by an XOR operation (xnor) and an addition operation. Great potential for acceleration.

The technical solution provided by this embodiment binarizes the input data, which greatly reduces the memory consumption of the neural network and improves the running speed of the neural network.

In some embodiments, the grouped convolution processing on the binary input data in the embodiment shown in FIG. 5 may include the steps shown in FIG. 7 .

Referring to FIG. 7 , performing a grouped convolution process on the binary input data may include the following steps.

Step S5031, grouping the binary input data and the convolution kernels in the neural network according to channel correspondence.

In the related art, the general convolution will perform the convolution operation on the entire input data, that is, the input data: H ₁ ×W ₁ ×C ₁ ; and the size of the convolution kernel is h1 × w1, there are C ₂ in total, Then the output data obtained by convolution is H ₂ ×W ₂ ×C ₂ . Here we assume that the output and output resolution are constant. The above-mentioned convolution process is completed in one go, which puts forward higher requirements on the capacity of the memory.

In the related art, grouped convolution divides the input data into g (for example, 2 groups) groups. It should be noted that this grouping is divided according to channels, that is, certain channels are grouped into a group, and each group includes C ₁ /g channels (C ₁ represents the number of channels of input data, and C ₁ can be, for example, 256). Because the input data changes, correspondingly, the convolution kernel also needs to make the same change. That is, the number of channels of the convolution kernel in each group becomes C ₁ /g, and the size of the convolution kernel does not need to be changed. At this time, the number of convolution kernels in each group becomes C ₂ / g instead of the original C ₂ . Then, the convolution kernels of each group are convolved with the input data in their corresponding groups, and after the output data is obtained, they are combined in a connected manner, and the final output data channel is still C ₂ . That is to say, after the number of groups g is determined, then we will operate g identical convolution processes in parallel. In each process (each group), the input data is H ₁ ×W ₁ ×C ₁ /g, and the convolution kernel The size is h ₁ ×w ₁ ×C ₁ /g, there are C ₂ /g in total, and the output data is H ₂ ×W ₂ ×C ₂ /g.

In some embodiments, the input data and the convolution kernels in the neural network are correspondingly grouped in depth according to a preset number of groups, that is, the input data of certain channels are grouped into one group, and the volume of the corresponding point channel is grouped. The product and data are divided into corresponding groups.

Step S5032, perform binary operation processing on the grouped binary input data and its corresponding convolution kernel.

In some embodiments, multiple convolution processes may be performed in parallel (each group of input data is convolved and convolved with the group corresponding to that group, respectively).

In some embodiments, the floating-point data convolution process mainly involves multiplication and addition operations, but the binary data convolution process may include some logical operation processes in addition to the addition process, for example, may include: XOR operation , XOR operation or XOR operation, etc.

In some embodiments, performing binary operation processing on the grouped binary input data and its corresponding convolution kernel, the binary operation processing may include: XOR operation, XOR operation or XOR operation, etc. The binary operation processing is not limited, and the actual needs of those skilled in the art shall prevail.

The technical solution provided by this embodiment, by grouping the input data and the convolution kernels in the neural network, enables the neural network to process multiple convolutions in parallel to realize the processing of the input data, which greatly reduces the memory usage. consumption, increasing the speed of operation.

Fig. 8 is a schematic structural diagram of a neural network according to an exemplary embodiment.

The neural network shown in FIG. 8 may include a data input network 801 , a channel shuffling network 802 , a binary processing network 803 , a binary group convolution processing network 804 , an accumulation network 805 and a result output network 806 .

The data input network 801 can be configured to obtain input data of the neural network. In some embodiments, the acquired input data to the neural network includes multiple channels. For example, the input data of the neural network may be an RGB color image, and the color image data may include data of three channels of R, G, and B.

The channel shuffling network 802 may be configured to perform channel splitting processing on the input data of the neural network, and perform channel shuffling processing on the input data after channel splitting.

In some embodiments, multiple channels of the input data can be separated, for example, the RGB color image data can be split according to three channels of R, G, and B.

In some embodiments, in order to enhance the expressive ability of the neural network, channel shuffling processing may be performed on the input data after channel splitting, that is, random arrangement and combination processing may be performed on the channels of the input data after channel splitting. For example, three channels R, G, B of the RGB color image data after channel splitting are randomly arranged and combined to obtain a set of random channel combinations, such as G, R, B.

The binary processing network 803 may be configured to perform binary processing on the input data after channel shuffling to generate binary input data.

In some embodiments, performing binarization processing on the channel-shuffled input data includes: if the channel-shuffled input data is greater than a preset threshold, converting the channel-shuffled input data into a first value; if the input data after channel shuffling is less than or equal to the preset threshold, convert the input data after channel shuffling into a second value.

In some embodiments, the first value is 1 and the second value is -1. Those skilled in the art can understand that the two values of the binary data are not limited to 1 and -1, but may also be 1 and 0, or other binary data. In the embodiments of the present disclosure, the binary data is 1 and -1 as examples for description, but the present disclosure does not limit this.

The binary group convolution processing network 804 may be configured to perform binary group convolution processing on the binary input data, and the weights in the convolution kernels corresponding to the grouped convolution processing are binary data.

In some embodiments, performing grouped convolution processing on the binary input data includes: grouping the binary input data and the convolution kernels in the neural network according to channel correspondence; grouping the grouped binary input data The data and its corresponding convolution kernel are processed by binary operation.

In some embodiments, the convolution kernel in the neural network may be a 3×3 convolution kernel.

The accumulation network 805 may be configured to perform accumulation processing on the input data after channel shuffling and the binary data after convolution processing by binary grouping according to the channel data.

A result output network 806, which may be configured to output accumulated processing results.

The neural network provided by this embodiment reduces the memory consumption of the neural network by performing binary processing on the input data, and uses binary group convolution to improve the running speed of the neural network. On this basis, channel shuffling is also used. Operates to enable the exchange of information between different groups of channels. The technical solution provided by the present disclosure can not only reduce the memory consumption of the neural network, but also reduce the information loss.

Fig. 9 is a block diagram of a data processing apparatus based on a neural network according to an exemplary embodiment. Referring to FIG. 9 , the apparatus 900 includes a channel shuffling module 901 , a binary processing module 902 , a grouping convolution module 903 and an accumulation module 904 .

The channel shuffling module 901 may be configured to perform channel splitting processing on the input data of the neural network, and perform channel shuffling processing on the input data after channel splitting. The binary processing module 902 may be configured to perform binary processing on the channel-shuffled input data to obtain binary input data. The grouped convolution module 903 may be configured to perform grouped convolution processing on the binary input data, and the weights in the convolution kernels corresponding to the grouped convolution processing are binary data. The accumulation module 904 may be configured to perform accumulation processing on the input data after channel shuffling and the binary data after packet convolution processing according to the channel order to obtain an output result.

In some embodiments, the channel shuffling module 901 may be further configured to perform random permutation and combination processing on the channels of the input data after the channel splitting.

In some embodiments, the binary processing module 902 in the embodiment shown in FIG. 9 may include a first value determination unit 9021 and a second value determination unit 9022 as shown in FIG. 10 .

Wherein, the unit 9021 for determining the first value can be configured to convert the input data after channel shuffling into a first value if the input data after channel shuffling is greater than a preset threshold. The second value determining unit 9022 It may be configured that if the input data after channel shuffling is less than or equal to the preset threshold, the input data after channel shuffling is converted into a second value.

In some embodiments, the first value is 1 and the second value is -1.

In some embodiments, the grouped convolution module 903 shown in the embodiment of FIG. 9 may include a grouping unit 9031 and a binary operation unit 9032 as shown in FIG. 11 .

The grouping unit 9031 can be configured to group the binary input data and the convolution kernels in the neural network according to the corresponding channel; the binary operation unit 9032 can be configured to group the binary input data and its corresponding volume The product kernel performs binary operation processing.

In some embodiments, the binary operation processing includes: XOR operation, XOR operation or XOR operation.

Since each functional module of the neural network-based data processing apparatus 900 of the exemplary embodiment of the present disclosure corresponds to the steps of the above-mentioned exemplary embodiment of the neural network-based data processing method, details are not described herein again.

Referring to FIG. 12 below, it shows a schematic structural diagram of a computer system 1200 suitable for implementing the terminal device according to the embodiment of the present application. The terminal device shown in FIG. 12 is only an example, and should not impose any limitations on the functions and scope of use of the embodiments of the present application.

As shown in FIG. 12, a computer system 1200 includes a central processing unit (CPU) 1201, which can be loaded into a random access memory (RAM) 1203 according to a program stored in a read only memory (ROM) 1202 or a program from a storage section 1208 Instead, various appropriate actions and processes are performed. In the RAM 1203, various programs and data necessary for the operation of the system 1200 are also stored. The CPU 1201 , the ROM 1202 , and the RAM 1203 are connected to each other through a bus 1204 . An input/output (I/O) interface 1205 is also connected to bus 1204 .

The following components are connected to the I/O interface 1205: an input section 1206 including a keyboard, a mouse, etc.; an output section 1207 including a cathode ray tube (CRT), a liquid crystal display (LCD), etc., and a speaker, etc.; a storage section 1208 including a hard disk, etc. ; and a communication section 1209 including a network interface card such as a LAN card, a modem, and the like. The communication section 1209 performs communication processing via a network such as the Internet. Drivers 1210 are also connected to I/O interface 1205 as needed. A removable medium 1211, such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, etc., is mounted on the drive 1210 as needed so that a computer program read therefrom is installed into the storage section 1208 as needed.

In particular, according to embodiments of the present disclosure, the processes described above with reference to the flowcharts may be implemented as computer software programs. For example, embodiments of the present disclosure include a computer program product comprising a computer program carried on a computer-readable medium, the computer program containing program code for performing the method illustrated in the flowchart. In such an embodiment, the computer program may be downloaded and installed from the network via the communication portion 1209, and/or installed from the removable medium 1211. When the computer program is executed by the central processing unit (CPU) 1201, the above-described functions defined in the system of the present application are executed.

It should be noted that the computer-readable medium shown in this application may be a computer-readable signal medium or a computer-readable storage medium, or any combination of the above two. The computer-readable storage medium can be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus or device, or a combination of any of the above. More specific examples of computer readable storage media may include, but are not limited to, electrical connections with one or more wires, portable computer disks, hard disks, random access memory (RAM), read only memory (ROM), erasable Programmable read only memory (EPROM or flash memory), fiber optics, portable compact disk read only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing. In this application, a computer-readable storage medium can be any tangible medium that contains or stores a program that can be used by or in conjunction with an instruction execution system, apparatus, or device. In this application, however, a computer-readable signal medium may include a data signal propagated in baseband or as part of a carrier wave, carrying computer-readable program code therein. Such propagated data signals may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the foregoing. A computer-readable signal medium can also be any computer-readable medium other than a computer-readable storage medium that can transmit, propagate, or transport the program for use by or in connection with the instruction execution system, apparatus, or device . Program code embodied on a computer readable medium may be transmitted using any suitable medium including, but not limited to, wireless, wireline, optical fiber cable, RF, etc., or any suitable combination of the foregoing.

The flowchart and block diagrams in the Figures illustrate the architecture, functionality, and operation of possible implementations of systems, methods and computer program products according to various embodiments of the present application. In this regard, each block in the flowchart or block diagrams may represent a module, segment, or portion of code that contains one or more logical functions for implementing the specified functions executable instructions. It should also be noted that, in some alternative implementations, the functions noted in the blocks may occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently, or the blocks may sometimes be executed in the reverse order, depending upon the functionality involved. It is also noted that each block of the block diagrams or flowchart illustrations, and combinations of blocks in the block diagrams or flowchart illustrations, can be implemented in special purpose hardware-based systems that perform the specified functions or operations, or can be implemented using A combination of dedicated hardware and computer instructions is implemented.

The units involved in the embodiments of the present application may be implemented in a software manner, and may also be implemented in a hardware manner. The described unit may also be provided in a processor, for example, it may be described as: a processor includes a sending unit, an obtaining unit, a determining unit and a first processing unit. Among them, the names of these units do not constitute a limitation on the unit itself under certain circumstances.

As another aspect, the present application also provides a computer-readable medium. The computer-readable medium may be included in the device described in the above embodiments, or may exist alone without being assembled into the device. The above-mentioned computer-readable medium carries one or more programs, and when the above-mentioned one or more programs are executed by a device, the device can realize functions including: performing channel splitting processing on the input data of the neural network, and Perform channel shuffling processing on the input data after channel splitting; perform binarization processing on the input data after channel shuffling to obtain binary input data; perform packet convolution processing on the binary input data, the The weight in the convolution kernel corresponding to the grouped convolution processing is binary data; the input data after channel shuffling and the binary data after grouped convolution processing are accumulated according to the channel order to obtain the output result.

From the description of the above embodiments, those skilled in the art can easily understand that the exemplary embodiments described herein may be implemented by software, or may be implemented by software combined with necessary hardware. Therefore, the technical solutions of the embodiments of the present disclosure may be embodied in the form of software products, and the software products may be stored in a non-volatile storage medium (which may be CD-ROM, U disk, mobile hard disk, etc.), including several instructions It is used to cause a computing device (which may be a personal computer, a server, a mobile terminal, or a smart device, etc.) to execute the method according to the embodiment of the present disclosure, such as one or more steps shown in FIG. 5 .

Furthermore, the above-mentioned figures are merely schematic illustrations of the processes included in the methods according to the exemplary embodiments of the present disclosure, and are not intended to be limiting. It is easy to understand that the processes shown in the above figures do not indicate or limit the chronological order of these processes. In addition, it is also readily understood that these processes may be performed synchronously or asynchronously, for example, in multiple modules.

Other embodiments of the present disclosure will readily occur to those skilled in the art upon consideration of the specification and practice of the disclosure disclosed herein. This disclosure is intended to cover any variations, uses, or adaptations of this disclosure that follow the general principles of this disclosure and include common general knowledge or techniques in the technical field to which this disclosure is not claimed . The specification and examples are to be regarded as exemplary only, with the true scope and spirit of the disclosure being indicated by the claims.

It should be understood that the present disclosure is not limited to the detailed structures, drawings, or implementations shown herein, but on the contrary, the present disclosure is intended to cover various modifications and equivalent arrangements included within the spirit and scope of the appended claims .

Claims (10)

1. A data processing method based on a neural network is characterized by comprising the following steps:

carrying out channel splitting processing on input data of the neural network, and carrying out channel shuffling processing on the input data after channel splitting;

carrying out binarization processing on the input data after channel mixing to obtain binary input data;

performing packet convolution processing on the binary input data, wherein the weight in a convolution kernel corresponding to the packet convolution processing is binary data;

and accumulating the input data after channel mixing and the binary data after the grouping convolution processing according to the channel sequence to obtain an output result.

2. The method of claim 1, wherein performing channel shuffling on the channel split input data comprises:

and carrying out random permutation and combination processing on the channels of the input data after the channels are split.

3. The method according to claim 1, wherein the binarization processing is performed on the input data after the channel mixing, and comprises:

if the input data after the channel mixing is larger than a preset threshold value, converting the input data after the channel mixing into a first value;

and if the input data after the channel mixing is smaller than or equal to the preset threshold, converting the input data after the channel mixing into a second value.

4. The method of claim 3, wherein the first value is 1 and the second value is-1.

5. The method of claim 1, wherein performing packet convolution processing on the binary input data comprises:

correspondingly grouping the binary input data and convolution kernels in the neural network according to channels;

and performing binary operation processing on the grouped binary input data and the corresponding convolution kernel.

6. The method according to claim 5, wherein the binary operation processing includes: an exclusive nor operation, an exclusive or operation, or an exclusive nor operation.

7. The method of claim 5, wherein the convolution kernel is a 3 x 3 convolution kernel.

8. A data processing apparatus based on a neural network, comprising:

the channel shuffling module is configured to perform channel splitting processing on input data of the neural network and perform channel shuffling processing on the input data after channel splitting;

the binary processing module is configured to carry out binary processing on the input data after the channel mixing so as to obtain binary input data;

the grouping convolution module is configured to perform grouping convolution processing on the binary input data, and the weight in a convolution kernel corresponding to the grouping convolution processing is binary data;

and the accumulation module is configured to perform accumulation processing on the input data after channel mixing and the binary data after packet convolution processing according to the channel sequence so as to obtain an output result.

9. An electronic device, comprising:

one or more processors;

a storage device for storing one or more programs,

when executed by the one or more processors, cause the one or more processors to implement the method of any one of claims 1-7.

10. A computer-readable medium, on which a computer program is stored, which program, when being executed by a processor, is adapted to carry out the method of any one of claims 1-7.

Images