CN118249817B - Decoding method and device, electronic equipment and computer readable storage medium - Google Patents

Abstract

The disclosed embodiment provides a decoding method and device, an electronic device, and a computer-readable storage medium. The decoding method includes: obtaining multiple coding sequences based on Huffman coding; multiple coding sequences are obtained by encoding multiple segments after segmenting the data to be encoded; multiple coding sequences are loaded onto a chip; a decoding model on the chip is used to perform on-chip parallel decoding on multiple coding sequences, and multiple codes within each coding sequence are decoded in parallel; the decoding model is a model with parallel decoding capability obtained by training or fitting multiple coding sequence samples; multiple coding sequence samples are obtained by encoding multiple segments after segmenting the data samples to be encoded. Through this embodiment scheme, parallel decoding of Huffman coding sequences is realized, thereby reducing decoding delay and improving decoding speed in scenarios such as large model bandwidth compression.

Description

Decoding method and device, electronic device, and computer-readable storage medium

Technical Field

The embodiments of the present disclosure relate to the field of decoding technology, and in particular to a decoding method and device, an electronic device, and a computer-readable storage medium.

Background Art

Huffman coding is an entropy coding method that determines a variable-length coding table based on the statistical characteristics of the data to achieve lossless data compression. Huffman coding is widely used in computer network transmission of files such as pictures to reduce bandwidth requirements.

However, Huffman decoding is generally serial, and this decoding method is generally difficult to adapt to vector matrix computing units that are widely equipped with GPUs (Graphical Processing Units) and brain-like chips. Therefore, the decoding scheme of Huffman coding will incur a large additional time cost, which will offset part of the performance advantages brought by the compressed bandwidth.

Summary of the invention

The embodiments of the present disclosure provide a decoding method and device, an electronic device, and a computer-readable storage medium.

In a first aspect, an embodiment of the present disclosure provides a decoding method, the decoding method comprising:

Acquire multiple coding sequences based on Huffman coding; the multiple coding sequences are obtained by segmenting the data to be encoded and encoding the multiple segments respectively;

loading the plurality of coding sequences onto a chip;

Using the decoding model on the chip, the multiple coding sequences are decoded in parallel on the chip, and the multiple codes within each coding sequence are decoded in parallel;

The decoding model is a model with parallel decoding capability obtained by training or fitting based on multiple coding sequence samples; the multiple coding sequence samples are obtained by segmenting the data samples to be encoded and then encoding the multiple segments separately.

In a second aspect, the present disclosure provides a decoding device, the decoding device comprising:

An acquisition module, used for acquiring multiple coding sequences based on Huffman coding; the multiple coding sequences are obtained by segmenting the data to be encoded and then encoding the multiple segments respectively;

A loading module, used for loading the plurality of coding sequences onto a chip;

A decoding module, used to use the decoding model on the chip to perform on-chip parallel decoding on the multiple coding sequences, and to perform parallel decoding on multiple codes within each coding sequence;

The decoding model is a model with parallel decoding capability obtained by training or fitting based on multiple coding sequence samples; the multiple coding sequence samples are obtained by segmenting the data samples to be encoded and then encoding the multiple segments separately.

In a third aspect, the present disclosure provides an electronic device, including:

multiple processing cores; and

An on-chip network is configured to exchange data between the multiple processing cores and external data; wherein one or more instructions are stored in one or more of the processing cores, and one or more of the instructions are executed by one or more of the processing cores, so that one or more of the processing cores can execute the decoding method.

In a fourth aspect, the present disclosure provides a computer-readable storage medium having a computer program stored thereon, wherein the computer program implements the above-mentioned decoding method when executed by a processor/processing core.

The embodiments provided by the present disclosure use a decoding model on a chip to perform on-chip parallel decoding on multiple coding sequences obtained through Huffman coding, and to perform parallel decoding on multiple codes within each coding sequence, thereby enabling parallel decoding of Huffman coding sequences, thereby reducing decoding delay and improving decoding speed in scenarios such as large model bandwidth compression.

It should be understood that the content described in this section is not intended to identify the key or important features of the embodiments of the present disclosure, nor is it intended to limit the scope of the present disclosure. Other features of the present disclosure will become easily understood through the following description.

BRIEF DESCRIPTION OF THE DRAWINGS

The accompanying drawings are used to provide a further understanding of the present disclosure and constitute a part of the specification. Together with the embodiments of the present disclosure, they are used to explain the present disclosure and do not constitute a limitation of the present disclosure. The above and other features and advantages will become more apparent to those skilled in the art by describing detailed example embodiments with reference to the accompanying drawings. In the accompanying drawings:

FIG1 is a flowchart of a decoding method provided by an embodiment of the present disclosure;

FIG2 is a schematic diagram of a decoding method provided by an embodiment of the present disclosure;

FIG3 is a block diagram of a decoding device provided by an embodiment of the present disclosure;

FIG. 4 is a block diagram of an electronic device provided in an embodiment of the present disclosure.

DETAILED DESCRIPTION

In order to enable those skilled in the art to better understand the technical solutions of the present disclosure, the exemplary embodiments of the present disclosure are described below in conjunction with the accompanying drawings, including various details of the embodiments of the present disclosure to facilitate understanding, which should be considered as merely exemplary. Therefore, it should be recognized by those of ordinary skill in the art that various changes and modifications can be made to the embodiments described herein without departing from the scope and spirit of the present disclosure. Similarly, for the sake of clarity and conciseness, the description of well-known functions and structures is omitted in the following description.

In the absence of conflict, the various embodiments of the present disclosure and the various features therein may be combined with each other.

As used herein, the term "and/or" includes any and all combinations of one or more of the associated listed items.

The terms used herein are only used to describe specific embodiments and are not intended to limit the present disclosure. As used herein, the singular forms "a" and "the" are also intended to include plural forms, unless the context clearly indicates otherwise. It will also be understood that when the terms "including" and/or "made of" are used in this specification, the presence of the features, wholes, steps, operations, elements and/or components is specified, but the presence or addition of one or more other features, wholes, steps, operations, elements, components and/or groups thereof is not excluded. "Connected" or "connected" and similar words are not limited to physical or mechanical connections, but can include electrical connections, whether direct or indirect.

Unless otherwise defined, all terms (including technical and scientific terms) used herein have the same meaning as commonly understood by those of ordinary skill in the art. It will also be understood that terms such as those defined in commonly used dictionaries should be interpreted as having a meaning consistent with their meaning in the context of the relevant art and the present disclosure, and will not be interpreted as having an idealized or overly formal meaning unless explicitly defined as such herein.

When deploying a large language model on a chip, especially on an edge chip for inference, the token generation process is mainly serial. This means that every time a token is generated, the entire model weight and other parameters need to be loaded from the memory (video memory) to the chip once. As a result, when a token is generated serially, the memory data transmission bandwidth is much larger than the amount of computation. Therefore, the memory transmission bandwidth is the main bottleneck for improving the inference speed.

In addition, Huffman coding can be used to compress neural network weights, thereby reducing bandwidth requirements. For example, the information content of chatglm3 weights accounts for about 72% of its storage capacity, so there is potential for compression.

However, Huffman decoding is generally serial, and the decoding method is generally difficult to adapt to the vector-matrix multiplication computing units that are widely equipped in GPUs (Graphical Processing Units) and brain-like chips. Therefore, the on-chip decoding solution of Huffman coding will incur a large additional time cost, which will offset part of the performance advantages brought by the compressed bandwidth.

At present, the basic Huffman decoding method is to compare the encoded data bit by bit according to the binary tree constructed according to the encoding table to complete serial decoding. Its main disadvantage is that it is difficult to parallelize, resulting in a long time to decode long sequences.

The main disadvantage of the current Huffman decoding is that it is mainly serial and difficult to parallelize. In addition, the decoding method is difficult to adapt to brain-like computing chips or GPUs equipped with large-scale vector-matrix multiplications, making it difficult to use in scenarios such as reducing bandwidth for large model inference.

The embodiments provided by the present disclosure use a decoding model on a chip to perform on-chip parallel decoding on multiple coding sequences obtained through Huffman coding, and perform parallel decoding on multiple codes within each coding sequence, thereby realizing parallel decoding of Huffman coding sequences, thereby improving decoding efficiency, reducing decoding delay, and increasing decoding speed in scenarios such as large model bandwidth compression.

The decoding method according to the embodiment of the present disclosure can be executed by an electronic device such as a terminal device or a server. The terminal device can be a vehicle-mounted device, a user equipment (User Equipment, UE), a mobile device, a user terminal, a terminal, a cellular phone, a cordless phone, a personal digital assistant (Personal Digital Assistant, PDA), a handheld device, a computing device, a vehicle-mounted device, a wearable device, etc. The method can be implemented by a processor calling a computer-readable program instruction stored in a memory, or the method can be executed by a server.

FIG1 is a flowchart of a decoding method provided by an embodiment of the present disclosure. Referring to FIG1 , the method includes steps S11-S13:

S11. Acquire multiple coding sequences based on Huffman coding; the multiple coding sequences are obtained by segmenting the data to be encoded and then encoding the multiple segments respectively.

A Huffman coding table of the data is constructed in advance according to the statistical characteristics of all the data to be encoded and transmitted, including: calculating the number of occurrences (probability) of each character in the data, adding the two smallest number of occurrences (probabilities), constructing a binary tree, and using it as the left and right subtrees, repeating this process until the probability value is 1, specifying the left side of each binary tree as 0 and the right side as 1, and sequentially counting the 0s and 1s specified in the path from the top of the binary tree to each character as the Huffman code corresponding to the character, mapping all characters and their corresponding Huffman codes into a table, and obtaining a Huffman coding table.

In the embodiment of the present disclosure, the data to be encoded may be large model weight data; before obtaining a plurality of encoding sequences based on Huffman coding, the method may further include:

The large model weight data of each row in the large model weight matrix is respectively used as a data segment to be encoded, and each data segment to be encoded is Huffman encoded to obtain a coding sequence; or,

The large model weight data in each column of the large model weight matrix is respectively used as a data segment to be encoded, and Huffman encoding is performed on each data segment to be encoded to obtain a coding sequence.

In an embodiment of the present disclosure, as shown in FIG2 , segmented encoding may be performed on the data to be encoded and transmitted. For example, for the data in a weight matrix, each row of data may be taken as a data segment to be encoded, or each column of data may be taken as a data segment to be encoded. Huffman encoding is performed on each data segment to obtain a coding sequence. Each coding sequence is a sequence composed of 0s and/or 1s, which may be referred to as a 0/1 sequence.

In the embodiment of the present disclosure, before loading the multiple coding sequences onto the chip, the method may further include:

Obtaining a first coding sequence with the longest length among multiple coding sequences;

According to the length of the first coding sequence, the second coding sequences other than the first coding sequence in the multiple coding sequences are padded with bits.

In the disclosed embodiment, the lengths of the multiple 0/1 sequences obtained after encoding may be padded according to the 0/1 sequence with the longest length among all 0/1 sequences.

In the embodiment of the present disclosure, according to the length of the first coding sequence, the second coding sequence other than the first coding sequence in the plurality of coding sequences is padded with bits, including:

The second coding sequence is supplemented with 0 and/or 1 according to the length of the first coding sequence.

In the disclosed embodiment, when padding the length, the last few bits of the 0/1 sequence may be padded with all 0s, or all 1s, or a combination of 0 and 1. The detailed padding method is not limited here and can be defined according to the needs. When padding, the padded bits can be clearly distinguished from the original 0/1 sequence, so that the decoding model can easily identify the original 0/1 sequence to be decoded.

In the embodiment of the present disclosure, before acquiring multiple coding sequences based on Huffman coding, the method may further include:

When the data to be encoded is segmented, the difference in the statistical characteristics of each data segment to be encoded is made to be within a preset difference range.

In the embodiments of the present disclosure, when length padding is performed, in order to minimize the number of padding 0s to reduce invalid transmission, the statistical characteristics of each data segment should be made close as much as possible when the data is segmented (for example, the statistical characteristics of each row (column) of the weight matrix should be close), for example, the data distribution of each segment of data should be close.

In the embodiments of the present disclosure, the difference range can be defined according to different application scenarios and is not limited here.

In the embodiment of the present disclosure, for example, when segmenting, statistics are obtained that the large model weight matrix contains data such as 3, 5, 8, and 9, among which the number of 3 and 8 is large, and the number of 5 and 9 is small. Moreover, if segmentation is performed according to each row of the large model weight matrix, the proportion of 3 and 8 in each row of the large model weight data will be more than 95%, and the proportion of 5 and 9 will be less than 5%. For example, the proportion of 3 and 8 in the first row of the large model weight data is 97%, the proportion of 3 and 8 in the second row of the large model weight data is 96%, and the proportion of 3 and 8 in the third row of the large model weight data is 98%. Therefore, the difference range can be determined as 1%-3%. At this time, when the data in the large model weight matrix is segmented by row, the statistical characteristic difference of each data segment to be encoded is within the preset difference range. On the contrary, if the large model weight matrix is segmented according to each column, the proportion of 3 and 8 in the large model weight data of some columns will be about 90%, and the proportion of 5 and 9 will be about 10%. The large model weight data of some columns will have a proportion of 3 and 8 and a proportion of 5 and 9 of about 50%. Therefore, the statistical characteristics of each column are too different, and the length difference between each data segment is too large, which is not conducive to length filling. Therefore, avoid segmenting according to each column of the large model weight matrix.

S12, loading multiple coding sequences onto the chip.

In the embodiment of the present disclosure, based on the on-chip decoding of the decoding model, multiple coding sequences obtained after segmented coding can be loaded onto the chip and decoded in parallel using the decoding model.

S13. Using a decoding model on the chip, perform on-chip parallel decoding on multiple coding sequences, and perform parallel decoding on multiple codes within each coding sequence; wherein the decoding model is a model with parallel decoding capability obtained by training or fitting based on multiple coding sequence samples; and the multiple coding sequence samples are obtained by segmenting the data samples to be encoded and then encoding the multiple segments separately.

In the embodiments of the present disclosure, the decoding model may include but is not limited to a neural network, which can be used to perform any one of image processing tasks, speech processing tasks, text processing tasks, and video processing tasks.

In the disclosed embodiment, the decoding model can be any model that can perform parallel decoding calculations, data-driven training, and has a small number of parameters. Data-driven refers to the collection of massive amounts of data through the Internet or other related software, organizing the data into itinerary information, and then integrating and refining the relevant information. On the basis of the data, an automated decision model is formed through training and fitting. In short, it is to make decisions and actions based on data. Among them, in the fields of statistics and machine learning, fitting refers to finding suitable model parameters or function forms so that the model can better describe or predict the characteristics and trends of the data on a given data set. Fitting can be performed by different methods, such as least squares, maximum likelihood estimation, or gradient descent. In the fitting process, different models or function forms are usually tried, and the difference between the model and the data is minimized by adjusting the parameters. The goal of fitting is to find a model that performs well on the entire data set and can be generalized to new data.

In the embodiment of the present disclosure, when the decoding model includes a neural network, a decoding model on a chip is used to perform on-chip parallel decoding on multiple coding sequences, and to perform parallel decoding on multiple codes within each coding sequence, which may include:

Multiple encoded sequences are input into one or more neural networks for parallel decoding.

In the embodiments of the present disclosure, the neural network may include but is not limited to a convolutional neural network, and any other neural network capable of implementing a decoding function may be included in the protection scope of the neural network of the present disclosure.

In the embodiment of the present disclosure, when the neural network is a one-dimensional neural network, the decoding model may include multiple one-dimensional neural networks; inputting multiple encoding sequences into one or more neural networks for parallel decoding includes:

Input multiple encoding sequences into multiple one-dimensional neural networks respectively;

Each one-dimensional neural network decodes a coding sequence respectively, and multiple codes in the corresponding coding sequence are decoded in parallel within each one-dimensional neural network, so as to realize parallel decoding of multiple coding sequences.

In the embodiment of the present disclosure, each coding sequence is one-dimensional data, and a one-dimensional neural network can only process one one-dimensional data in the same time period. Therefore, for multiple coding sequences, multiple one-dimensional neural networks can be used to process them simultaneously, thereby achieving the purpose of parallel decoding.

In the embodiments of the present disclosure, since many neural networks have good parallel computing performance, such as one-dimensional convolutional networks, multiple codes within the corresponding coding sequence can be decoded in parallel within each one-dimensional neural network, thereby realizing a dual parallel decoding scheme of parallel decoding of multiple coding sequences and parallel decoding of multiple codes within each coding sequence, thereby improving decoding efficiency.

In the embodiment of the present disclosure, when the neural network is a multi-dimensional neural network, multiple encoding sequences are input into one or more neural networks for parallel decoding, including:

Input multiple encoding sequences into a multidimensional neural network;

Multiple coding sequences are decoded in parallel through a multi-dimensional neural network, and multiple codes within the corresponding coding sequence are decoded in parallel within each dimensional neural network.

In the disclosed embodiment, one-dimensional coding sequences in multiple coding sequences can be decoded by one-dimensional neural networks in the multidimensional neural network, respectively, to achieve parallel decoding of multiple coding sequences by the multidimensional neural network. Each dimensional neural network can also have parallel decoding capabilities. For example, each dimensional neural network can have a structure similar to a one-dimensional convolutional network, to achieve parallel decoding of multiple codes within the corresponding coding sequence within each dimensional neural network.

In the embodiments of the present disclosure, the neural network may include but is not limited to a convolutional neural network. For example, it may be a one-dimensional convolutional neural network. Multiple encoding sequences may be decoded in parallel by multiple identical one-dimensional convolutional neural networks.

In the disclosed embodiment, the advantages of using a one-dimensional convolutional neural network for decoding include:

1. Due to weight sharing, the one-dimensional convolutional neural network has fewer parameters, resulting in less additional transmission cost.

2. Multiple one-dimensional convolutional neural networks can be calculated in parallel, so that parallel decoding can be completed, and the convolution operation is highly optimized on brain-like chips or GPUs, so it is highly efficient.

3. In the large-model bandwidth compression scenario, the amount of computation brought about will not result in a significant cost, because the computational cost of generating a token is relatively low (hundreds of GFLOPS (Giga Floating-point Operations Per Second), and the chip generally has tens of TFLOPS (Floating-point operations per second), and the on-chip computing resources can complete these decoding calculations.

4. The design of neural networks needs to deal with the problem of coding alignment. Since the statistical characteristics of the coding stream (i.e., coding sequence) are basically uniform, the coding stream input to the network and the output decoding result are roughly corresponding in position. Therefore, this characteristic of the Huffman coding sequence is suitable for processing by neural networks, especially convolutional neural networks.

In an embodiment of the present disclosure, on the basis that the decoding model includes a neural network, the neural network may include but is not limited to a residual connection layer and/or a batch normalization layer to achieve computational optimization of the neural network.

When an input x is forcibly added to the output F(x) of the function, although G(x) can still be used to describe the relationship between the input and the output, this G(x) can be clearly split into the linear superposition of F(x) and x. This is the idea of the skip connect, which expresses the output as the linear superposition of the input and a nonlinear transformation of the input. The role of the skip connect is to solve the gradient vanishing and gradient exploding problems, and can also help the model converge faster. Residual connections are usually used in neural networks with multiple layers, such as residual networks (ResNet) and deformable convolutional networks (DenseNet).

Batch Normalization (BN) is an algorithm designed to overcome the difficulty of training caused by the deepening of the number of neural network layers. When the distribution of the sample data of the training set is inconsistent with that of the target sample set, the trained model cannot generalize well. When the number of neural network layers increases during training, the overall distribution of the input value of the activation function gradually approaches the upper and lower limits of the activation function's value range, resulting in the disappearance of the gradient of the shallow neural network during back propagation. The role of batch normalization is to pull the increasingly biased distribution back to the standardized distribution through normalization, so that the input value of the activation function falls in the area where the activation function is more sensitive to the input, thereby increasing the gradient, accelerating the learning convergence speed, and avoiding the problem of gradient disappearance.

In the embodiment of the present disclosure, when the decoding model includes a neural network, before using the decoding model on the chip to perform on-chip parallel decoding on multiple coding sequences and performing parallel decoding on multiple codes within each coding sequence, the method may further include:

Use multiple encoding sequences as training data to train the neural network;

When the loss value of the neural network meets the preset conditions, the decoding model is obtained.

In the disclosed embodiment, in order to ensure that the neural network completes the correct decoding function, the neural network needs to be trained. For example, a training set can be obtained based on all the data to be compressed for training, that is, a coding sequence is obtained after encoding the current data to be compressed, and these coding sequences are used as training data, and the neural network does not need to have the ability to generalize other data, because only the group of data (such as the weight data of a network) needs to be decoded. The neural network can also be trained with other data so that the neural network has the ability to generalize other data.

In an embodiment of the present disclosure, the data sample to be encoded may include a large model weight data sample; the multiple encoding sequence samples may include multiple encoding sequences obtained by segmenting the large model weight data sample and then performing Huffman encoding on each segment.

In the embodiments of the present disclosure, a neural network is trained based on multiple coding sequences obtained from large model weight data samples to obtain a decoding model capable of decoding the large model weight data in parallel. Based on the decoding model, in a large model weight data compression scenario, the large model weight data in the large model weight matrix can be segmented to obtain multiple coding sequences, and the multiple coding sequences can be input into the trained decoding model to quickly obtain decoding results, thereby accelerating the data decoding speed in the large model weight data compression scenario.

In the disclosed embodiment, since the neural network has a strong fitting ability, a higher decoding accuracy can be obtained. Especially in the compression scenario of the neural network weights such as large models, since the network is robust to the error of the weight value, it is not necessary to guarantee 100% decoding accuracy, so it is suitable to use a neural network approximation solution.

In the embodiment of the present disclosure, the original weight matrix in FIG. 2 is taken as an example to explain the embodiment of the present disclosure in detail. The original weight matrix contains multiple rows of data: the first row, -8, -2, 4, 3, -9; the second row, 1, 5, -3, -1, 7; the third row, -2, -9, 5, 0, -1; the fourth row, -8, -4, 6, 7, -4; the above data can be segmented by row, each row of data as a data segment, and Huffman encoding is performed on each data segment. For example, Huffman encoding of the data segment -8, -2, 4, 3, -9 consisting of the first row of data can obtain the encoding sequence: 01001110. Following the length padding principle, 0 can be aligned and padded after the code sequence 01001110 to obtain the padded encoding sequence 0100111000. The padded encoding sequence 0100111000 is loaded into the chip and decoded by the neural network on the chip to restore the original weight data: -8, -2, 4, 3, -9. The above operation can be performed for each row of data in the original weight matrix (which can be regarded as a data segment respectively), and after obtaining the Huffman-coded coding sequence of each data segment, multiple coding sequences are loaded onto the chip and decoded in parallel through the neural network, thereby increasing the parallelism of Huffman decoding, reducing decoding delay, and improving the decoding speed in scenarios such as large model bandwidth compression.

It can be understood that the above-mentioned various method embodiments mentioned in the present disclosure can be combined with each other to form a combined embodiment without violating the principle logic. Due to space limitations, the present disclosure will not repeat them. It can be understood by those skilled in the art that in the above-mentioned method of the specific implementation method, the specific execution order of each step should be determined according to its function and possible internal logic.

In addition, the present disclosure also provides a decoding device, an electronic device, and a computer-readable storage medium, all of which can be used to implement any decoding method provided by the present disclosure. The corresponding technical solutions and descriptions are referred to in the corresponding records of the method part and will not be repeated here.

FIG3 is a block diagram of a decoding device provided by an embodiment of the present disclosure.

3 , an embodiment of the present disclosure provides a decoding device 300, which may include:

The acquisition module 301 is used to acquire multiple coding sequences based on Huffman coding; the multiple coding sequences are obtained by segmenting the data to be encoded and encoding the multiple segments respectively;

A loading module 302, used to load multiple coding sequences onto the chip;

The decoding module 303 is used to use the decoding model on the chip to perform on-chip parallel decoding on multiple coding sequences and to perform parallel decoding on multiple codes within each coding sequence;

The decoding model is a model with parallel decoding capability obtained by training or fitting multiple coding sequence samples; the multiple coding sequence samples are obtained by segmenting the data samples to be encoded and then encoding the multiple segments separately.

FIG. 4 is a block diagram of an electronic device provided in an embodiment of the present disclosure.

4 , an embodiment of the present disclosure provides an electronic device, which includes multiple processing cores 401 and an on-chip network 402 , wherein the multiple processing cores 401 are all connected to the on-chip network 402 , and the on-chip network 402 is used to exchange data between the multiple processing cores and external data.

One or more instructions are stored in one or more processing cores 401 , and the one or more instructions are executed by one or more processing cores 401 , so that one or more processing cores 401 can execute the above decoding method.

In some embodiments, the electronic device may be a brain-like chip, and since the brain-like chip can use vectorized computing and needs to load the weight information and other parameters of the neural network model through an external memory such as a double data rate (DDR) synchronous dynamic random access memory, the batch processing in the disclosed embodiment has a higher computing efficiency.

The present disclosure also provides a computer-readable storage medium on which a computer program is stored, wherein the computer program implements the above decoding method when executed by a processor/processing core. The computer-readable storage medium may be a volatile or non-volatile computer-readable storage medium.

The embodiment of the present disclosure also provides a computer program product, including a computer-readable code, or a non-volatile computer-readable storage medium carrying the computer-readable code. When the computer-readable code runs in a processor of an electronic device, the processor in the electronic device executes the above decoding method.

It will be appreciated by those skilled in the art that all or some of the steps, systems, and functional modules/units in the methods disclosed above may be implemented as software, firmware, hardware, and appropriate combinations thereof. In a hardware implementation, the division between the functional modules/units mentioned in the above description does not necessarily correspond to the division of physical components; for example, a physical component may have multiple functions, or a function or step may be performed by several physical components in cooperation. Some or all physical components may be implemented as software executed by a processor, such as a central processing unit, a digital signal processor, or a microprocessor, or may be implemented as hardware, or may be implemented as an integrated circuit, such as an application-specific integrated circuit. Such software may be distributed on a computer-readable storage medium, which may include a computer storage medium (or a non-transitory medium) and a communication medium (or a transient medium).

As is known to those of ordinary skill in the art, the term computer storage media includes volatile and nonvolatile, removable and non-removable media implemented in any method or technology for storing information such as computer-readable program instructions, data structures, program modules or other data. Computer storage media include, but are not limited to, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM), static random access memory (SRAM), flash memory or other memory technology, portable compact disk read-only memory (CD-ROM), digital versatile disk (DVD) or other optical disk storage, magnetic cassettes, magnetic tapes, magnetic disk storage or other magnetic storage devices, or any other medium that can be used to store the desired information and can be accessed by a computer. In addition, as is known to those of ordinary skill in the art, communication media typically contain computer-readable program instructions, data structures, program modules or other data in a modulated data signal such as a carrier wave or other transport mechanism, and may include any information delivery media.

The computer-readable program instructions described herein can be downloaded from a computer-readable storage medium to each computing/processing device, or downloaded to an external computer or external storage device via a network, such as the Internet, a local area network, a wide area network, and/or a wireless network. The network can include copper transmission cables, optical fiber transmissions, wireless transmissions, routers, firewalls, switches, gateway computers, and/or edge servers. The network adapter card or network interface in each computing/processing device receives the computer-readable program instructions from the network and forwards the computer-readable program instructions for storage in the computer-readable storage medium in each computing/processing device.

The computer program instructions for performing the operations of the present disclosure may be assembly instructions, instruction set architecture (ISA) instructions, machine instructions, machine-dependent instructions, microcode, firmware instructions, state setting data, or source code or object code written in any combination of one or more programming languages, including object-oriented programming languages, such as Smalltalk, C++, etc., and conventional procedural programming languages, such as "C" language or similar programming languages. The computer-readable program instructions may be executed entirely on the user's computer, partially on the user's computer, as a separate software package, partially on the user's computer, partially on a remote computer, or entirely on a remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., using an Internet service provider to connect through the Internet). In some embodiments, by using the state information of the computer-readable program instructions to personalize an electronic circuit, such as a programmable logic circuit, a field programmable gate array (FPGA), or a programmable logic array (PLA), the electronic circuit may execute the computer-readable program instructions, thereby implementing various aspects of the present disclosure.

The computer program product described herein may be implemented in hardware, software, or a combination thereof. In one optional embodiment, the computer program product is embodied as a computer storage medium, and in another optional embodiment, the computer program product is embodied as a software product, such as a software development kit (SDK), etc.

Various aspects of the present disclosure are described herein with reference to flowcharts and/or block diagrams of methods, devices (systems) and computer program products according to embodiments of the present disclosure. It should be understood that each box in the flowchart and/or block diagram and the combination of boxes in the flowchart and/or block diagram can be implemented by computer-readable program instructions.

These computer-readable program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, thereby producing a machine, so that when these instructions are executed by the processor of the computer or other programmable data processing device, a device that implements the functions/actions specified in one or more boxes in the flowchart and/or block diagram is generated. These computer-readable program instructions can also be stored in a computer-readable storage medium, and these instructions cause the computer, programmable data processing device, and/or other equipment to work in a specific manner, so that the computer-readable medium storing the instructions includes a manufactured product, which includes instructions for implementing various aspects of the functions/actions specified in one or more boxes in the flowchart and/or block diagram.

Computer-readable program instructions may also be loaded onto a computer, other programmable data processing apparatus, or other device so that a series of operating steps are performed on the computer, other programmable data processing apparatus, or other device to produce a computer-implemented process, thereby causing the instructions executed on the computer, other programmable data processing apparatus, or other device to implement the functions/actions specified in one or more boxes in the flowchart and/or block diagram.

The flow chart and block diagram in the accompanying drawings show the possible architecture, function and operation of the system, method and computer program product according to multiple embodiments of the present disclosure. In this regard, each square box in the flow chart or block diagram can represent a part of a module, program segment or instruction, and a part of the module, program segment or instruction includes one or more executable instructions for realizing the specified logical function. In some alternative implementations, the functions marked in the square box can also occur in a sequence different from that marked in the accompanying drawings. For example, two continuous square boxes can actually be executed substantially in parallel, and they can sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each square box in the block diagram and/or flow chart, and the combination of the square boxes in the block diagram and/or flow chart can be implemented with a dedicated hardware-based system that performs the specified function or action, or can be implemented with a combination of special hardware and computer instructions.

Example embodiments have been disclosed herein, and although specific terms are employed, they are used and should be interpreted only in a general illustrative sense and not for limiting purposes. In some instances, it will be apparent to those skilled in the art that, unless otherwise expressly noted, features, characteristics, and/or elements described in conjunction with a particular embodiment may be used alone or in combination with features, characteristics, and/or elements described in conjunction with other embodiments. Therefore, those skilled in the art will appreciate that various changes in form and detail may be made without departing from the scope of the present disclosure as set forth in the appended claims.

Claims (13)

1. A decoding method, the method comprising:

Acquiring a plurality of coding sequences based on Huffman coding; the plurality of coding sequences are obtained by respectively coding a plurality of segments after segmenting the data to be coded; the data to be encoded comprises: large model weight data;

loading the plurality of coding sequences onto a chip;

adopting the decoding model on the chip to perform on-chip parallel decoding on the plurality of coding sequences and performing parallel decoding on a plurality of codes inside each coding sequence;

The decoding model is a model with parallel decoding capability, which is obtained by training or fitting based on a plurality of coded sequence samples; the plurality of code sequence samples are obtained by respectively coding a plurality of segments after segmenting the data sample to be coded; the plurality of code sequence samples are obtained after being coded based on the current data to be compressed, and the decoding model does not need to have generalization capability for data other than the current data to be compressed; the decoding model includes a neural network; the neural network is used for executing any one of an image processing task, a voice processing task, a text processing task and a video processing task.

2. The decoding method of claim 1, wherein,

The on-chip decoding model is adopted to decode the plurality of coding sequences in parallel on a chip, and decode a plurality of codes inside each coding sequence in parallel, and the method comprises the following steps:

The multiple coding sequences are input into one or more of the neural networks for parallel decoding.

3. The decoding method of claim 2, wherein the neural network is a one-dimensional neural network, and the decoding model includes a plurality of the one-dimensional neural networks;

the inputting the plurality of coding sequences into one or more of the neural networks for parallel decoding includes:

inputting a plurality of the coding sequences into a plurality of the one-dimensional neural networks respectively;

and decoding one coding sequence through each one-dimensional neural network, and decoding a plurality of codes in the corresponding coding sequence in parallel in each one-dimensional neural network so as to realize parallel decoding of the plurality of coding sequences.

4. The decoding method of claim 2, wherein the neural network is a multidimensional neural network;

the inputting the plurality of coding sequences into one or more of the neural networks for parallel decoding includes:

inputting the plurality of coding sequences into the multi-dimensional neural network;

And decoding the plurality of coding sequences in parallel through the multidimensional neural network, and decoding a plurality of codes in the corresponding coding sequences in parallel in each dimensional neural network.

5. The decoding method of claim 2, wherein the neural network comprises a convolutional neural network.

6. The decoding method of claim 1, wherein prior to said loading said plurality of encoded sequences onto a chip, said method further comprises:

acquiring a first coding sequence with the longest length in the plurality of coding sequences;

and according to the length of the first coding sequence, carrying out bit number compensation on a second coding sequence except the first coding sequence in the plurality of coding sequences.

7. The decoding method of claim 1, wherein prior to acquiring the plurality of encoded sequences based on huffman coding, the method further comprises:

when the data to be encoded is segmented, the obtained statistical characteristic difference of each data segment to be encoded is within a preset difference range.

8. The decoding method of claim 1, wherein the decoding model comprises a neural network; before the on-chip decoding model is adopted to perform on-chip parallel decoding on the plurality of coding sequences and perform parallel decoding on a plurality of codes inside each coding sequence, the method further comprises:

training the neural network by taking the plurality of code sequence samples as training data;

and under the condition that the loss value of the neural network meets a preset condition, obtaining the decoding model.

9. The decoding method of claim 8, wherein,

The data sample to be encoded comprises: large model weight data samples;

The plurality of code sequence samples includes: and segmenting the large model weight data sample, and then respectively carrying out Huffman coding on each segment to obtain a plurality of coding sequences.

10. The decoding method according to claim 1 or 9, wherein the data to be encoded is large model weight data; before acquiring the plurality of coding sequences based on huffman coding, the method further comprises:

respectively taking the large model weight data of each row in the large model weight matrix as a data segment to be encoded, and carrying out Huffman encoding on each data segment to be encoded to obtain an encoding sequence; or alternatively

And taking the large model weight data of each column in the large model weight matrix as a data segment to be encoded, and carrying out Huffman encoding on each data segment to be encoded to obtain a coding sequence.

11. A decoding apparatus, comprising:

an acquisition module, configured to acquire a plurality of coding sequences based on huffman coding; the plurality of coding sequences are obtained by respectively coding a plurality of segments after segmenting the data to be coded; the data to be encoded comprises: large model weight data;

the loading module is used for loading the plurality of coding sequences onto a chip;

the decoding module is used for carrying out on-chip parallel decoding on the plurality of coding sequences by adopting the on-chip decoding model and carrying out parallel decoding on a plurality of codes in each coding sequence;

The decoding model is a model with parallel decoding capability, which is obtained by training or fitting based on a plurality of coded sequence samples; the plurality of code sequence samples are obtained by respectively coding a plurality of segments after segmenting the data sample to be coded; the plurality of code sequence samples are obtained after being coded based on the current data to be compressed, and the decoding model does not need to have generalization capability for data other than the current data to be compressed; the decoding model includes a neural network; the neural network is used for executing any one of an image processing task, a voice processing task, a text processing task and a video processing task.

12. An electronic device, comprising:

A plurality of processing cores; and

A network on chip configured to interact data between the plurality of processing cores and external data; wherein one or more of said processing cores have one or more instructions stored therein, one or more of said instructions being executable by one or more of said processing cores to enable one or more of said processing cores to perform the decoding method of any one of claims 1-10.

13. A computer readable storage medium, on which a computer program is stored, characterized in that the computer program, when being executed by a processor, implements the decoding method according to any of claims 1-10.