CN119232171B

CN119232171B - A data intelligent compression method for twin systems

Info

Publication number: CN119232171B
Application number: CN202411745562.9A
Authority: CN
Inventors: 潘成胜; 张晨曦; 施建锋
Original assignee: Nanjing University of Information Science and Technology
Current assignee: Nanjing University of Information Science and Technology
Priority date: 2024-12-02
Filing date: 2024-12-02
Publication date: 2025-04-25
Anticipated expiration: 2044-12-02
Also published as: CN119232171A

Abstract

The invention discloses an intelligent data compression method for a twin system, which is used for collecting video, audio and text data for preprocessing, constructing a Bootstrap model for carrying out feature extraction and preliminary compression on input data, constructing a Combined model consisting of the Bootstrap model and a Supporter model, combining probability distribution output by the Bootstrap model with probability distribution output by the Supporter model, carrying out final compression on the output probability distribution by using an arithmetic coding method to reduce the volume of data transmission and transmit the data to target equipment, decoding and reconstructing the received data by the target equipment, and recovering the original data before compression.

Description

Data intelligent compression method for twin system

Technical Field

The invention relates to the technical fields of computers and electronic information engineering, in particular to a data intelligent compression method oriented to a twin system.

Background

In a digital twin system, a twin model enables accurate monitoring, simulation and optimization of the real world by mapping physical objects in real time. The digital twin system is widely applied to a plurality of fields such as manufacturing, energy, smart city, traffic and the like, can improve the operation efficiency, reduce the failure rate and provide higher flexibility and automation degree. Digital twinning systems rely on the transmission and processing of large amounts of data to ensure real-time interaction between virtual models and physical entities. The data comprises various formats such as video, audio, sensor data and text information, and the like, and extremely low data transmission delay is required, so that with the continuous increase of the data volume, how to efficiently transmit the data while ensuring the real-time performance becomes one of the key challenges faced by the digital twin system. In response to this challenge, data compression techniques have proven to be a reliable and effective solution. The data compression technology can effectively reduce the data volume by reducing redundant information, thereby improving the transmission speed and reducing the bandwidth occupation. The method is particularly important for a digital twin system for processing diversified data and coping with complex environments, and particularly in a scene with low time delay requirements, the data compression technology can effectively cope with the challenges and can ensure the real-time performance and the high efficiency of the digital twin system. Therefore, there is a need for data compression techniques that enable low latency, efficient transmission of data. However, the existing data compression method is usually optimized for specific types of data, has remarkable effect when processing single type of data, has poor adaptability when facing multiple data types, has remarkably reduced compression effect, and is difficult to meet the requirement of environment on low-delay transmission.

In order to solve the problems and the current situation, the invention provides a general data compression method based on a neural network and arithmetic coding. By designing a Bootstrap model and a Combined model and combining a multi-head attention mechanism, a position coding and a residual error network, the data compression ratio and the compression speed are improved, so that the time delay of data transmission is effectively reduced, multiple data types in a twin system are adapted, and an efficient data compression transmission scheme is provided for the construction of the twin system.

Disclosure of Invention

The invention aims to solve the problems that in a twin system, data transmission faces low time delay and diversification, and the real-time interaction of the data between the twin system and a physical entity is seriously influenced. The existing compression method is usually used for compressing specific types of data, and has remarkable effect when processing single type of data, but has poor adaptability when facing multiple data types, and the compression effect is remarkably reduced. In addition, the conventional compression method has difficulty in achieving efficient compression of diversified data in a rapidly changing complex environment. Aiming at the problems, the invention provides a data intelligent compression method for a twin system, which aims to improve the data compression ratio and the compression time efficiency by designing a Bootstrap model and a Combined model and combining a multi-head attention mechanism, a position coding and a residual error network, thereby effectively reducing the time delay of data transmission, adapting to various data types in the twin system and ensuring the real-time processing of data.

In order to realize the functions, the invention designs a twin system-oriented intelligent data compression method, which comprises the following steps S1 to S5, and the compression and transmission of data are completed:

Step S1, collecting video data, audio data and text data as data to be compressed, preprocessing the data to be compressed, including standardized processing and format conversion of the data, and obtaining preprocessed data;

Step S2, constructing and training a Bootstrap model, taking the preprocessed data as input, and carrying out feature extraction and preliminary compression on the preprocessed data based on a position coding module, a bidirectional gating circulating unit, a multi-head attention mechanism, a residual error module and a functional module, and outputting probability distribution logits _b;

s3, constructing and training a coded model, wherein the coded model consists of a Bootstrap model and a Supporter model, the Supporter model is based on a position coding module, a multi-head attention mechanism, a residual error module and a functional module, probability distribution logits _s is output, the coded model combines probability distribution logits _b output by the Bootstrap model with probability distribution logits _s output by the Supporter model, and sequence characteristic representation and further compression of data are enhanced to generate probability distribution logits _c;

S4, finally compressing the probability distribution output in the step S3 by using an arithmetic coding method to reduce the volume of data transmission and transmitting the data to target equipment;

And S5, decoding and reconstructing the received data by utilizing a pre-trained Bootstrap model, a Combined model and an arithmetic decoding method on target equipment, and restoring the original data before compression.

The beneficial effects of the invention are that compared with the prior art, the invention has the following advantages:

The invention provides a twin system-oriented data intelligent compression method which is suitable for various types of data including text, video, audio and sensor data. First, consistent, efficient compression is achieved between different types of data through specific data preprocessing steps and a generic compression model. And secondly, a Bootstrap model and a Combined model are used, a multi-head attention mechanism and a residual block are Combined, so that efficient compression of diversified data is realized, and the method has remarkable advantages in compression ratio and compression speed compared with the traditional method. Meanwhile, by combining with a probability model of arithmetic coding, high reliability of data compression is ensured, efficient lossless compression is realized at a lower bit rate, and loss and error rate of data in the transmission process are reduced. Finally, the method of the invention adopts a standard neural network architecture and an arithmetic coding algorithm, has good realizability and expandability, can be conveniently integrated into the existing twin system, and can be customized and expanded according to specific requirements. In summary, the general data compression method based on the neural network and the arithmetic coding not only can adapt to various data types, but also has remarkable advantages in compression efficiency, further can remarkably reduce the time delay of data transmission, particularly in a complex environment, can ensure the rapid transmission and real-time processing of the data of the twin system, and provides an efficient data compression transmission scheme for the construction of the twin system.

Drawings

FIG. 1 is a flow chart of a data intelligent compression method for a twin system provided according to an embodiment of the present invention;

FIG. 2 is a schematic diagram of a training process of a Bootstrap model provided according to an embodiment of the present invention;

FIG. 3 is a block diagram of a Combined model provided in accordance with an embodiment of the present invention;

FIG. 4 is a graph of training and testing loss of a Bootstrap model as a function of iteration number, provided in accordance with an embodiment of the present invention;

FIG. 5 is a graph of the change in loss during training of a Combined model provided in accordance with an embodiment of the present invention;

FIG. 6 is a graph comparing the performance of the method of the present invention with a series of compression algorithms on a diverse data set, provided in accordance with an embodiment of the present invention.

Detailed Description

The invention is further described below with reference to the accompanying drawings. The following examples are only for more clearly illustrating the technical aspects of the present invention, and are not intended to limit the scope of the present invention.

The embodiment of the invention provides a twin system-oriented data intelligent compression method, referring to fig. 1, the following steps S1-S5 are executed to complete data compression and transmission:

Step S1, collecting video data, audio data and text data as data to be compressed, preprocessing the data to be compressed to ensure that different types of data can be effectively processed, wherein the preprocessing comprises standardized processing and format conversion of the data, and obtaining preprocessed data;

The method for preprocessing the data to be compressed in step S1 is as follows:

inter-sampling is performed for data frames of video data and audio data, the sampling rate is reduced by the following formula:

;

Wherein, Is a sampled data frame, S _i represents an i-th frame in the original data frame sequence, and n is a sampling interval, i.e. every n frames;

and carrying out normalization processing on the sampled video data and audio data, and unifying data scales, wherein the normalization processing comprises the following specific formula:

;

wherein x _norm is a normalized data value, x is an original data value, and min (x) and max (x) are respectively a minimum value and a maximum value in the data;

For text data, the mapping of characters to integers is performed by the following formula:

;

Where C _int is the mapped integer value, C is the original character, and ord is a function that converts the character to a corresponding ASCII integer value.

Step S2, constructing and training a Bootstrap model, taking the preprocessed data as input, and carrying out feature extraction and preliminary compression on the preprocessed data based on a position coding module, a bidirectional gating circulating unit, a multi-head attention mechanism, a residual error module and a functional module to optimize the representation of the data and improve the compression efficiency;

the Bootstrap model in the step S2 sequentially comprises a position coding module, a bidirectional gating circulating unit, a multi-head attention mechanism, a residual error module and a functional module;

The Bootstrap model inputs preprocessed data, adds position information for the data through a position coding module, then adopts a bidirectional gating circulation unit (BiGRU) to output embedded features, captures bidirectional dependency of the data, further improves understanding capability of the model on sequence data, then utilizes a multi-head attention mechanism to output attention scores, captures complex relationships among the data through flattening operation, solves gradient disappearance problem in a depth network through a residual error module, finally generates unscaled probability distribution logits _b through two parallel functional modules to complete preliminary prediction, wherein the two parallel functional modules are a linear layer and a full-connection layer, performs preliminary transformation on the extracted features through the linear layer, and then maps the features to an output space through the full-connection layer.

The training process of the Bootstrap model is described with reference to fig. 2. The Bootstrap model is trained by traversing the input data multiple times, optimizing parameters to minimize cross entropy loss, and generating a high-quality symbol probability prediction model. The Bootstrap model adapts to the requirements of different data sets by automatically selecting super parameters, and captures long-term dependency and complex modes of data by utilizing a multi-head self-attention mechanism, a position coding and residual error module. After training, the model parameters are saved and used as a part of a compressed file, so that an efficient prediction basis is provided for the compression process of the data intelligent compression method for the twin system.

In the Combined model in step S3, the Supporter model inputs the preprocessed data, adds position information for the data through the position coding module, splices the data added with the position information with the embedded features output by the bidirectional gating circulation unit of the Bootstrap model, outputs attention scores through a multi-head attention mechanism, captures complex relations among the data, sequentially passes through a plurality of residual modules, enhances the stability and training efficiency of the model, and finally passes through three parallel functional modules, namely a linear module, a dense module and a residual module, wherein each module serves as an independent predictor for learning features with different complexity. The output vector of each module is reduced to the dimension matched with the vocabulary size through linear transformation, and finally weighted summation is carried out to generate probability distribution logits _s, and Supporter model can extract basic mode, complex feature and deep information respectively, so that the accuracy of overall prediction is improved.

The Combined model combines the probability distribution output by the Bootstrap model and the Supporter model, and generates final probability distribution logits _c through convex sum;

;

Where λ is a learnable parameter constrained within the range of [0,1] by a sigmoid activation function. Through the combination, the Combined model remarkably improves compression efficiency and prediction precision, and ensures excellent performance of the model on various data sets.

Combinedshape structure referring to fig. 3, in the combinedshape, bootstrap model is Combined with the random initialized Supporter model, and final symbol probability prediction is generated through convex sum. When the training phase of the Combined model starts, the Bootstrap model keeps the parameters trained in step S2 unchanged, and the Supporter model performs adaptive updating according to the need. The input data sequence is divided into a plurality of equal-sized portions, and batch predictions are made in parallel. The predicted result of the coded model is converted into probability distribution through a softmax activation function and then is input into an arithmetic coder for symbol coding. The Supporter model continuously performs parameter updating according to actual symbol prediction in the encoding process, and optimizes the prediction capacity by minimizing cross entropy loss. The multi-head self-attention mechanism, the position coding and residual error module further improves the adaptability and the prediction precision of the model, and ensures that the model can be rapidly adapted to the non-stationary statistical characteristics in the sequence. Finally, the coded symbols are input into an arithmetic coder together with the probability prediction generated by the coded model to generate an efficiently compressed file, thereby realizing higher compression ratio and faster coding speed.

The processing process of the position coding module to the data is as follows:

;

Wherein, 、For position-coded data, pos is the sequence position, i is the dimension index, and d is the model dimension.

The processing process of the multi-head attention mechanism on the data is as follows:

;

Wherein Attention (Q, K, V) is the Attention score, Q, K, V represent query, key and value matrices, respectively, d _k is the dimension of the key vector for scaling the dot product, preventing the result after the dot product from being too large, affecting the gradient of softmax, which is a normalization function for converting the Attention score into a probability distribution.

S4, finally compressing the probability distribution output in the step S3 by using an arithmetic coding method, wherein the arithmetic coding method is an efficient coding mode based on the probability distribution of the data so as to reduce the volume of data transmission and transmit the data to target equipment;

the arithmetic coding method in step S4 is as follows:

;

where new_interval is a new coding section, low and high are the lower and upper bounds of the current section, and probability is the occurrence probability of the current symbol.

And S5, decoding and reconstructing the received data by utilizing a pre-trained Bootstrap model, a Combined model and an arithmetic decoding method on target equipment, and restoring the original data before compression. The arithmetic decoding method in the decoding process cooperates with model reasoning work, so that the integrity and accuracy of the data in the transmission and recovery processes are ensured.

Fig. 4 shows the variation of training and testing loss of the Bootstrap model with the iteration number in step S2 in the application of the data intelligent compression method for the twin system designed by the present invention. As the number of iterations increases, the training loss drops significantly from 1.35 to near 0.98, while the test loss drops from 1.20 to near 0.98, indicating that the performance of the Bootstrap model on the training and validation data set continues to improve and eventually tends to stabilize. In the whole process, the test loss is slightly lower than the training loss all the time, which indicates that the Bootstrap model has no overfitting and has good generalization capability. Overall, the Bootstrap model successfully converges in the training process, and the expected compression effect is achieved.

Fig. 5 shows the change in loss during training of the Combined model in step S3. The partial training is based on the model training of step S2, and results obtained by further combining and optimizing the model structure. It can be seen that the loss value gradually decreases with the training, and the error of the model gradually decreases in the continuous optimization process, which means that the performance of the model is further improved in the training process of the Combined model. By comparing the training results of step S2, it can be seen that the training of the Combined model is based on the performance of the initial model and the loss value continues to be reduced in further combinations and optimizations. This demonstrates that the Combined model successfully inherits the advantages of the early training part and further improves the generalization ability and compression effect of the model. From the overall trend, fig. 5 shows that the model is effective in this stage of optimization, and the test loss reaches a relatively stable level, proving the rationality and effectiveness of the model design and training strategy.

FIG. 6 shows the performance of the method designed by the present invention versus a series of compression algorithms on a diverse data set. These algorithms include conventional compressors 7-Zip, BSC, zip, bzip, gzip and Tar, and neural network based compressors CMIX and Dzip. Test datasets cover a wide range of fields including Text (e.g., webster and Text 8), audio (Audio), genomic data (e.g., ill-quality), floating point data (e.g., num-control), human chromosome data (e.g., h.chr1 and h.chr20), image data (e.g., kodim 02), and mixed datasets XOR60 and HMM60, as well as specific datasets such as Np-bases and Enwiki. Experimental results show that the method provided by the invention is excellent in all data sets, and is particularly superior to other traditional compressors and neural network compressors in text and genome data sets. Compared with the original compression algorithm, the method provided by the invention has obvious improvement on the bit rate per character (BPC) and the overall compression efficiency. In addition, compared with other neural network compressors, the design method of the invention not only has excellent compression efficiency, but also has obvious advantages in processing speed. For example, although the compression effect on text data is slightly lower than that of CMIX method, the encoding and decoding efficiency is three to four times that of CMIX, and the performance advantage is fully reflected. The outstanding performance of the method disclosed by the invention benefits from a multi-head self-attention mechanism, a position coding and a residual block architecture, the adaptability and generalization capability of the method on different data types are greatly improved by the technologies, and the wide application potential and the technical advantages of the method in the modern data compression field are shown.

In summary, the method of the present invention provides significant advantages by comparing and analyzing the number of Bits Per Character (BPC) of a plurality of compressors on different types of real data sets. The compression effect on text, audio and genome data sets is significantly better than that of traditional compressors and other neural network compressors by introducing a multi-head self-attention mechanism, position coding and residual modules. These improvements not only improve compression efficiency and prediction accuracy, but also enhance the adaptability and generalization ability of the model. Thus, the remarkable performance of the proposed method over a wide variety of data types demonstrates its feasibility and effectiveness as an efficient, reliable compression method.

The embodiments of the present invention have been described in detail with reference to the drawings, but the present invention is not limited to the above embodiments, and various changes can be made within the knowledge of those skilled in the art without departing from the spirit of the present invention.

Claims

1. A data intelligent compression method for a twin system, characterized in that the following steps S1 to S5 are executed to complete data compression and transmission:

Step S1: collecting video data, audio data and text data as data to be compressed, and performing preprocessing on the data to be compressed, including data standardization and format conversion, to obtain preprocessed data;

Step S2: Build and train the Bootstrap model, take the preprocessed data as input, extract features and perform preliminary compression on the preprocessed data based on the position encoding module, bidirectional gated recurrent unit, multi-head attention mechanism, residual module and functional module, and output the probability distribution logits _b ;

Step S3: Build and train a Combined model. The Combined model consists of a Bootstrap model and a Supporter model. The Supporter model outputs a probability distribution logits _s based on a position encoding module, a multi-head attention mechanism, a residual module, and a functional module. The Combined model combines the probability distribution logits _b output by the Bootstrap model with the probability distribution logits _s output by the Supporter model to strengthen the sequence feature representation and further compress the data, and generate a probability distribution logits _c .

Step S4: finally compressing the probability distribution outputted in step S3 using an arithmetic coding method to reduce the volume of data transmission, and transmitting the data to the target device;

Step S5: On the target device, the received data is decoded and reconstructed using the pre-trained Bootstrap model, Combined model and arithmetic decoding method to restore the original data before compression.

2. According to the twin system-oriented data intelligent compression method of claim 1, it is characterized in that the method for preprocessing the data to be compressed in step S1 is as follows:

Inter-frame sampling is performed on the data frames of video data and audio data, and the sampling rate is reduced by the following formula:

;

in, is the sampled data frame, Si _represents the i- th frame in the original data frame sequence, and n is the sampling interval;

For the sampled video data and audio data, normalization is performed as follows:

;

Among them, x _norm is the normalized data value, x is the original data value, min( x ) and max( x ) are the minimum and maximum values in the data respectively;

For text data, characters are mapped to integers using the following formula:

;

Where C _int is the integer value after mapping, c is the original character, and ord is the function that converts the character to the corresponding ASCII integer value.

3. According to a twin system-oriented data intelligent compression method according to claim 1, it is characterized in that the Bootstrap model described in step S2 includes a position encoding module, a bidirectional gated recurrent unit, a multi-head attention mechanism, a residual module and a functional module in sequence;

The Bootstrap model inputs the preprocessed data, adds position information to the data through the position encoding module, and then uses a bidirectional gated recurrent unit to output embedded features to capture the bidirectional dependencies of the data. After that, it flattens and uses a multi-head attention mechanism to output attention scores to capture the complex relationship between the data. The unscaled probability distribution logits _b is generated through the residual module and two parallel functional modules to complete the preliminary prediction. The two parallel functional modules are the linear layer and the fully connected layer. The linear layer is first used to perform a preliminary transformation on the extracted features, and then the features are mapped to the output space through the fully connected layer.

4. According to claim 1, a data intelligent compression method for twin systems is characterized in that, in the Combined model described in step S3, the Supporter model inputs the preprocessed data, adds position information to the data through the position encoding module, splices the data with the added position information with the embedded features output by the bidirectional gated recurrent unit of the Bootstrap model, outputs the attention score of the spliced features through a multi-head attention mechanism, captures the complex relationship between the data, passes through multiple residual modules in turn, and finally passes through three parallel functional modules, the three parallel functional modules are a linear module, a dense module and a residual module, the output vector of each module is reduced to a dimension matching the vocabulary size through a linear transformation, and finally a weighted sum is performed to generate a probability distribution logits _s ;

The Combined model combines the probability distributions output by the Bootstrap model and the Supporter model, and generates the final probability distribution logits _c through convex sum;

;

Here, λ is a learnable parameter constrained to the range [0,1] by a sigmoid activation function.

5. According to a twin system-oriented data intelligent compression method according to claim 3 or 4, it is characterized in that the position encoding module processes the data as follows:

;

in, , The data is position-encoded, pos is the sequence position, i is the dimension index, and d is the model dimension.

6. According to a twin system-oriented data intelligent compression method according to claim 3 or 4, it is characterized in that the multi-head attention mechanism processes data as follows:

;

Where Attention(Q,K,V) is the attention score, Q, K, V represent query, key and value matrices respectively, _dk is the dimension of the key vector, and softmax is the normalization function.

7. According to a twin system-oriented data intelligent compression method according to claim 1, it is characterized in that the arithmetic coding method in step S4 is as follows:

;

Among them, new_interval is the new encoding interval, low and high are the lower and upper bounds of the current interval, and probability is the probability of occurrence of the current symbol.