CN114827287B - Data compression method and system - Google Patents

Abstract

The present invention discloses a data compression method and system, and belongs to the technical field of data compression. The method of the present invention comprises: obtaining a short character description of the data to be compressed, determining the forward active electric energy data block of the data to be compressed according to the short descriptor of the data to be compressed, determining the data Data0 to be transmitted and the data Data1 transmitted last time in the forward active electric energy data block of the data to be compressed, and performing a difference between Data1 and Data0 to generate Data2; generating compression header information of the data Data0 to be transmitted according to Data2; generating CRC check data according to Data0; and using Data2, compression header information and CRC check data to generate compressed data of the data Data0 to be transmitted. The present invention improves the data transmission efficiency and bandwidth utilization, and stores the data efficiently.

Description

A data compression method and system

Technical Field

The present invention relates to the technical field of data compression, and more specifically, to a data compression method and system.

Background technique

Energy Internet technology and business are developing rapidly, the "new infrastructure" strategy of power grid companies is advancing steadily, and the types and number of customer-side measurement equipment are growing explosively. More and more equipment is connected to customer-side energy, including electric vehicle charging facilities, distributed energy, residential households, commercial buildings, enterprises and parks, regional multi-energy measurement equipment, etc., which puts forward higher requirements for communication access coverage, bandwidth rate, communication delay, system stability, etc.

Summary of the invention

In order to improve data transmission efficiency and bandwidth utilization and to efficiently store data, the present invention proposes a data compression method, comprising:

Obtain a short descriptor of the data to be compressed, determine the forward active electric energy data block of the data to be compressed according to the short descriptor of the data to be compressed, determine the data Data0 to be transmitted and the data Data1 transmitted last time in the forward active electric energy data block of the data to be compressed, and perform a difference between Data1 and Data0 to generate Data2;

Generate compression header information of the data Data0 to be transmitted according to Data2;

Generate CRC check data according to Data0;

The compressed data of the data Data0 to be transmitted is generated using Data2, the compressed header information and the CRC check data.

After the compressed data is generated, the transmission protocol used for external transmission includes: transmission by requesting compressed transmission or responding to compressed transmission;

If the transmission method is to request compressed transmission, the generated compressed data is transmitted after carrying the compression algorithm, compression mode information and the processing result of the compressed data.

Optionally, the compression method of the compressed data is lossless difference data compression;

If the compression method of the compressed data cannot use lossless difference data compression, the compression method of the compressed data uses lossless data compression, and the lossless data compression compresses CRC check data and Data0 to generate compressed data.

Optional, compression header information, including: compression header byte length and field compression bit;

The field compression bit uses two digits to indicate the data block compression method. 0 indicates lossless data compression, 1 indicates lossless differential data compression, and 2 indicates no compression.

Optional, short descriptor, including: object identifier, object characteristics, object attribute identifier and element index within the attribute.

Optionally, the method further includes: after using the CRC check data generated by the Data0 data, generating CRC16 check data as additional check data based on the CRC check data.

The present invention also proposes a data compression system, comprising:

Generate a data difference unit, obtain a short descriptor of the data to be compressed, determine the forward active electric energy data block that needs to be compressed according to the short descriptor of the data to be compressed, determine the data Data0 that needs to be transmitted and the data Data1 that was transmitted last time in the forward active electric energy data block that needs to be compressed, and make a difference between Data1 and Data0 to generate Data2;

A compression header information generating unit generates compression header information of the data Data0 to be transmitted according to Data2;

A check data generating unit generates CRC check data according to Data0;

The compressed data generating unit generates compressed data of the data Data0 to be transmitted using Data2, the compressed header information and the CRC check data.

Optionally, after the compressed data is generated, the transmission protocol used for external transmission includes: transmitting in a manner of requesting compressed transmission or responding to compressed transmission;

If the transmission method is to request compressed transmission, the generated compressed data is transmitted after carrying the compression algorithm, compression mode information and the processing result of the compressed data.

Optionally, the compression method of the compressed data is lossless difference data compression;

If the compression method of the compressed data cannot use lossless difference data compression, the compression method of the compressed data uses lossless data compression, and the lossless data compression compresses CRC check data and Data0 to generate compressed data.

Optional, compression header information, including: compression header byte length and field compression bit;

The field compression bit uses two digits to indicate the data block compression method. 0 indicates lossless data compression, 1 indicates lossless differential data compression, and 2 indicates no compression.

Optional, short descriptor, including: object identifier, object characteristics, object attribute identifier and element index within the attribute.

Optionally, the verification data generation unit is further used to: after using the CRC verification data generated by the Data0 data, generate CRC16 verification data as additional verification data according to the CRC verification data.

The present invention compresses Data2, compression header information and CRC check data to generate compressed data, transmits the compressed data, improves data transmission efficiency and bandwidth utilization, and stores data efficiently.

BRIEF DESCRIPTION OF THE DRAWINGS

Fig. 1 is a flow chart of the method of the present invention;

FIG. 2 is a structural diagram of the system of the present invention.

Detailed ways

Now, exemplary embodiments of the present invention are described with reference to the accompanying drawings. However, the present invention can be implemented in many different forms and is not limited to the embodiments described herein. These embodiments are provided to disclose the present invention in detail and completely and to fully convey the scope of the present invention to those skilled in the art. The terms used in the exemplary embodiments shown in the accompanying drawings are not intended to limit the present invention. In the accompanying drawings, the same units/elements are marked with the same reference numerals.

Unless otherwise specified, the terms (including technical terms) used herein have the commonly understood meanings to those skilled in the art. In addition, it is understood that the terms defined in commonly used dictionaries should be understood to have the same meanings as those in the context of the relevant fields, and should not be understood as idealized or overly formal meanings.

The present invention proposes a data compression method, as shown in FIG1 , comprising:

Obtain a short descriptor of the data to be compressed, determine the forward active electric energy data block of the data to be compressed according to the short descriptor of the data to be compressed, determine the data Data0 to be transmitted and the data Data1 transmitted last time in the forward active electric energy data block of the data to be compressed, and perform a difference between Data1 and Data0 to generate Data2;

Among them, the data to be compressed include various power data such as electric energy, voltage, current, and power;

Generate compression header information of the data Data0 to be transmitted according to Data2;

Generate CRC check data according to Data0;

The compressed data of the data Data0 to be transmitted is generated using Data2, the compressed header information and the CRC check data.

After the compressed data is generated, the CRC16 check data generated by the Data0 data is used as additional check data. The additional check data is provided to the receiver of the compressed data to check the correctness of the received compressed data.

After the compressed data is generated, the transmission protocol used for external transmission includes: transmission by requesting compressed transmission or responding to compressed transmission;

If the transmission method is to request compressed transmission, the generated compressed data is transmitted after carrying the compression algorithm, compression mode information and the processing result of the compressed data.

The compression header information includes: the compression header byte length and the field compression bit;

The field compression bit uses two digits to indicate the data block compression method. 0 indicates lossless data compression, 1 indicates lossless differential data compression, and 2 indicates no compression.

The short descriptor includes: object identifier, object feature, object attribute identifier and element index within the attribute.

The transmission protocol used for the transmission of compressed data in the present invention supports compression coding;

The transmission protocol is based on the client-side energy measurement data exchange protocol, and supports two types of compressed transmission: request compressed transmission and response compressed transmission. When requesting compressed transmission, in addition to supporting the transmission of compressed data, it also supports carrying parameter fields such as compression algorithm, compression mode information and compression processing results.

Supports local channels, including RS-485, M-Bus, CAN, RF, PLC, HPLC, micro-power wireless, and HPLC and micro-power wireless dual-mode communications. Supports remote channels, including GPRS, 3G, 4G, 5G, power network 230M, Ethernet, optical fiber, etc.

Because it supports carrying compression algorithm parameters, it supports multiple common compression algorithms such as LZW, LZ77, LZ77/78, RLE, GZIP, and customized client-side energy measurement data exchange protocol lossless data transmission compression algorithms.

Whether each data needs to be compressed and whether the compression method is to use lossless differential data compression or lossless full compression depends on the range of the data difference value and the transmission requirements. For example, if the difference value is 0, that is, there is no difference, lossless full compression is used. If the difference is small, lossless differential data compression is used. If the difference is large, no compression is performed and original data transmission is used. The compression method is indicated by the data compression header information.

The compression header information includes: compression header byte length and field compression bit.

The field compression bit uses two bits to indicate the data block compression method, 0 indicates full compression, 1 indicates differential compression, and 2 indicates no compression.

The present invention will be further described below in conjunction with embodiments:

For a forward active electric energy data block, the data Data0 that needs to be transmitted this time is:

The last transmitted data Data1 is:

The metering data Data0 transmitted this time and the metering data Data1 transmitted last time are subtracted from each other in the corresponding rate value fields to generate Data2:

Among them, the specific difference data and compression header of Data2 are as follows:

Bit7: 0, no compression header, 1: with compression header;

Bit5: Compression ratio 0:1/2 compression, 1:1/4 compression;

Bit0~Bit4: N data, specific compression method used;

In the difference data, each byte represents 4 groups of compression encoding methods

The compressed header information is represented as: 0x85, 0x52, 0x01.

Among them, 0x85 indicates that there is a length field, 1/2 compression, and the length is 5 Mbit. Here is an example of electric energy data. When the electric energy data is 4 bytes, the width of the current compressed single group of data is 2 bytes.

Bit0 to bit1 of 0x52 is 0x02, indicating no compression;

Bit 2 to bit 3 of 0x52 are 0x00, indicating full compression;

Bit 4 to bit 5 of 0x52 are 0x01, indicating differential compression;

Bit6 to bit7 of 0x52 are 0x01, indicating differential compression;

Bit0 to bit1 of 0x01 are 0x01, indicating differential compression.

Then use the CRC-CCITT verification algorithm to verify the Data0 data, completing a complete data compression process.

The present invention also proposes a data compression system 200, as shown in FIG2 , comprising:

The data difference generating unit 201 obtains the short descriptor of the data to be compressed, determines the forward active electric energy data block that needs to be compressed according to the short descriptor of the data to be compressed, determines the data Data0 to be transmitted and the data Data1 transmitted last time in the forward active electric energy data block that needs to be compressed, and performs a difference between Data1 and Data0 to generate Data2;

A compression header information generating unit 202 generates compression header information of the data Data0 to be transmitted according to Data2;

The verification data generating unit 203 generates CRC verification data according to Data0; uses the CRC16 verification data generated by Data0 data as additional verification data, and the attached verification data is provided to the receiver of the compressed data to verify the correctness of the received compressed data.

The compressed data generating unit 204 generates compressed data of the data Data0 to be transmitted using Data2, the compressed header information and the CRC check data;

Wherein, after the compressed data is generated, the transmission protocol used for external transmission includes: transmitting in a manner of requesting compressed transmission or responding to compressed transmission;

If the transmission method is to request compressed transmission, the generated compressed data is transmitted after carrying the compression algorithm, compression mode information and the processing result of the compressed data.

Wherein, the compression method of the compressed data is lossless difference data compression;

If the compression method of the compressed data cannot use lossless difference data compression, the compression method of the compressed data uses lossless data compression, and the lossless data compression compresses CRC check data and Data0 to generate compressed data.

The compression header information includes: the compression header byte length and the field compression bit;

The field compression bit uses two digits to indicate the data block compression method. 0 indicates lossless data compression, 1 indicates lossless differential data compression, and 2 indicates no compression.

The short descriptor includes: object identifier, object feature, object attribute identifier and element index within the attribute.

The verification data generating unit is further used for: after using the CRC verification data generated by the Data0 data, generating CRC16 verification data as additional verification data according to the CRC verification data.

The present invention improves data transmission efficiency and bandwidth utilization, and stores data efficiently.

It will be appreciated by those skilled in the art that embodiments of the present invention may be provided as methods, systems, or computer program products. Therefore, the present invention may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program codes. The schemes in the embodiments of the present invention may be implemented in various computer languages, for example, object-oriented programming language Java and literal scripting language JavaScript, etc.

The present invention is described with reference to the flowchart and/or block diagram of the method, device (system), and computer program product according to the embodiment of the present invention. It should be understood that each process and/or box in the flowchart and/or block diagram, as well as the combination of the process and/or box in the flowchart and/or block diagram can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device produce a device for implementing the functions specified in one or more processes in the flowchart and/or one or more boxes in the block diagram.

These computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable data processing device to work in a specific manner, so that the instructions stored in the computer-readable memory produce a manufactured product including an instruction device that implements the functions specified in one or more processes in the flowchart and/or one or more boxes in the block diagram.

These computer program instructions may also be loaded onto a computer or other programmable data processing device so that a series of operational steps are executed on the computer or other programmable device to produce a computer-implemented process, whereby the instructions executed on the computer or other programmable device provide steps for implementing the functions specified in one or more processes in the flowchart and/or one or more boxes in the block diagram.

Although the preferred embodiments of the present invention have been described, those skilled in the art may make other changes and modifications to these embodiments once they have learned the basic creative concept. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments and all changes and modifications that fall within the scope of the present invention.

Obviously, those skilled in the art can make various changes and modifications to the present invention without departing from the spirit and scope of the present invention. Thus, if these modifications and variations of the present invention fall within the scope of the claims of the present invention and their equivalents, the present invention is also intended to include these modifications and variations.

Claims (8)

1. A data compression method, the method comprising: Obtain a short descriptor of the data to be compressed, determine the forward active electric energy data block of the data to be compressed according to the short descriptor of the data to be compressed, determine the data Data0 to be transmitted and the data Data1 transmitted last time in the forward active electric energy data block of the data to be compressed, and perform a difference between Data1 and Data0 to generate Data2; The short descriptor includes: object identification, object characteristics, object attribute identification and element index within the attribute; Generate compression header information of the data Data0 to be transmitted according to Data2; The compression header information includes: compression header byte length and field compression bit; The field compression bit uses two digits to indicate the data block compression method, 0 indicates lossless data compression, 1 indicates lossless difference data compression, and 2 indicates no compression; Generate CRC check data according to Data0; The compressed data of the data Data0 to be transmitted is generated using Data2, the compressed header information and the CRC check data. 2. According to the method of claim 1, after the compressed data is generated, the transmission protocol used for external transmission includes: transmitting in a manner of requesting compressed transmission or responding to compressed transmission; If the transmission method is to request compressed transmission, the generated compressed data is transmitted after carrying the compression algorithm, compression mode information and the processing result of the compressed data. 3. The method according to claim 1, wherein the compression method of the compressed data is lossless difference data compression; If the compression method of the compressed data cannot use lossless difference data compression, the compression method of the compressed data uses lossless data compression, and the lossless data compression compresses CRC check data and Data0 to generate compressed data. 4. The method according to claim 1 further comprises: after using Data0 data to generate CRC check data, generating CRC16 check data as additional check data according to the CRC check data. 5. A data compression system, the system comprising: Generate a data difference unit, obtain a short descriptor of the data to be compressed, determine the forward active electric energy data block that needs to be compressed according to the short descriptor of the data to be compressed, determine the data Data0 that needs to be transmitted and the data Data1 that was transmitted last time in the forward active electric energy data block that needs to be compressed, and make a difference between Data1 and Data0 to generate Data2; The short descriptor includes: object identification, object characteristics, object attribute identification and element index within the attribute; A compression header information generating unit generates compression header information of the data Data0 to be transmitted according to Data2; The compression header information includes: compression header byte length and field compression bit; The field compression bit uses two digits to indicate the data block compression method, 0 indicates lossless data compression, 1 indicates lossless difference data compression, and 2 indicates no compression; A check data generating unit generates CRC check data according to Data0; The compressed data generating unit generates compressed data of the data Data0 to be transmitted using Data2, the compressed header information and the CRC check data. 6. The system according to claim 5, after the compressed data is generated, the transmission protocol used for external transmission includes: transmitting in a manner of requesting compressed transmission or responding to compressed transmission; If the transmission method is to request compressed transmission, the generated compressed data is transmitted after carrying the compression algorithm, compression mode information and the processing result of the compressed data. 7. The system according to claim 5, wherein the compression method of the compressed data is lossless difference data compression; If the compression method of the compressed data cannot use lossless difference data compression, the compression method of the compressed data uses lossless data compression, and the lossless data compression compresses CRC check data and Data0 to generate compressed data. 8. The system according to claim 5, wherein the verification data generating unit is further used to: after using Data0 data to generate CRC verification data, generate CRC16 verification data as additional verification data according to the CRC verification data.