CN117950598B - Intelligent storage method for design data of electronic product - Google Patents

Abstract

The invention relates to the technical field of data compression, in particular to an intelligent storage method of design data of an electronic product, which comprises the steps of determining correction necessary parameters when judging conditions are met whenever the current character to be compressed in a preceding buffer area is not matched with the character in a searching buffer area in the process of dictionary coding compression of data to be stored, judging whether the correction conditions are met or not, modifying the character in the preceding buffer area if the correction conditions are met, continuing dictionary coding compression of the modified preceding buffer area, and finally obtaining a first compressed file; determining a correction coefficient sequence according to the difference of characters in the advance buffer area before and after modification, and performing probability coding compression on the correction coefficient sequence to obtain a second compressed file; the first compressed file and the second compressed file are stored. The invention effectively improves the data compression effect and reduces the occupied storage space of the data.

Description

A method for intelligent storage of electronic product design data

Technical Field

The present invention relates to the technical field of data compression, and in particular to an intelligent storage method for electronic product design data.

Background technique

With the development of electronic products, their designs are becoming more and more sophisticated, their functions are becoming more and more complex, the design data is increasing, and the storage space occupied is also increasing. Therefore, the storage space control of design data has become a major issue in electronic product data management. In order to reduce the storage space of design data, it is an inevitable choice to compress the design data.

Traditional LZ77 coding is a lossless coding and can be used to compress the design data of electronic products, thereby effectively reducing the storage space of design data. However, in the process of using LZ77 coding for data compression, the data compression effect is affected by the matching degree between the data in the first buffer and the data in the search buffer. When the continuity of data matching is higher, the data compression effect is better. However, when the continuity of data matching is worse, using LZ77 coding for data encoding compression will increase the encoding length of the data, resulting in poor data compression effect, which ultimately affects the storage of design data.

Summary of the invention

The purpose of the present invention is to provide an intelligent storage method for electronic product design data, which is used to solve the problem that the data occupies a large storage space due to poor data compression effect.

To solve the above technical problems, the present invention provides an electronic product design data intelligent storage method, comprising the following steps:

Acquire the data to be stored, perform dictionary coding compression on the data to be stored, and during the dictionary coding compression process, whenever the current character to be compressed in the advance cache area does not match the character in the search cache area, determine whether a determination condition is met, and if the determination condition is met, determine the necessary correction parameters according to the character lengths of all characters between the current character to be compressed and the corresponding character when the mismatch occurred last time, and the character lengths of continuous characters in the advance cache area that match the characters in the search cache area;

According to the correction necessary parameters, it is determined whether the correction condition is met. If the correction condition is met, the characters in the preceding cache area are modified according to the characters in the search cache area, and the modified preceding cache area is further compressed by dictionary coding to finally obtain a first compressed file;

Determine the correction coefficient corresponding to each character in the preceding cache area according to the difference between the characters in the preceding cache area before and after the modification, thereby obtaining a correction coefficient sequence corresponding to the data to be stored, and perform probability coding compression on the correction coefficient sequence, thereby obtaining a second compressed file;

The first compressed file and the second compressed file are stored.

Furthermore, the calculation formula corresponding to the necessary correction parameters is determined as follows:

; Among them, /> Indicates necessary parameters for correction; /> Indicates the length of all characters between the current character to be compressed and the corresponding character when it did not match the last time; /> Indicates the character length of the continuous characters in the look-ahead buffer area that match the characters in the look-up buffer area; /> Represents the normalization function.

Furthermore, the characters in the advance buffer area are modified, including:

The continuous characters that first appear in the advance cache area and match the characters in the search cache area are determined as replacement characters, and all characters before the replacement characters in the advance cache area that do not match the characters in the search cache area are determined as replaced characters, and the positions of the replacement characters and the replaced characters are replaced to obtain a modified advance cache area.

Furthermore, the characters in the advance buffer area are modified, including:

The first consecutive characters that appear in the advance cache area and match the characters in the search cache area are determined as target characters, and all characters before the target characters in the advance cache area that do not match the characters in the search cache area are determined as modified characters, and the modified characters are modified to the characters existing in the search cache area.

Further, the correction coefficient corresponding to each character in the advance cache area is determined, and the corresponding calculation formula is:

; Among them, /> Indicates the correction coefficient corresponding to the nth character in the advance buffer area;/> Indicates the data value of the nth character in the advance buffer area after correction; /> Indicates the data value of the nth character in the advance buffer area before correction is made.

Furthermore, the determination condition includes at least: the character length of all characters between the current character to be compressed and the corresponding character when there was no match last time is less than a first set character length threshold, and the character length in the search cache area is not less than a second set character length threshold.

Furthermore, the correction condition at least includes: the correction necessary parameter is greater than a set parameter threshold.

Furthermore, LZ77 encoding is used for dictionary encoding compression.

Furthermore, when the determination condition is not met, dictionary coding compression is continued.

Furthermore, run-length coding is used to perform probability coding compression on the correction coefficient sequence, thereby obtaining a second compressed file.

The present invention has the following beneficial effects: in the process of dictionary coding compression of data to be stored, whenever a data interruption occurs, that is, when the current characters to be compressed in the advance cache area do not match the characters in the search cache area, the present invention determines whether a determination condition is met, that is, whether it is necessary to modify the characters at the location where the data interruption occurs. If the determination condition is met, the necessary degree of modifying the characters at the location where the data interruption occurs is measured, and the necessary correction parameters are determined. When it is necessary to modify, the characters in the advance cache area are modified according to the characters in the search cache area to increase the length of the characters that match the characters in the search cache area at the location where the data interruption occurs in the advance cache area, and the modified advance cache area is further subjected to dictionary coding compression, and finally a first compressed file is obtained. At the same time, in order to facilitate the subsequent decompression of the first compressed file obtained after compression and finally obtain the original data, after each modification of the characters in the advance cache area, the correction coefficient sequence corresponding to the data to be stored is determined according to the difference between the characters in the advance cache area before and after the modification. Since only part of the characters in the data to be stored are modified, a large number of repeated characters are contained in the correction coefficient sequence, so the correction coefficient sequence is subjected to probability coding compression, and finally a second compressed file is obtained. The present invention modifies the characters at the data interruption locations during the dictionary coding compression process of the stored data, thereby effectively avoiding the need for the existing LZ77 coding to construct multiple triplets for the data at the data interruption locations in the advance buffer area, which in turn causes the problem of increased data volume after compression. The present invention effectively improves the data compression effect, reduces the storage space occupied by data, and realizes efficient data storage.

BRIEF DESCRIPTION OF THE DRAWINGS

In order to more clearly illustrate the technical solutions and advantages in the embodiments of the present invention or the prior art, the drawings required for use in the embodiments or the prior art descriptions are briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

FIG1 is a flow chart of a method for intelligently storing electronic product design data according to an embodiment of the present invention;

FIG. 2 is a schematic diagram of a window according to an embodiment of the present invention.

Detailed ways

In order to further explain the technical means and effects adopted by the present invention to achieve the predetermined invention purpose, the specific implementation methods, structures, features and effects of the technical solutions proposed by the present invention are described in detail below in conjunction with the accompanying drawings and preferred embodiments. In the following description, different "one embodiment" or "another embodiment" does not necessarily refer to the same embodiment. In addition, specific features, structures or characteristics in one or more embodiments may be combined in any suitable form.

Unless otherwise defined, all technical and scientific terms used herein have the same meaning as those commonly understood by those skilled in the art of the present invention. In addition, all parameters or indicators in the formulas referred to herein are normalized values that eliminate the influence of dimensions.

In order to solve the problem that data occupies a large storage space due to poor data compression effect, this embodiment provides an electronic product design data intelligent storage method, the corresponding flowchart of the method is shown in FIG1, and includes the following steps:

(1) Obtaining data to be stored and performing dictionary coding compression on the data to be stored. During the dictionary coding compression process, whenever a current character to be compressed in the advance cache area does not match a character in the search cache area, determining whether a determination condition is satisfied. If the determination condition is satisfied, determining necessary correction parameters based on the character lengths of all characters between the current character to be compressed and the corresponding character when the mismatch last occurred, and the character lengths of consecutive characters in the advance cache area that match the characters in the search cache area.

First, the data to be stored is obtained. In this embodiment, the data to be stored refers to the electronic product design data. After the electronic product design data is obtained, the data is pre-processed for data cleaning, such as correcting errors, missing data, and abnormal data in the data, improving the accuracy and consistency of the data, and eliminating noise and invalid information, so that the data used for subsequent data compression is more complete and accurate, which is convenient for subsequent compression. Of course, as other implementation methods, when the obtained electronic product design data itself is complete and accurate, there is no need to perform this pre-processing process at this time.

In order to achieve coding compression of pre-processed electronic product design data, the existing LZ77 coding is first used to determine the coding window size of the coding. The existing LZ77 coding usually uses 4096 bytes as the coding window size of the data, and divides the coding window into two parts, as shown in Figure 2, the left part is the search cache area when the data is encoded, and the right part is the advance cache area. Since a smaller coding window may cause the LZ77 coding to be unable to find a longer match, thereby limiting the compression performance; and a larger coding window may cause the search and matching process of the LZ77 coding to slow down, increasing the computational burden of compression, so for electronic product design data, this embodiment selects a window size of 8 bytes as the coding window size for subsequent coding compression.

After determining the size of the coding window, the existing LZ77 coding is first used to perform dictionary coding compression on the pre-processed electronic product design data. During the coding compression process, the coding window will slide in the electronic product design data and perform sliding coding on the data. During the sliding coding process, every time the data in the advance cache area does not match the data in the search cache area, that is, the current character to be compressed in the advance cache area does not match the character in the search cache area, it is considered that a data interruption has occurred. In other words, when a data interruption occurs, it means that the current character to be compressed in the advance cache area cannot be found in the search cache area. At this time, if the LZ77 coding is used for encoding, the coding length will become longer when the data is re-encoded, resulting in a worse data compression effect.

In order to improve the data compression effect, every time a data interruption occurs, the character length of all characters between the current character to be compressed and the corresponding character when the previous mismatch occurred in the advance cache area is obtained. , that is, to obtain the character length of all characters between the last data interruption and the current data interruption/> . If data interruption occurs for the first time during the encoding process, the character length/> Refers to the character length of all characters before the current data interruption. When the character length/> The larger the value, the better the compression effect of the current data using LZ77 encoding. Even if data interruption occurs, LZ77 encoding can still be used to continue compression. When the character length is / > When it is smaller, it means that the effect of compressing the current data using LZ77 encoding is worse. At this time, LZ77 encoding needs to be improved to improve the data compression effect.

Based on the above analysis, the character length of all characters between the current character to be compressed and the corresponding character when it did not match the last time in the cache area is obtained. Afterwards, it is necessary to determine whether the determination condition is met according to the character length of all characters between the current character to be compressed and the corresponding character when the previous character does not match, and the character length in the search cache area. In this embodiment, the determination condition is: the character length of all characters between the current character to be compressed and the corresponding character when the previous character does not match is less than the first set character length threshold, and the character length in the search cache area is not less than the second set character length threshold. The size of the first set character length threshold can be selected and set based on experience. In this embodiment, the value of the set character length threshold is set to 10. Considering that it is convenient to modify the characters in the advance cache area later, the value of the second set character length threshold is set to the length of the search cache area, that is, the maximum number of characters that can be accommodated in the search cache area. When the determination condition is not met, LZ77 encoding is continued to be used for dictionary encoding compression. At this time, no judgment is made on the subsequent data, that is, it is not necessary to judge whether the characters in the advance cache area need to be modified; and when the determination condition is met, it is necessary to judge the subsequent data to determine whether the characters in the advance cache area need to be modified.

In order to determine whether to modify the characters in the advance buffer area, the character length of the continuous characters in the advance buffer area that match the characters in the search buffer area after the data interruption occurs is obtained. That is, the current character to be compressed that does not match the character in the search cache area is skipped in the advance cache area, and character matching is continued until a single or continuous character that matches the character in the search cache area is found, and the character length of the matched single or continuous character is recorded as/> When no single or continuous characters matching the characters in the search cache area are found in the look-ahead cache area, the character length is considered to be The value of is 0.

On this basis, the necessary correction parameters are determined according to the character lengths of all characters between the current character to be compressed and the corresponding character when the mismatch occurred last time, and the character lengths of the continuous characters in the advance cache area that match the characters in the search cache area. The corresponding calculation formula is:

; Among them, /> Indicates necessary parameters for correction; /> Indicates the length of all characters between the current character to be compressed and the corresponding character when it did not match the last time; /> Indicates the character length of the continuous characters in the look-ahead buffer area that match the characters in the look-up buffer area; /> Represents the normalization function.

In the calculation formula for the above correction necessary parameters, the character length of all characters between the current character to be compressed and the corresponding character when it did not match the last time The smaller the value of is, the worse the effect of compressing the current data using LZ77 encoding is. When the character length of the continuous characters in the look-up cache area that match the characters in the look-up cache area is greater than the character length of the continuous characters in the look-up cache area, the smaller the value of is. The larger the value of , the more it means that after the unmatched current characters to be compressed are modified according to the characters in the lookup cache area, the number of character bits in the advance cache area that match the characters in the lookup cache area can be effectively increased, thereby reducing the number of triples constructed when using LZ77 encoding. The better the effect of LZ77 encoding compression on the data, the more the unmatched current characters to be compressed should be modified, and the larger the value of the corresponding correction necessary parameter will be.

(2) judging whether the correction condition is satisfied according to the correction necessary parameters; if the correction condition is satisfied, modifying the characters in the preceding cache area according to the characters in the search cache area, and continuing to perform dictionary encoding compression on the modified preceding cache area, and finally obtaining a first compressed file.

After determining the necessary correction parameters through the above steps, determine whether the correction conditions are met based on the necessary correction parameters. In this embodiment, the correction condition is: the necessary correction parameters are greater than the set parameter threshold. The set parameter threshold can be reasonably set based on experience. In this embodiment, the value of the set parameter threshold is set to 0.7. When the correction condition is not met, there is no need to modify the characters in the advance cache area, and LZ77 encoding continues to be used for dictionary encoding compression. When the correction condition is met, it is necessary to modify the characters in the advance cache area based on the characters in the search cache area.

Optionally, the characters in the preceding cache area are modified according to the characters in the search cache area, and the implementation steps include:

The continuous characters that first appear in the advance cache area and match the characters in the search cache area are determined as replacement characters, and all characters before the replacement characters in the advance cache area that do not match the characters in the search cache area are determined as replaced characters, and the positions of the replacement characters and the replaced characters are replaced to obtain a modified advance cache area.

For ease of understanding, when modifying the characters in the advance cache area, the characters in the advance cache area are matched with the characters in the search cache area in order, so as to determine the first occurrence of the continuous matching characters, and use these characters as replacement characters. At the same time, all characters before the replacement character in the advance cache area are determined as the replaced characters.

The positions of the replacement character and the replaced character are changed to obtain a modified advance cache area. For example, the characters in the advance cache area are EFABG, wherein the character EF does not match the characters in the search cache area, that is, the search cache area does not contain characters E and F, and thus the data interruption referred to above occurs. The character AB matches the characters in the search cache area, that is, the search cache area contains characters A and B. At the same time, the character G does not match the characters in the search cache area either. At this time, the character AB is determined as the replacement character, and the character EF is determined as the replaced character, and the positions of the characters EF and AB are swapped, and the characters in the modified advance cache area are ABEFG. By modifying the characters in the advance cache area in this way, the effect of subsequent data encoding and compression can be effectively improved.

Optionally, as another implementation, the characters in the preceding cache area are modified according to the characters in the search cache area, and the implementation steps include:

The first consecutive characters that appear in the advance cache area and match the characters in the search cache area are determined as target characters, and all characters before the target characters in the advance cache area that do not match the characters in the search cache area are determined as modified characters, and the modified characters are modified to the characters existing in the search cache area.

For ease of understanding, when modifying the characters in the look-ahead buffer area, the first consecutive matching characters in the look-ahead buffer area are determined in the same manner as described above, and for ease of distinction, these characters are referred to as target characters. At the same time, all characters before the target characters in the look-ahead buffer area are referred to as modifying characters. At this time, the modifying characters are modified according to the characters in the look-up buffer area, so that a longer consecutive character matching the characters in the look-up buffer area appears in the look-ahead buffer area as much as possible.

Among them, when modifying the modified character, if the string composed of these target characters exists in the search cache area, and the length of the characters before the string is not less than the character length of the modified character, the modified character is directly replaced with the character before the string and the character length is equal to the character length of the modified character. After the replacement, the string composed of the target character and all the characters before it in the advance cache area will appear in the search cache area. For example, for the character EFABG in the advance cache area, without considering the window size, if the characters in the search cache area are ABABC at the same time, the characters A and B before the string AB composed of the second appearing characters A and B in the search cache area are directly used to replace the modified character EF in the advance cache area. After the replacement, the characters in the advance cache area are ABABG. If the string composed of these target characters exists in the search cache area, but the length of the characters before the string is less than the character length of the modified character, the characters before the target character in the advance cache area are directly replaced with all the characters before the string. For example, for the character EFABG in the advance cache area, if the character in the search cache area is BABC, the character B before the character string AB composed of the characters A and B in the search cache area is used to replace the modified character EF in the advance cache area. After the replacement, the characters in the advance cache area are EBABG. If the character string composed of these target characters exists in the search cache area, but there is no character before the character string, the modified character is directly replaced with the character in the search cache area that is located before the modified character and has a character length equal to the character length of the modified character. For example, for the character EFABG in the advance cache area, if the character in the search cache area is ABCD, the character CD in the search cache area is used to replace the modified character EF in the advance cache area. After the replacement, the characters in the advance cache area are CDABG.

Of course, when modifying the modified characters, if the character string consisting of these target characters does not exist in the search cache area, and only the front part of the target characters exists, then in the same way as above, the character string before the part of the characters in the search cache area is used to replace the characters before the target characters in the advance cache area, so that after the replacement, as long as possible continuous characters matching the characters in the search cache area appear in the advance cache area.

After the characters in the advance cache area are modified in the above manner, LZ77 encoding is continued to be used for dictionary coding compression based on the modified advance cache area. When data interruption occurs again next time, the characters in the advance cache area are modified again in the same manner as above, and LZ77 encoding is continued to be used for dictionary coding compression after modification until the compression is completed, and finally the first compressed file is obtained.

(3) Determine the correction coefficient corresponding to each character in the advance cache area according to the difference between the characters in the advance cache area before and after the modification, thereby obtaining a correction coefficient sequence corresponding to the data to be stored, and perform probability coding compression on the correction coefficient sequence to obtain a second compressed file.

In order to facilitate the subsequent decompression of the first compressed file obtained after compression and finally obtain the original data, after each modification of the characters in the advance cache area, it is necessary to determine the correction coefficient corresponding to each character in the advance cache area according to the difference between the characters in the advance cache area before and after the modification, so as to finally obtain the correction coefficient corresponding to each character in the data to be stored. That is, by comparing the set of modified data with the set of original data, the correction coefficient between the modified data and the original data is obtained, and the corresponding calculation formula is:

; Among them, /> Indicates the correction coefficient corresponding to the nth character in the advance buffer area;/> Indicates the decimal value of the nth character in the advance buffer area converted into ASCII code after correction;/> It means that the nth character in the buffer area is converted into the decimal value of the ASCII code before correction. The decimal value does not contain zero and can be directly used as the denominator.

The correction coefficient corresponding to each character in the data to be stored can be determined in the same manner as the correction coefficient corresponding to each character in the advance cache area, and a correction coefficient sequence can be obtained according to the arrangement order of the characters in the data to be stored. The correction coefficient corresponding to the unmodified character in the correction coefficient sequence is 1. Since only some characters in the data to be stored are modified, there are more correction coefficients that are continuously 1 in the correction coefficient sequence. Therefore, run-length coding is used to perform probability coding compression on the correction coefficient sequence, and the data with a high continuous repetition rate in the correction coefficient sequence is uniformly encoded to reduce data redundancy, and finally obtain a second compressed file.

(4) Storing the first compressed file and the second compressed file.

After the first compressed file and the second compressed file are obtained through the above steps, the first compressed file and the second compressed file are stored and transmitted. When the data needs to be decompressed, the first compressed file is decoded by using the decoding method of LZ77 encoding, and the second compressed file is decoded by using the decoding method of run-length encoding, and the two decoding results are combined to finally obtain the original data for use. Since the decompression process is opposite to the above data compression process, and the data compression process has been described in detail above, the decompression process will not be repeated here.

The present invention improves the character compression efficiency of LZ77 encoding at the data interruption point by modifying the characters at the data interruption point during the dictionary coding compression process of the stored data, effectively avoiding the problem that the existing LZ77 encoding needs to construct multiple triplets for the data at the data interruption point in the advance buffer area, which instead causes the data volume to increase after compression, effectively improving the data compression effect, reducing the storage space occupied by data, and realizing efficient data storage.

It should be noted that the above-described embodiments are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. These modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present application, and should all be included in the protection scope of the present application.

Claims (6)

1. A method for intelligent storage of electronic product design data, characterized in that it comprises the following steps: Acquire the data to be stored, perform dictionary coding compression on the data to be stored, and during the dictionary coding compression process, whenever the current character to be compressed in the advance cache area does not match the character in the search cache area, determine whether a determination condition is met, and if the determination condition is met, determine the necessary correction parameters according to the character lengths of all characters between the current character to be compressed and the corresponding character when the mismatch occurred last time, and the character lengths of continuous characters in the advance cache area that match the characters in the search cache area; According to the correction necessary parameters, it is determined whether the correction condition is met. If the correction condition is met, the characters in the preceding cache area are modified according to the characters in the search cache area, and the modified preceding cache area is further compressed by dictionary coding to finally obtain a first compressed file; Determine the correction coefficient corresponding to each character in the preceding cache area according to the difference between the characters in the preceding cache area before and after the modification, thereby obtaining a correction coefficient sequence corresponding to the data to be stored, and perform probability coding compression on the correction coefficient sequence, thereby obtaining a second compressed file; Storing the first compressed file and the second compressed file; The calculation formula for determining the necessary correction parameters is: ; Among them, /> Indicates necessary parameters for correction; /> Indicates the length of all characters between the current character to be compressed and the corresponding character when it did not match the last time; /> Indicates the character length of the continuous characters in the look-ahead buffer area that match the characters in the look-up buffer area; /> represents the normalization function; Modify the characters in the look-ahead buffer area, including: Determine the first occurrence of the continuous characters in the advance cache area that match the characters in the search cache area as replacement characters, and determine all characters in the advance cache area that precede the replacement characters and do not match the characters in the search cache area as replaced characters, and replace the positions of the replacement characters and the replaced characters, thereby obtaining a modified advance cache area; Modify the characters in the look-ahead buffer area, including: Determine the first consecutive characters in the advance cache area that match the characters in the search cache area as target characters, and determine all characters before the target characters in the advance cache area that do not match the characters in the search cache area as modified characters, and modify the modified characters to the characters existing in the search cache area; Determine the correction coefficient corresponding to each character in the advance cache area. The corresponding calculation formula is: ; Among them, /> Indicates the correction coefficient corresponding to the nth character in the advance cache area;/> Indicates the data value of the nth character in the advance buffer area after correction; /> Indicates the data value of the nth character in the advance buffer area before correction is made. 2. According to claim 1, a method for intelligent storage of electronic product design data is characterized in that the judgment conditions include at least: the character length of all characters between the current character to be compressed and the corresponding character when it did not match the last time is less than a first set character length threshold, and the character length in the search cache area is not less than a second set character length threshold. 3. The method for intelligently storing electronic product design data according to claim 1, wherein the correction condition at least includes: a correction-necessary parameter being greater than a set parameter threshold. 4. The method for intelligently storing electronic product design data according to claim 1, characterized in that LZ77 encoding is used for dictionary encoding compression. 5. The method for intelligently storing electronic product design data according to claim 1, wherein when the judgment condition is not met, dictionary coding compression is continued. 6. The method for intelligent storage of electronic product design data according to claim 1 is characterized in that the correction coefficient sequence is probability-coded and compressed using run-length coding to obtain a second compressed file.