CN120068173A - Cold storage management method with data protection function - Google Patents

Abstract

The invention discloses a cold storage management method with a data protection function, which relates to the technical field of cold storage management methods and comprises the following steps of evaluating a storage environment, acquiring temperature and humidity data by using a plurality of sensors, and analyzing and alarming; the data multi-layer encryption comprises an AES algorithm and an RSA algorithm, storage layout planning according to data characteristics, redundancy setting, strict access control and record, regular data integrity checking and repairing, and data migration ensuring safety and relevant information updating. The process comprises prediction analysis and confusion processing optimization. The invention ensures the safety of cold storage data, including environmental monitoring, encryption, reasonable layout, access control, integrity check and efficient migration, avoids data loss and damage, improves access efficiency, can track abnormality, and enhances the overall data protection capability.

Description

Cold storage management method with data protection function

Technical Field

The invention relates to the technical field of cold storage management systems, in particular to a cold storage management method with a data protection function.

Background

With the rapid development of information technology, the volume of data has increased explosively, wherein large amounts of data need to be stored for a long period of time to meet compliance, history, and backup recovery requirements. Cold storage is widely used in the fields of finance, medical treatment, scientific research and government as a data storage mode due to the characteristics of low power consumption, high storage density and suitability for long-term archiving.

In a cold storage environment, the security and integrity of data presents a number of challenges. First, the physical environment in which the cold storage device is located has a critical impact on data storage. Abnormal changes in temperature and humidity may cause damage to the storage medium, such as the tape getting wet and moldy, and the hard disk head being damaged by excessive temperature. Magnetic field disturbances can destroy data stored on the magnetic media, and vibrations can cause mechanical failure or head misalignment within the storage device, which can cause data loss or corruption. However, many cold storage systems currently lack accurate monitoring and effective response mechanisms to physical environments, or are not monitored sufficiently comprehensively to discover potential environmental threats in time.

Second, data encryption is critical in cold storage. Because cold-stored data often contains large amounts of sensitive information, such as corporate financial data, patient medical records, scientific research secrets, the consequences of data leakage are not envisaged once. Traditional encryption methods may have vulnerabilities or be inadequate in key management. And part of the system adopts only a single encryption mode, so that the system is easy to crack, and the encrypted secret key is possibly lost or stolen due to insufficient storage and protection measures of the secret key, so that the data is not protected.

Furthermore, there are problems with data management in cold storage. The data storage layout is unreasonable, and the differences of the type, the access frequency and the importance of the data are not fully considered, so that the data access efficiency is low. For example, frequently accessed data may be stored deep in the storage device, increasing access latency. Meanwhile, in the aspect of data access control, the authority management is not strict and fine enough, and unauthorized access is easy to occur. Moreover, existing access logging mechanisms are not reliable enough and it is difficult to effectively audit and track abnormal access behavior.

In addition, data integrity checking and data migration are weak links in cold storage management. The data may have integrity problems due to aging of storage media, physical damage or software errors in the long-term storage process, but the current integrity checking method may not be timely and accurate, and the data repairing capability after the problem is found is limited. In a data migration scenario, when a storage device is upgraded or the storage capacity is insufficient and data is required to be migrated, the existing method is often low in efficiency, and the situation of data loss or damage is easy to occur.

In view of the foregoing, the existing cold storage management has many disadvantages in terms of data protection, and a cold storage management method with comprehensive and efficient data protection function is urgently needed to ensure the safety, integrity and accessibility of data.

Disclosure of Invention

The invention provides a cold storage management method with a data protection function, which aims to solve the problems in the prior art.

In order to achieve the above purpose, the invention adopts the following technical scheme that the cold storage management method with the data protection function comprises the following steps:

The method comprises the steps of carrying out comprehensive evaluation on the physical environment where cold storage equipment is located, distributing a temperature sensor, a humidity sensor, a Hall effect sensor and an acceleration sensor in a cold storage area to form a sensor network, collecting sensor data by a data acquisition system at a preset frequency, immediately triggering acquisition by vibration and abnormal magnetic field intensity, transmitting the acquired data to a data analysis unit, judging by the data analysis unit based on a preset environmental parameter threshold value, enabling the magnetic field intensity not to exceed a set Gaussian value, and immediately giving an alarm to a manager through an audible and visual alarm and a remote notification system;

A data encryption step, namely encrypting the data by adopting a multi-layer encryption algorithm before the data is stored in the cold storage equipment; firstly, carrying out initial encryption on data by using an AES-256 symmetric encryption algorithm, wherein a key is generated by a key generation algorithm based on a hash function and a pseudo-random number generator, dividing the data into data blocks with fixed sizes, independently encrypting each data block, carrying out multiple rounds of transformation on the data blocks in the encryption process, wherein each round comprises byte substitution, line shifting, column confusion and round key addition operation;

Classifying data according to the type, access frequency and importance of the data, wherein the data type is divided into structured data, semi-structured data and unstructured data, backup data of an enterprise financial database is stored in an inner layer area of cold storage equipment, the area is provided with an independent power supply and a redundant cooling system, an enterprise-level solid state disk storage medium is adopted, access delay is reduced, data safety is improved, backup data of recent business records are stored in an area at the opposite outer layer, a mechanical hard disk is used, meanwhile, a redundant area is set, the size of the redundant area is divided according to the total data amount and the importance proportion, checksum of the data is calculated by utilizing an SHA-256 hash algorithm, each data block corresponds to one checksum, and the checksum and the data are stored in the redundant area together;

the access control step comprises the steps of establishing a user access authorization mechanism, wherein a user needs to pass multi-factor identity verification, including user name and strong password, fingerprint identification and smart card identification;

The method comprises the steps of checking data integrity, namely checking the data integrity in cold storage at regular intervals, re-calculating an SHA-256 hash value of the data in a parallel calculation mode by taking a data block as a unit according to a checksum copy set in a storage layout plan, comparing the SHA-256 hash value with the stored checksum, starting data recovery operation if the data integrity problem is found, and carrying out intelligent restoration by combining the copy stored in a redundant area with associated information between the data block through a data recovery algorithm;

The method comprises the steps of data migration, namely when the storage capacity utilization rate of a cold storage device reaches 80% or a storage technology is updated, firstly, marking data to be migrated, realizing by adding migration marking bits in metadata of the data, suspending access operation to the data, then reading the data from the original cold storage device, adopting a high-speed network channel based on SSL/TLS encryption in the transmission process, simultaneously carrying out integrity check on the data again, carrying out hash check on 100MB data once, after the data is received by a new cold storage device, re-storing the data according to a new storage layout plan, determining the new layout according to the latest classification and importance evaluation result of the current data, updating access authority, adjusting according to the latest setting in a user authority management system, updating related metadata information, deleting or marking the data in the original cold storage device as migrated after the migration is completed, and notifying all modules in the system to resume normal access operation to the data through a broadcasting mechanism.

Further, in the storage environment assessment step, the machine learning algorithm is utilized to conduct prediction analysis on environment data, the long-term memory network LSTM algorithm is adopted, temperature, humidity, magnetic field intensity and vibration data of the past year are used as training sets, the training model is used for learning seasonal change, periodic fluctuation and characteristics of sudden abnormal conditions, the prediction performance of the model is optimized by continuously adjusting the neuron number, the layer number and the learning rate parameters of the model, in actual operation, the model predicts abnormal rise or fall of the temperature, abrupt change of the humidity, abnormal fluctuation of the magnetic field intensity and the occurrence of vibration events in advance for 1-2 hours, and preventive measures are started in time, including adjusting parameters of a cooling system, starting dehumidifying equipment or strengthening physical fixation of equipment.

Further, in the data encryption step, the encrypted data is subjected to confusion processing, after the data encryption is completed, random filling data with specific length is generated according to the size of a data block and the output result of an encryption algorithm, the generation of the filling data is based on a cryptographically safe pseudo-random number generator, the seed value of the pseudo-random number generator is determined by a plurality of factors of system time and a hardware identifier together, randomness is ensured, and the filling data is inserted into the encrypted data in a specific mode.

Further, in the step of data storage layout planning, a distributed storage technology is adopted to store data in a plurality of cold storage devices in a scattered manner, a Ceph distributed file system is utilized to manage the cold storage devices, data is uniformly distributed on each storage node through a consistent hash algorithm, each data block is divided into a plurality of sub-blocks during storage, the sub-blocks are stored in different cold storage devices according to hash values, meanwhile, redundant copies of the data are stored on different cold storage devices, the number of the copies is determined according to the importance of the data, when the cold storage devices fail, the data is quickly recovered from other devices stored with the copies, a heartbeat detection mechanism is established between the storage nodes, heartbeat signals are mutually transmitted every 10 seconds, if the nodes continuously receive no heartbeat signals for 3 times, the node failure is judged, and a data recovery and load balancing mechanism is automatically started.

Further, in the access control step, a blockchain technology is introduced to store the access records of the users in a non-tamperable manner, detailed information of each user access is used as a transaction record, each transaction record is linked by utilizing a hash chain structure of the blockchain, each block contains a plurality of transaction records, the integrity of the block is ensured by calculating the hash value of a block header, the consistency confirmation of nodes in the blockchain network to the transaction records is ensured by adopting a PoW workload certification or PoS rights proving consensus mechanism, and the access records are stored on the blockchain, so that any tampering of the access records needs to modify the subsequent blocks of the whole blockchain, and meanwhile, the access control strategy is automatically executed by utilizing the intelligent contract function of the blockchain, including automatically limiting the access rights of the users when the users try illegal access for a plurality of times continuously.

Further, in the step of checking the data integrity, when the data integrity problem is found, the data is intelligently repaired by combining a data recovery algorithm with redundant data, the data recovery algorithm adopts an error correction technology based on Reed-Solomoncode of Reed-Solomon codes, the technology can correct errors in data blocks within a certain range, for each data block, parameters of error correction codes are determined according to the importance and storage mode of each data block, when the data integrity problem is detected, the type and degree of the errors are analyzed first, if the errors are bit errors, the error correction codes are directly utilized for correction, if the errors are lost, the data is recovered by a data reconstruction algorithm according to the association relation between copies stored in a redundant area and the data blocks, and in the process of data reconstruction, the original content of the data blocks is gradually restored by combining the association of index information in a storage layout, the logic sequence of the data blocks and hash values, so that the repair efficiency is improved.

In the data migration step, in the process of migrating data, a mode of combining incremental backup and differential backup is adopted to reduce data transmission quantity and migration time, after marking data to be migrated, full backup is firstly carried out once as a basic version, then, for subsequent data change, newly added data blocks and changed data blocks are identified through log records of a file system and time stamp information of the data blocks, the data blocks form incremental backup, hash values of the same data blocks in original cold storage equipment and new cold storage equipment are compared, data blocks with differences are found out and serve as contents of differential backup, and during migration, only the data blocks of the incremental backup and the differential backup are transmitted.

Further, a system for implementing the cold storage management method with a data protection function is characterized by comprising:

the environment evaluation module is used for realizing the function of storing the environment evaluation step and comprises a sensor, a data acquisition system and an analysis alarm unit, wherein the sensor has a self-calibration function, the measurement precision of the sensor is calibrated regularly through a built-in calibration circuit and a standard reference source, in the calibration process, the temperature sensor is compared with a high-precision standard thermometer, the humidity sensor is calibrated with a standard humidity generator, and the Hall effect sensor and the acceleration sensor are calibrated through calibration equipment;

the data encryption module is used for executing a data encryption step, encrypting data by utilizing a multi-layer encryption algorithm and a key management mechanism, having an encryption algorithm updating function, periodically checking whether a new encryption algorithm vulnerability is issued or not by connecting with an external security updating server, automatically downloading and updating the encryption algorithm if a new security threat exists, adopting a double-key mechanism in the updating process, namely simultaneously reserving an old encryption key and a new encryption key for a period of time, and simultaneously using the new encryption algorithm for the newly stored data;

the storage layout planning module is responsible for planning a storage area and setting redundancy according to data characteristics, supporting dynamic adjustment of the storage layout, and reclassifying and planning storage positions of the data by using a clustering algorithm based on machine learning through monitoring access frequency, importance change and performance indexes of the storage equipment in real time;

The access control module establishes a user authorization and access recording mechanism according to the access control step, is integrated with an external identity management system, and realizes the butt joint with the existing user management system of an enterprise through standard LDAP and OAuth interface protocols;

The data integrity checking module completes the data integrity checking step, periodically checks data and restores the data when the data is in a problem, processes the integrity checking of a plurality of data blocks in parallel, distributes the number of threads according to the hardware resources of the system through a multithreading technology, simultaneously starts 8 threads to perform hash calculation and check sum comparison on the data blocks for the system with an 8-core CPU, and reduces the time of data reading and writing by utilizing the high-efficiency memory management and data caching technology in the data restoring process;

And the data migration module is used for realizing the migration of data among cold storage devices according to the data migration step, supporting breakpoint continuous transmission in the migration process, storing the information in a local temporary file or database by recording the transmission progress information of each data block in the migration process, and when the migration process is interrupted due to network failure and device restarting reasons, continuing to transmit from the position of last interruption according to the recorded progress information after the system is recovered.

Compared with the prior art, the invention has the beneficial effects that:

In the aspect of storage environment monitoring, the cold storage environment parameters can be accurately mastered by comprehensively deploying high-precision sensors and reasonable data acquisition frequency. By combining an advanced environment assessment mechanism, abnormal conditions of temperature, humidity, magnetic field and vibration can be found in time, early warning is carried out in advance, storage medium damage and data loss caused by environmental problems are effectively avoided, and physical safety of data storage is guaranteed.

In data encryption, a multi-layer encryption algorithm and perfect key management are adopted. The multi-layer encryption greatly improves the confidentiality of data, and even if one layer of encryption is cracked, other encryption protection exists. The strict key management ensures the key security, prevents data leakage and protects sensitive information.

For the data storage layout, the access efficiency is optimized according to the data characteristic planning. Important and low-frequency access data are stored in a high-safety area, the high-frequency access data are convenient to acquire, and meanwhile, the redundancy arrangement ensures the data integrity, so that the overall storage performance is improved.

Access control links, multi-factor authentication and fine rights management, preventing unauthorized access. And a reliable access record and audit mechanism is combined with a blockchain technology, so that record is guaranteed to be untampered, anomaly is effectively tracked, and data security is enhanced.

The data integrity check can accurately find out problems in time, and the intelligent repair algorithm is combined with the rapid repair of redundant data, so that the influence of data damage is reduced. When data is migrated, the transmission quantity and time are greatly reduced by combining incremental backup and differential backup, migration stability is ensured, and flexibility and reliability of cold storage management are improved.

Drawings

Fig. 1 is a schematic block diagram of a cold storage management method with data protection function according to the present invention.

Detailed Description

The following description of the embodiments of the present invention will be made clearly and completely with reference to the accompanying drawings, in which it is apparent that the embodiments described are only some embodiments of the present invention, but not all embodiments. All other embodiments, which can be made by those skilled in the art based on the embodiments of the invention without making any inventive effort, are intended to be within the scope of the invention.

In the description of the present invention, it should be understood that the terms "center", "longitudinal", "lateral", "length", "width", "thickness", "upper", "lower", "front", "rear", "left", "right", "vertical", "horizontal", "top", "bottom", "inner", "outer", "clockwise", "counterclockwise" indicate orientations or positional relationships are based on the orientations or positional relationships shown in the drawings, are merely for convenience in describing the present invention and simplify the description, and do not indicate or imply that the devices or elements referred to must have a specific orientation, be configured and operated in a specific orientation, and therefore should not be construed as limiting the present invention.

Furthermore, the terms "first," "second," and the like, are used for descriptive purposes only and are not to be construed as indicating or implying a relative importance or implicitly indicating the number of technical features indicated. Thus, a feature defining "a first" or "a second" may explicitly or implicitly include one or more of the described features. In the description of the present invention, the meaning of "a plurality" is two or more, unless explicitly defined otherwise. The terms "mounted," "connected," "coupled," and "connected" are used in a broad sense, and may be, for example, fixedly connected, detachably connected, or integrally connected, mechanically connected, electrically connected, directly connected, or indirectly connected via an intermediate medium, or may be in communication with the interior of two elements. The specific meaning of the terms in the present invention will be understood by those skilled in the art in detail, and the present invention will be further described in detail with reference to the accompanying drawings.

Referring to fig. 1, a cold storage management method with a data protection function includes the steps of:

And the storage environment assessment step is to comprehensively assess the physical environment in which the cold storage equipment is positioned, wherein the physical environment comprises temperature, humidity, magnetic field intensity and vibration condition parameters. And reasonably distributing a high-precision temperature sensor, a humidity sensor, a Hall effect sensor and an acceleration sensor in the cold storage area to form a sensor network. The precision of the temperature sensor can reach +/-0.1 ℃, the precision of the humidity sensor is +/-2% RH, and the accuracy of environmental data acquisition is ensured. The data acquisition system collects sensor data at a preset frequency, the acquisition frequency is set to be once every 2 hours for a large cold storage warehouse with relatively stable temperature, the sensor data are acquired every 30 minutes for an environment with a slightly rapid humidity change, and the acquisition can be triggered immediately due to abnormal vibration and magnetic field intensity. The collected data are transmitted to a data analysis unit, the unit judges based on a preset environmental parameter threshold, for example, the temperature threshold is set between-20 ℃ and 5 ℃, the humidity threshold is 30% -50% RH, the magnetic field intensity does not exceed a specific Gaussian value, and the vibration acceleration is smaller than a certain standard value. Once the threshold is exceeded, the system immediately alerts the manager via an audible and visual alarm and a remote notification system.

And the data encryption step is to encrypt the data by adopting a multi-layer encryption algorithm before the data is stored in the cold storage device. First, data is initially encrypted using an AES-256 symmetric encryption algorithm, the key of which is generated by a key generation algorithm based on a hash function and a pseudo-random number generator. The data is divided into data blocks with fixed sizes, each data block is independently encrypted, the data blocks are subjected to multiple rounds of transformation in the encryption process, and each round comprises byte substitution, row shifting, column confusion and round key addition operation. And then, encrypting the AES key by using a 2048-bit RSA asymmetric encryption algorithm, wherein the RSA algorithm ensures the randomness and the safety of prime numbers through a large prime number generation algorithm when generating public and private key pairs. The public key is distributed inside the system for encrypting the AES key, while the private key is stored in a key management center that employs multiple physical protections and access control, only authorized personnel can access through strict authentication.

And a data storage layout planning step, namely classifying the data according to the type, the access frequency and the importance of the data. Data types can be categorized into structured data (e.g., database files), semi-structured data (e.g., XML files), and unstructured data (e.g., images, video). Data with low access frequency but extremely high importance is backed up for an enterprise core financial database and is stored in an inner layer area of cold storage equipment, the area is provided with an independent power supply and a redundant cooling system, and a high-reliability storage medium such as an enterprise-level solid state disk is adopted to reduce access delay and improve data security. For data with a slightly higher access frequency of recent business record backups, the data is stored in a relatively outer area, and a high-capacity mechanical hard disk is used. Meanwhile, a redundant area is set, and the size of the redundant area is divided according to the total data amount and the importance according to a certain proportion, for example, the redundancy of important data is 30%. And calculating the checksum of the data by using the SHA-256 hash algorithm, wherein each data block corresponds to one checksum, and the checksum and the data are stored in a redundant area together, so that the integrity and the restorability of the data are ensured.

Access control step, setting up strict user access authorization mechanism. The user needs to pass multi-factor identity authentication, including user name and strong password (the password needs to contain case letters, numbers and special characters, and the length is not less than 8 bits), fingerprint identification (a high-resolution fingerprint sensor is adopted, the identification precision reaches 99.9%), and intelligent card identification (an encryption chip is arranged in the intelligent card and unique identity identification and encryption keys are stored). After the user inputs authentication information, the information is sent to the authentication server through a secure transport protocol (such as TLS protocol). The authentication server verifies according to pre-stored user information (including user basic material, authority level and history access record) and authority setting. For ordinary users, only the read right of public data is granted, for medium-level users, partial service related data can be read and written, and the high-level manager has complete read-write and management right of all data. Each access operation is recorded in an audit log that includes details of the accessing user, access time accurate to milliseconds, data content identification of the access (represented by a data hash value or unique identifier), and operation type (read, write, delete) for subsequent detailed audit and anomaly tracking.

And a data integrity checking step, namely periodically checking the integrity of the data in the cold storage. And re-calculating the SHA-256 hash value of the data by taking the data block as a unit according to the checksum copy set in the storage layout plan in a parallel calculation mode, and comparing with the stored checksum. For important data, the checking period is set to be carried out in a period with low system load of 2-4 points in the morning every day, and for general data, the checking period is set to be once a week. If the data integrity problem is found, starting the data recovery operation. And performing intelligent repair by a data recovery algorithm in combination with association information (such as logic sequence and index information of the data blocks) between the copies stored in the redundant area and the data blocks. And if the data is suspected to be in a software error, checking through a system log and an error report, and updating or repairing related software in time.

And a data migration step, namely migrating the data when the storage capacity utilization rate of the cold storage device reaches 80% or the storage technology is updated (such as new storage media appear and the cost performance is higher). First, the data to be migrated is marked by adding a migration marking bit to the data metadata and suspending access operations to these data. And then, reading the data from the original cold storage equipment, and adopting a high-speed network channel based on SSL/TLS encryption in the transmission process, wherein the transmission speed can reach more than 10 Gbps. Meanwhile, the integrity check is performed on the data again, and hash check is performed once every 100MB of data is transmitted. After receiving the data, the new cold storage device re-stores the data according to a new storage layout plan, and the new layout is determined according to the latest classification and importance evaluation result of the current data. Updating access rights, adjusting according to the latest settings in the user rights management system, and updating related metadata information such as data storage positions and access paths. After the migration is completed, deleting or marking the data in the original cold storage equipment as migrated, and informing all relevant modules in the system to resume the normal access operation of the data through a broadcasting mechanism.

In the storage environment evaluation step, the present invention uses a machine learning algorithm to perform predictive analysis on the environment data. Long and short term memory network (LSTM) algorithms are employed that are capable of learning a time-varying sequence pattern of environmental parameters. The temperature, humidity, magnetic field intensity and vibration data of the past year are used as training sets, and the training model learns the features of seasonal variation, periodic fluctuation and sudden abnormal conditions. The prediction performance of the model is optimized by continuously adjusting the neuron number, the layer number and the learning rate parameters of the model. In actual operation, the model may predict an abnormal rise or fall in temperature, a sudden change in humidity, an abnormal fluctuation in magnetic field strength, and possibly a vibration event 1-2 hours in advance in order to timely initiate preventive measures such as adjusting cooling system parameters, activating dehumidifying equipment, or enhancing physical fixation of equipment.

In the present invention, in the data encryption step, the encrypted data is subjected to confusion processing. After the data encryption is completed, random filling data with specific length is generated according to the size of the data block and the output result of the encryption algorithm. The generation of the filling data is based on a cryptographically secure pseudo-random number generator, the seed value of which is determined by a plurality of factors including system time and a hardware identifier, so that randomness is ensured. The stuffing data is inserted into the encrypted data in a specific mode, such as stuffing data with a certain length is inserted every fixed byte, and the content and the insertion position of the stuffing data dynamically change in each encryption process, so that the data security is further improved, and an attacker is difficult to crack the encryption by analyzing the data mode.

In the data storage layout planning step, the data are stored in a plurality of cold storage devices in a scattered manner by adopting a distributed storage technology. These cold storage devices are managed using a distributed file system (e.g., ceph) to evenly distribute data across the storage nodes via a consistent hashing algorithm. Each data block is split into a plurality of sub-blocks at storage, which are stored in different cold storage devices according to hash values. Meanwhile, redundant copies of data are stored on different cold storage devices, the number of the copies is determined according to the importance of the data, the number of important data copies is generally 3, and the number of key data copies can be set to be 5. In this way, when a certain cold storage device fails, data can be quickly recovered from other devices with copies stored therein, and the availability and failure resistance of the data are improved. And a heartbeat detection mechanism is established between the storage nodes, heartbeat signals are mutually sent every 10 seconds, if a certain node continuously receives no heartbeat signal for 3 times, the node is judged to be faulty, and a data recovery and load balancing mechanism is automatically started.

In the invention, in the access control step, a blockchain technology is introduced to store the access record of the user in a non-tamperable way. The detailed information (including access user, access time, accessed data content and operation type) of each user access is taken as a transaction record, and each transaction record is linked by using a hash chain structure of a blockchain. Each block contains a plurality of transaction records and the integrity of the block is ensured by calculating the hash value of the block header. A proof of work (PoW) or proof of equity (PoS) consensus mechanism is employed to ensure consistency validation of transaction records by nodes in a blockchain network. By storing the access records on the blockchain, any tampering with the access records requires modification of subsequent blocks of the entire blockchain, and the reliability of the audit is guaranteed due to extremely high computational cost. Meanwhile, by utilizing the intelligent contract function of the blockchain, some access control strategies can be automatically executed, such as the access authority is automatically limited when a user continuously tries illegal access for a plurality of times.

In the invention, in the data integrity checking step, when the data integrity problem is found, the data recovery algorithm is utilized to carry out intelligent repair by combining redundant data. The data recovery algorithm employs a Reed-solomon code (Reed-Solomoncode) based error correction technique that is capable of correcting errors in a block of data over a range. For each data block, the parameters of the error correction code are determined according to their importance and storage means, e.g. for important data blocks, higher error correction capability parameters are used. When the data integrity problem is detected, firstly analyzing the error type and degree, if the error is a small number of bit errors, directly correcting by using an error correction code, and if the data block is lost or seriously damaged, recovering the data by a data reconstruction algorithm according to the association relationship between the copy stored in the redundant area and the data block. In the data reconstruction process, the original content of the data block is gradually restored by combining the index information in the storage layout, the logic sequence of the data block and the relevance of the hash value, so that the repair efficiency is improved.

In the data migration step, in the process of migrating data, the method adopts a mode of combining incremental backup and differential backup, so that the data transmission quantity and migration time are reduced. After marking the data to be migrated, a full back-up is first performed as a base version. Then, for subsequent data changes, the newly added data blocks and changed data blocks are identified through log records of the file system and time stamp information of the data blocks, and the data blocks form incremental backups. Meanwhile, hash values of the same data blocks in the original cold storage device and the new cold storage device are compared, and data blocks with differences are found out and serve as contents of the difference backup. During migration, only the data blocks of the incremental backup and the differential backup are transmitted, so that the data transmission quantity is greatly reduced. For large-scale cold storage systems, the migration time can be shortened from a plurality of days to a plurality of hours of traditional full-volume migration, and the migration efficiency is improved.

The invention also discloses a cold storage management system with a data protection function, which comprises:

The environment assessment module is used for realizing the functions of the storage environment assessment step in the claim 1 and comprises a sensor, a data acquisition system and an analysis alarm unit. The sensor has a self-calibration function, and the measurement accuracy of the sensor is calibrated periodically (e.g. once a week) through a built-in calibration circuit and a standard reference source. In the calibration process, the temperature sensor is compared with a high-precision standard thermometer, the humidity sensor is calibrated with a standard humidity generator, the Hall effect sensor and the acceleration sensor are calibrated through special calibration equipment, the accuracy of environmental data acquisition is ensured to be within an allowable error range, and the reliability of subsequent environmental assessment is ensured.

A data encryption module, which executes the data encryption step in claim 1 and encrypts the data by using a multi-layer encryption algorithm and a key management mechanism. The module has an encryption algorithm updating function, and is connected with an external security updating server to periodically (e.g. once a month) check whether a new encryption algorithm vulnerability is released. And if a new security threat exists, automatically downloading and updating the encryption algorithm. In the updating process, a double-key mechanism is adopted, namely, an old encryption key and a new encryption key are reserved for a period of time, so that the ongoing read-write operation is not affected, and meanwhile, a new encryption algorithm is used for the newly stored data, so that smooth transition is realized.

The storage layout planning module is responsible for the data storage layout planning step in the claim 1, planning storage areas and setting redundancy according to the data characteristics. The module supports dynamic adjustment of storage layout, and reclassifies and plans storage locations for data by monitoring access frequency, importance change and performance indexes (such as storage capacity and read-write speed) of the storage device in real time through intelligent algorithms (such as clustering algorithms based on machine learning). For example, when the access frequency of a certain type of data suddenly increases, the data is migrated from an inner low-temperature storage area to an outer relatively high-access-speed area, and when the performance of a certain area of the storage device is reduced, the data is automatically migrated to other areas with good performance.

An access control module establishes a user authorization and access record mechanism according to the access control step in claim 1. The module can be integrated with an external identity management system, and can realize seamless connection with the existing user management system of an enterprise through standard interface protocols (such as LDAP and OAuth). Thus, the identity information and authority setting of the user in the cold storage system can be consistent with the user management strategy of the whole enterprise, and unified management and maintenance are convenient. Meanwhile, in the process of integrating with an external system, a safe data transmission mode and an encryption authentication mechanism are adopted, so that user information leakage is prevented.

The data integrity checking module completes the data integrity checking step of claim 1, periodically checks the data and resumes when there is a problem. The module can process the integrity check of a plurality of data blocks in parallel, and reasonably distributes the number of threads according to the hardware resources (such as the number of CPU cores) of the system through a multithreading technology. For example, for a system with an 8-core CPU, 8 threads can be started simultaneously to perform hash calculation and checksum comparison on the data blocks, so that the checking speed is greatly improved. In the data recovery process, the high-efficiency memory management and data caching technology is utilized, so that the time for reading and writing data is reduced, and the repair process is quickened.

And the data migration module is used for realizing the migration of data among the cold storage devices according to the data migration step in the method disclosed by the claim 1. Breakpoint resume is supported during migration, and by recording transmission progress information (e.g., number of bytes transmitted, transmission status) for each data block during migration, such information is stored in a local temporary file or database. When the migration process is interrupted due to network failure and equipment restarting, the system can continue to transmit from the position of last interruption according to the recorded progress information after recovery, so that the migration stability is ensured, the part with completed repeated transmission is avoided, and the migration efficiency is improved.

The present invention is not limited to the above-mentioned embodiments, and any person skilled in the art, based on the technical solution of the present invention and the inventive concept thereof, can be replaced or changed within the scope of the present invention.

Claims (8)

1. The cold storage management method with the data protection function is characterized by comprising the following steps of:

The method comprises the steps of carrying out comprehensive evaluation on the physical environment where cold storage equipment is located, distributing a temperature sensor, a humidity sensor, a Hall effect sensor and an acceleration sensor in a cold storage area to form a sensor network, collecting sensor data by a data acquisition system at a preset frequency, immediately triggering acquisition by vibration and abnormal magnetic field intensity, transmitting the acquired data to a data analysis unit, judging by the data analysis unit based on a preset environmental parameter threshold value, enabling the magnetic field intensity not to exceed a set Gaussian value, and immediately giving an alarm to a manager through an audible and visual alarm and a remote notification system;

A data encryption step, namely encrypting the data by adopting a multi-layer encryption algorithm before the data is stored in the cold storage equipment; firstly, carrying out initial encryption on data by using an AES-256 symmetric encryption algorithm, wherein a key is generated by a key generation algorithm based on a hash function and a pseudo-random number generator, dividing the data into data blocks with fixed sizes, independently encrypting each data block, carrying out multiple rounds of transformation on the data blocks in the encryption process, wherein each round comprises byte substitution, line shifting, column confusion and round key addition operation;

Classifying data according to the type, access frequency and importance of the data, wherein the data type is divided into structured data, semi-structured data and unstructured data, backup data of an enterprise financial database is stored in an inner layer area of cold storage equipment, the area is provided with an independent power supply and a redundant cooling system, an enterprise-level solid state disk storage medium is adopted, access delay is reduced, data safety is improved, backup data of recent business records are stored in an area at the opposite outer layer, a mechanical hard disk is used, meanwhile, a redundant area is set, the size of the redundant area is divided according to the total data amount and the importance proportion, checksum of the data is calculated by utilizing an SHA-256 hash algorithm, each data block corresponds to one checksum, and the checksum and the data are stored in the redundant area together;

the access control step comprises the steps of establishing a user access authorization mechanism, wherein a user needs to pass multi-factor identity verification, including user name and strong password, fingerprint identification and smart card identification;

The method comprises the steps of checking data integrity, namely checking the data integrity in cold storage at regular intervals, re-calculating an SHA-256 hash value of the data in a parallel calculation mode by taking a data block as a unit according to a checksum copy set in a storage layout plan, comparing the SHA-256 hash value with the stored checksum, starting data recovery operation if the data integrity problem is found, and carrying out intelligent restoration by combining the copy stored in a redundant area with associated information between the data block through a data recovery algorithm;

The method comprises the steps of data migration, namely when the storage capacity utilization rate of a cold storage device reaches 80% or a storage technology is updated, firstly, marking data to be migrated, realizing by adding migration marking bits in metadata of the data, suspending access operation to the data, then reading the data from the original cold storage device, adopting a high-speed network channel based on SSL/TLS encryption in the transmission process, simultaneously carrying out integrity check on the data again, carrying out hash check on 100MB data once, after the data is received by a new cold storage device, re-storing the data according to a new storage layout plan, determining the new layout according to the latest classification and importance evaluation result of the current data, updating access authority, adjusting according to the latest setting in a user authority management system, updating related metadata information, deleting or marking the data in the original cold storage device as migrated after the migration is completed, and notifying all modules in the system to resume normal access operation to the data through a broadcasting mechanism.

2. The cold storage management method with the data protection function according to claim 1, wherein in the storage environment assessment step, environment data are predicted and analyzed by a machine learning algorithm, a long-short-term memory network LSTM algorithm is adopted, temperature, humidity, magnetic field intensity and vibration data of the past year are used as training sets, the training model learns seasonal changes, periodic fluctuation and characteristics of sudden abnormal conditions, the number of neurons, the number of layers and learning rate parameters of the model are continuously adjusted, the prediction performance of the model is optimized, in actual operation, the model predicts abnormal rise or fall of temperature, abrupt change of humidity, abnormal fluctuation of magnetic field intensity and vibration events occurring 1-2 hours in advance, and preventive measures are started in time, including adjustment of cooling system parameters, starting of dehumidification equipment or physical fixation of reinforcement equipment.

3. The method for cold storage management with data protection according to claim 1, wherein in the step of data encryption, the encrypted data is subjected to confusion processing, after the data encryption is completed, random padding data with specific length is generated according to the size of a data block and the output result of an encryption algorithm, the generation of the padding data is based on a cryptographically secure pseudo-random number generator, the seed value of which is determined by a plurality of factors including system time and a hardware identifier together, randomness is ensured, and the padding data is inserted into the encrypted data in a specific mode.

4. The cold storage management method with a data protection function according to claim 1, wherein in the data storage layout planning step, data is stored in a plurality of cold storage devices in a scattered manner by adopting a distributed storage technology, the cold storage devices are managed by using a Ceph distributed file system, the data is uniformly distributed on each storage node through a consistent hash algorithm, each data block is divided into a plurality of sub-blocks during storage, the sub-blocks are stored in different cold storage devices according to hash values, at the same time, redundant copies of the data are stored on different cold storage devices, the number of copies is determined according to the importance of the data, when the cold storage devices fail, a heartbeat detection mechanism is established among the other devices stored with the copies, heartbeat signals are mutually transmitted every 10 seconds, if the nodes continuously receive no heartbeat signals for 3 times, the node fails, the data recovery and load balancing mechanism is automatically started.

5. The method according to claim 1, wherein in the access control step, a blockchain technology is introduced to store the access records of the user in a non-tamperable manner, the detailed information of each user access is used as a transaction record, each transaction record is linked by using a hash chain structure of the blockchain, each block contains a plurality of transaction records, the integrity of the block is ensured by calculating a hash value of a block header, consistency confirmation of the transaction records by nodes in the blockchain network is ensured by using a PoW workload certification or PoS benefit certification consensus mechanism, and any tampering of the access records needs to modify subsequent blocks of the whole blockchain by storing the access records on the blockchain, and meanwhile, the access control strategy is automatically executed by using an intelligent contract function of the blockchain, including automatically limiting the access authority of the user when the user tries illegal access a plurality of times continuously.

6. The cold storage management method with the data protection function according to claim 1, wherein in the data integrity checking step, when the data integrity problem is found, intelligent restoration is performed by combining redundant data through a data restoration algorithm, the data restoration algorithm adopts an error correction technology based on Reed-Solomoncode of a Reed-Solomon code, the technology can correct errors in data blocks within a certain range, for each data block, parameters of error correction codes are determined according to importance and a storage mode of each data block, when the data integrity problem is detected, the type and degree of the errors are analyzed first, if the data integrity problem is detected, the error correction is directly performed by using the error correction codes, if the data block is lost, the data is restored through a data reconstruction algorithm according to an association relation between copies stored in a redundant area and the data block, and in the data reconstruction process, the original content of the data block is gradually restored by combining index information in a storage layout, the logical sequence of the data block and the association of hash values, so that the restoration efficiency is improved.

7. The method for managing cold storage with data protection function according to claim 1, wherein in the step of data migration, a mode of combining incremental backup and differential backup is adopted to reduce data transmission amount and migration time, after marking data to be migrated, full backup is firstly performed as a basic version, then for subsequent data changes, newly added data blocks and changed data blocks are identified through log records of a file system and timestamp information of the data blocks, the data blocks form incremental backup, meanwhile, the hash values of the same data blocks in original cold storage equipment and new cold storage equipment are compared, data blocks with differences are found out and used as contents of differential backup, and during migration, only the data blocks of the incremental backup and the differential backup are transmitted.

8. A system for implementing the cold storage management method with data protection function of any one of claims 1 to 7, comprising:

the environment evaluation module is used for realizing the function of storing the environment evaluation step and comprises a sensor, a data acquisition system and an analysis alarm unit, wherein the sensor has a self-calibration function, the measurement precision of the sensor is calibrated regularly through a built-in calibration circuit and a standard reference source, in the calibration process, the temperature sensor is compared with a high-precision standard thermometer, the humidity sensor is calibrated with a standard humidity generator, and the Hall effect sensor and the acceleration sensor are calibrated through calibration equipment;

the data encryption module is used for executing a data encryption step, encrypting data by utilizing a multi-layer encryption algorithm and a key management mechanism, having an encryption algorithm updating function, periodically checking whether a new encryption algorithm vulnerability is issued or not by connecting with an external security updating server, automatically downloading and updating the encryption algorithm if a new security threat exists, adopting a double-key mechanism in the updating process, namely simultaneously reserving an old encryption key and a new encryption key for a period of time, and simultaneously using the new encryption algorithm for the newly stored data;

the storage layout planning module is responsible for planning a storage area and setting redundancy according to data characteristics, supporting dynamic adjustment of the storage layout, and reclassifying and planning storage positions of the data by using a clustering algorithm based on machine learning through monitoring access frequency, importance change and performance indexes of the storage equipment in real time;

The access control module establishes a user authorization and access recording mechanism according to the access control step, is integrated with an external identity management system, and realizes the butt joint with the existing user management system of an enterprise through standard LDAP and OAuth interface protocols;

The data integrity checking module completes the data integrity checking step, periodically checks data and restores the data when the data is in a problem, processes the integrity checking of a plurality of data blocks in parallel, distributes the number of threads according to the hardware resources of the system through a multithreading technology, simultaneously starts 8 threads to perform hash calculation and check sum comparison on the data blocks for the system with an 8-core CPU, and reduces the time of data reading and writing by utilizing the high-efficiency memory management and data caching technology in the data restoring process;

And the data migration module is used for realizing the migration of data among cold storage devices according to the data migration step, supporting breakpoint continuous transmission in the migration process, storing the information in a local temporary file or database by recording the transmission progress information of each data block in the migration process, and when the migration process is interrupted due to network failure and device restarting reasons, continuing to transmit from the position of last interruption according to the recorded progress information after the system is recovered.