CN101808095A - Encryption copy organization method under distributed storage environment - Google Patents

Abstract

The invention discloses a method for organizing encrypted copies in a distributed storage environment. The method divides a management unit data block of system data into multiple data segments of equal size. Units encrypt data, which provides finer-grained control over data blocks. Since the data block is encrypted by segments, there is no correlation between each ciphertext data segment, which can be encrypted and decrypted in parallel, avoiding the huge cost of encrypting and decrypting the entire data block when reading and writing a small amount of data. Overhead; for reading large amounts of data, read requests are grouped, and different group requests are sent in parallel to each storage node that maintains a copy of the requested file data block, and each group is read in parallel to improve the efficiency of reading data. The invention realizes the application of the encryption technology and the copy technology in the distributed storage environment, and the proposed encryption copy organization method greatly improves the efficiency of reading and writing data.

Description

An Encrypted Copy Organization Method in Distributed Storage Environment

technical field

The invention belongs to the field of computer distributed storage security, and in particular relates to an organization method of encrypted copies in a distributed storage system environment. This method distributes user data to multiple storage nodes in the form of blocks, and the data of each block is segmented for encryption. At the same time, in order to improve the availability of the system, each encrypted block maintains multiple copies in multiple storage nodes. , so as to ensure high security and high availability of user data, and all data organization information is maintained by a separate metadata server.

Background technique

Encryption technology is usually used to ensure the confidentiality of user data. The user's private data is encrypted by a certain encryption algorithm. The user only needs to protect the security of the encryption key to ensure that the private data will not be leaked, even if a malicious attacker obtains the data. ciphertext, and its original content cannot be known.

Encryption technology has been widely used in stand-alone storage environments. There are application-level encryption products, file system-level encryption products, disk block-level data products, and disk products that directly integrate encryption functions. However, encryption technology There is no mature implementation method for the application in the distributed storage environment.

In recent years, there have been some researches on the security of distributed storage systems, but they mainly focus on authentication and access control. Once an attacker breaks through this level, all user data can be obtained. Distributed storage systems usually use data blocks to organize user file data. Data blocks refer to a collection of file data of a certain length. Usually, in order to reduce the number of interactions between the client and the server, the length of the data block is generally larger. Some distributed storage systems encrypt data in units of data blocks. When the data requested by the user belongs to this data block, this data block needs to be encrypted and decrypted in its entirety, and many encryption modes are not conducive to parallel computing, so The overhead of encryption and decryption is very high, and when a malicious attacker destroys a data block, the data of the entire file is also destroyed. This method is not conducive to the expansion of the system, and the availability of file data is very low.

Copy technology refers to maintaining a complete copy of a certain data in multiple geographically dispersed locations in a distributed storage system, so that when a certain piece of data is destroyed, the system can still obtain correct data from other copies of the data, thereby improving The availability of the system solves the problem of malicious users destroying a certain part of the data, which will cause the entire file data to be unrecoverable.

Some researchers combine encryption technology with copy technology, encrypt data blocks of system files, and back up encrypted data to near-line or offline backup servers. This method improves data security and availability, but in When some data in the system is damaged, the backup data cannot be put into use immediately, and needs to be migrated from the backup server, resulting in the system not being able to guarantee timely application services.

Contents of the invention

In order to make up for the lack of data security protection in the existing distributed storage system, the present invention proposes a method for organizing encrypted copies in the distributed storage system. The huge overhead caused by decryption can also improve the efficiency of read requests with large amounts of data.

A method for organizing encrypted copies in a distributed storage environment provided by the present invention is characterized in that the method includes the process (A1)-(A7) of organizing copies in a write request:

(A1) The metadata server receives the write request from the client, including the file name, the starting position O of the write request and the length L of the write request;

(A2) Let M be the larger value in (O+L) and the length of the original file, check whether the existing storage space of the original file is greater than or equal to M, if so, use the existing storage space to store the data to be written , otherwise, the metadata server reads the configuration information of the administrator or uses the default value to determine the number of file copies R, the block size B, the segment size S and the encryption algorithm type; the metadata server selects R according to the total load of the storage node A storage node with the smallest load is used as a new storage space, the existing storage space is used to store the first part of the data to be written, and the new storage space is used to store the remaining data of the written data, a total of R storage nodes;

Relevant data required by the write request, including block size B, segment size S, and encryption algorithm type, are returned to the client in a secure manner;

(A3) The client calculates the block number O/B where the start position of the write request is located, and calculates the start position P1 and the end position P2 of the write request in the data block;

(A4) Determine whether the start position P1 and the end position P2 are at the boundary of the segment, if yes, go to step (A5), otherwise, the start position P1 or the end position P2 is in the middle of a certain segment, then read the segment Original information, and decipher, enter step (A5);

(A5) segmentally encrypt the data to be written in the block, and write it to R storage nodes;

(A6) If this step is executed for the first time, and the end position and start position of the write request are not in the same block, then calculate the start position P1 and end position P2 of the write request in the second block, and then Go to step (A4); otherwise, go to step (A7);

(A7) The client sends the write request completion feedback information to the metadata server, and the metadata server fills the file metadata information according to the system configuration information and the file request information.

Read the encrypted copy formed in the above process:

(B1) The metadata server receives the read request from the client, including the file name, the read request starting position O, and the read request length L;

(B2) make W be the file length to be read, check whether O exceeds the size of the file to be read, if exceed, then proceed to step (B8), otherwise enter step (B3);

(B3) check whether O+L exceeds the size of the file to be read, if so, make L=W-O, modify the end position of the read request to the position of the file to be read, otherwise, use O+L as the end position of the read request;

According to the metadata information of the file to be read, the metadata server returns the relevant data required by the read request, including the basic attributes of the file, the block size B, the segment size S, and the type of encryption algorithm to the client in a safe manner;

(B4) The client calculates the block number O/B where the start position is located, and calculates the start position P3 and the end position P4 of the read request in the data block;

(B5) The client first divides the segments that need to be read into R groups, and the number of segments contained in each group is as equal as possible; then the client sends requests to R storage nodes that maintain copies of the block at the same time, and reads the R groups in parallel. data;

(B6) After the client obtains the data of all required sections, the data of these sections are decrypted respectively, and the data outside the range of P3 and P4 are discarded, that is, the actual data to be read is obtained;

(B7) If this step is executed for the first time, and the end position and start position of the read request are not in a block, then calculate the start position P3 and end position P4 of the read request in the second block, and then go to Step (B5); otherwise, go to step (B8);

(B8) The client sends feedback information that the read request is completed or the request exceeds the range to the metadata server.

The method of the present invention proposes a new encrypted copy organization method, which divides the management unit data block of the system data into a plurality of equal-sized data segments, the system still manages with the block as the unit, and the client encrypts the data with the segment as the unit, like this It can provide finer-grained control over the data block, and avoid the huge overhead caused by encrypting and decrypting the entire data block for a small amount of reading and writing; for large read requests, the read requests can be grouped, and different Group requests are sent in parallel to each storage node that maintains a copy of the requested file data block, and each group is read in parallel, thereby greatly improving the efficiency of reading data.

Description of drawings

FIG. 1 is a schematic diagram of a storage node management structure;

FIG. 2 is a schematic diagram of file metadata structure;

Figure 3 is a schematic diagram of the organization of encrypted copies;

Fig. 4 is a schematic diagram of the generation process of an encrypted copy;

Fig. 5 is a schematic diagram of the extraction process of the encrypted copy;

Fig. 6 is a flow chart of client write request processing;

FIG. 7 is a flowchart of client read request processing.

Detailed ways

In a large-scale storage system, the amount of data is usually huge, and a large amount of data is organized and managed through metadata, which is information describing other data, that is, data of data.

The application environment of the present invention is a typical distributed storage system environment, which consists of three parts: client, metadata server, and storage node, wherein the metadata server is mainly responsible for file metadata management of storage nodes, user information management, file classification Block information management, system security information, copy information management, storage node load balancing management, etc.; storage nodes are used as data storage warehouses to store actual data of files in the system; clients are responsible for providing users with a transparent storage system access interface, To provide users with system services.

Below in conjunction with accompanying drawing and example the present invention is described in further detail.

The metadata server allocates new storage nodes to the client according to the load of the storage nodes according to the client's request, and the storage nodes dynamically send load information to the metadata server so that the metadata server can know its load in time. The server maintains a storage node information chain according to the load from small to large. As shown in Figure 1, the storage node information mainly includes the following fields: storage node IP address, storage node management data block number, storage node total load, CPC load and weight , storage load and weight, network load and weight and other information, among which the weights of CPU load, storage load, and network load are set by default: network load weight is 0.3, storage load weight is 0.4, CPU load weight is 0.3, It can be dynamically configured by the system administrator according to the system requirements, that is, write the weight value into the corresponding configuration file. It is required that the three weight values are all in the range of 0 to 1, and the sum of the three is 1. When the storage node calculates the total load, it reads Fetch the config file to get these values.

Every time the client needs to request a new data block for write operation, the metadata server only needs to select multiple storage nodes at the head of the storage node information chain. As the system continues to run, the load of the storage nodes will be dynamically updated. Through This method can effectively realize system load balancing.

After the client's write request is completed, the metadata server will generate file metadata information as shown in FIG. 2 for the written file. File metadata information includes the following fields: file name, file name MD5 value, file basic attributes, number of blocks, block size, segment size, encryption algorithm type, encryption key, number of file copies, and copy location chain. Some of the information can be configured by the system administrator, such as the block size, which is set to 8MB by default. This value should be set within the range of 8-64MB according to different application requirements, so as to ensure that for any read and write requests, the request involves at most two Data block, so as to simplify the processing of client read and write requests; Segment size, the default value is set to 4KB, and should be set within the range of 1-16KB; Encryption algorithm type, AES algorithm is used by default, and the configurable encryption algorithm includes DES, AES, BlowFish algorithm; the number of file copies, 3 copies are maintained by default, and the number of copies should be set in the range of 1-5; these values are written by the administrator to the corresponding configuration file before the system starts, and are read from the metadata server when needed The configuration file is read. Other information including file name, file name MD5 value, file basic attributes, number of blocks, copy location chain, etc. are generated under the user's write request and change dynamically as the file changes. When the client sends a read request, the metadata server provides relevant information of the requested file according to the structure, so as to satisfy the client's request.

The invention organizes the encrypted copy of the system in segments, and proposes an improved method for reading and writing in the context of the organization. The final organizational form of the encrypted copy in the storage node is shown in Figure 3. The data of each file is divided into multiple data blocks according to the size of the configured data block, and the data block is the unit of metadata management. For each data block, it is divided into multiple data segments, the segment is used as the unit of data encryption, and the client encrypts each data segment separately, so that the data between each segment has no correlation, that is, the data of each segment can Parallel encryption and decryption, which is conducive to parallel processing, thereby improving the efficiency of the system. Compared with the traditional encryption for the whole block of data, this method only needs to additionally record the relevant information of the segment, and will not add a great burden to the metadata server.

The generation of an encrypted copy depends on the write request of the client. The following is a detailed description of the generation process of the encrypted copy when the client writes a file. The generation and organization of encrypted copies when writing files mainly includes the following steps:

(A1) The metadata server receives the write request from the client, including the file name, the start position O of the write request (the start position is 0 when creating a new file), and the length L of the write request.

(A2) Let T be the larger value of (O+L) and the original length of the file, check whether the existing storage space of the file to be written is greater than or equal to T, if so, use the existing storage space to store the data to be written , otherwise, allocate a new storage space for the file, use the existing storage space to store the previous part of the data to be written, and use the new storage space to store the remaining data of the written data;

When allocating new storage space, it is necessary to find new storage nodes. According to the total load of storage nodes, the metadata server selects R storage nodes with the smallest load to store R copies. a field.

According to the configuration information of the administrator, the metadata server returns the relevant data required by the write request, including block size B, segment size S, and encryption algorithm type, to the client in a secure manner.

(A3) The client calculates the block number O/B where the start position of the write request is located, and calculates the start position P1 and the end position P2 of the write request in the data block. P1 is equal to the remainder of O divided by B, and P2 is equal to (P1 +L) is divided by the remainder of B.

(A4) P1 and P2 are the start and end positions of the write request in the block. First, it is necessary to calculate whether P1 and P2 are at the boundary of the segment. The position P1 and P2 can be removed by using the segment size S. If they can be divisible, then the description The position is just at the boundary of the segment, that is, the segment where the position is located is completely covered; otherwise, the position is in the middle of a segment, that is, the segment is partially covered. Because the data is stored in ciphertext in units of segments, for partially covered segments, the entire segment is not completely covered by new encrypted data, so before updating the data of these segments, it is necessary to read the original information of the segment, and Decrypt, then modify some data, and re-encrypt. P1/S, P2/S are the segment numbers where the start and end positions of the write request are located in the block.

As shown in Figure 4, P1 is in the middle of segment S1, and P2 is in the middle of segment S3, then update the data between P1 and P2 (for data exceeding the original file size range, the original data is considered to be a sequence composed of 0) , you need to read the data of the two segments S1 and S3 first. P1 divides segment S1 into two parts, S1-1 and S1-2, and P2 divides segment S3 into two parts, S3-1 and S3-2. The client needs to allocate 3 segment-sized buffers to store new data. The data in the buffer includes the data of S1-1, the data requested to be written, and the data of S3-2.

(A5) The client encrypts the buffer in segments, and writes the encrypted multiple ciphertext segments to the corresponding R storage nodes, thereby generating R copies of the data; when both P1 and P2 are in the segment When the boundary of the segment is reached, the original data does not need to be read; when P1 and P2 have a position in the middle of a certain segment, the data of the segment needs to be read. That is, a write request needs to read at most two segments of data, which is much more efficient than the traditional method of encrypting the entire block.

(A6) If this step is executed for the first time, and the end position and start position of the write request are not in the same block, calculate the start position P1 and end position P2, P1 of the write request in the second block Equal to 0, P2 is equal to the remainder of dividing (O+L) by B, then go to (A4); otherwise, go to (A7).

(A7) The content of the write request of the client has been written to each storage node, and the data of each block in the storage node is also stored according to the encrypted copy organization method proposed in the present invention, and the client sends the write request completion feedback information to the metadata server. The data server fills file metadata information according to system configuration information and file request information.

The client's read request extracts the data in the encrypted copy. Because there are multiple copies of the data and it is segmented encryption, the read request can be divided into multiple groups. The number of groups is equal to the number of file copies, and each group contains as many segments as possible. , the client reads each group from multiple storage nodes in parallel, thereby improving the efficiency of read requests. The processing of client read requests mainly includes the following steps:

(B1) The metadata server receives the read request from the client, including the file name, the read request starting position O, and the read request length L;

(B2) make W be the file length to be read, check whether O exceeds the size of the file to be read, if exceed, then proceed to step (B8), otherwise enter step (B3);

(B3) check whether O+L exceeds the size of the file to be read, if so, make L=W-O, modify the end position of the read request to the position of the file to be read, otherwise, use O+L as the end position of the read request;

According to the metadata information of the file to be read, the metadata server returns the relevant data required by the read request, including the basic attributes of the file, the block size B, the segment size S, and the type of encryption algorithm to the client in a safe manner;

(B4) The client calculates the block number O/B where the start position is located, calculates the start position P3 and the end position P4 of the read request in the data block, P3 is equal to the remainder of O divided by B, and P4 is equal to (P3+L ) divided by B;

(B5) As shown in Figure 5, P3 and P4 are the start and end positions of the read request in the block, and the client needs to read all the segments T1, T2 and T3 spanned by P3 and P4. The client first divides the segments to be read into R groups, and each group contains as many segments as possible. Then the client sends requests to R storage nodes that maintain the copy of the block at the same time, and reads the data of R groups in parallel, thereby improving the efficiency of reading;

(B6) After the client obtains the data of all required segments, it decrypts the data of these segments respectively, and discards the data outside the range of P3 and P4, that is, obtains the actual data to be read.

(B7) If this step is executed for the first time, and the end position and start position of the read request are not in the same block, calculate the start position P3 and end position P4 of the read request in the second block, and P3 is equal to 0 , P4 is equal to the remainder of (O+L) divided by B, then go to (B5); otherwise, go to (B7);

(B8) The client sends feedback information that the read request is completed or the request exceeds the range to the metadata server.

The present invention is not limited to the above-mentioned specific embodiments, and those skilled in the art can implement the present invention by using other various specific embodiments according to the disclosed content of the embodiments and accompanying drawings. Some simple changes or modified designs all fall within the protection scope of the present invention.

Claims (2)

1. the encryption copy organization method under the distributed storage environment is characterized in that: organize the process of copy to be in the write request of this method:

(A1) meta data server is received the write request of client, the original position O of include file name, write request and write request length L;

(A2) make that T is a higher value in the length of (O+L) and original document, check that whether the existing memory space of original document is more than or equal to T, if, existing memory space is used to store data to be written, otherwise, meta data server reads keeper's configuration information or Uses Defaults, and determines duplicate of the document number R, divides block size B, fragment size S and encryption algorithm type; Meta data server is according to the memory node total load, the memory node of choosing R load minimum is as new memory space, existing memory space is used to store the preceding part of data to be written, and new memory space is used to store the remaining data of the data that write, and amounts to R memory node;

The relevant data that write request is needed comprise piecemeal size B, fragment size S, and encryption algorithm type returns to client in the mode of safety;

(A3) client calculates the piece O/B at write request original position place, calculates write request original position P1 and end position P2 in data block;

(A4) whether judge original position P1 and end position P2 at section boundary, if, change step (A5) over to, otherwise original position P1 or end position P2 read the original information of this section in the centre of certain section, and deciphering, step (A5) entered;

(A5) data to be written in this piece are carried out segmentation and encrypt, and be written to R memory node;

(A6) if for the first time carry out this step, and write request end position and original position then calculate original position P1 and the end position P2 of write request in second piece not in same, forwards step (A4) then to; Otherwise, forward step (A7) to;

(A7) client sends write request to meta data server and finishes feedback information, and meta data server is filled file metadata information according to system configuration information and file request information.

2. encryption copy organization method according to claim 1 is characterized in that: read formed encryption copy according to following process:

(B1) meta data server receives the read request of client, include file name, read request original position O, read request length L;

(B2) make W be the file size that continues, check that whether O surpasses the size of file that continues, if surpass, then changes step (B8) over to, otherwise enters step (B3);

(B3) check whether O+L surpasses the size of the file that continues, if, make L=W-O, the position that the end position of read request is revised as the file that continues, otherwise, with the end position of O+L as read request;

Meta data server is according to the metadata information of the file that continues, and the relevant data that read request is needed comprise the file base attribute, divide block size B, fragment size S, and encryption algorithm type returns to client in the mode of safety;

(B4) client calculates the piece O/B at original position place, calculates read request original position P3 and end position P4 in data block;

(B5) section that at first needs read of client is divided into R group, and the hop count that each group comprises is tried one's best equal; Client safeguards that the memory node of this piece copy sends request simultaneously, the parallel data that read R group to R then;

(B6) after client is obtained all data that need section, the data of these sections are deciphered respectively, and with P3, the data outside the P4 scope abandon, and promptly obtain the real data that will read;

(B7) if for the first time carry out this step, and read request end position and original position then calculate original position P3 and the end position P4 of read request in second piece not in a piece, forwards step (B5) then to; Otherwise, forward step (B8) to;

(B8) client sends read request to meta data server and finishes or ask off-limits feedback information.

Images