CN100565530C - A kind of snapshot system and using method thereof - Google Patents

Abstract

The present invention provides a snapshot system, which includes a device management tool set, a snapshot implementation module, and a physical device; the snapshot implementation module includes a snapshot mechanism layer for data search and time point management, and is used for the physical device A physical resource management layer for allocation and reclaim management, the snapshot mechanism layer includes at least one snapshot group, and different snapshot groups can use different snapshot technologies; the physical resource management layer contains The metadata device and data device corresponding to the group. The present invention can simultaneously support two snapshot technologies of COW and ROW, and users can choose flexibly according to actual needs.

Description

A kind of snapshot system and using method thereof

technical field

The invention relates to the field of data protection, and relates to a snapshot system and a using method thereof.

Background technique

Snapshot is a static set generated for source data, which is widely used in data backup, data mining and other fields. According to different implementation technologies, snapshots can be divided into: split mirror (SplitMirror), copy on write (COW: Copy On Write), redirect on write (ROW: Redirect onWrite) and write anywhere (WA: Write Anywhere). At present, the latter three types of technologies are the mainstream snapshot technologies, among which ROW and WA technologies are basically the same, but differ in the organization of data. ROW uses a snapshot space that is isolated from the data space and is used to store snapshot data. When the source data is updated, new data is written to the snapshot space. WA does not reserve snapshot space, and snapshot data and source data share the same physical space and are stored together. The performance of different snapshot implementation technologies varies greatly under different application loads. For example, for applications that mainly focus on write operations, such as online transaction processing (OLTP) applications, the ROW and WA snapshot technologies will achieve better performance than the COW snapshot technology due to their better write performance. For applications that mainly read operations, such as decision support system (DSS) applications, COW snapshot technology has better performance, because COW snapshot technology does not disrupt the sequence of source data, and the performance is not affected when reading source data. The impact of snapshots, while ROW and WA snapshot technologies will cause source data to be scattered and disrupt the sequence of source data, and may face a large number of read merge operations during read operations, resulting in low performance.

Existing snapshot products and prototype systems are designed for specific applications, and a specific snapshot product or prototype system is usually implemented using a single technology. For example, IBM ESS FlashCopy only uses COW technology, while the Clotho system of the University of Toronto uses Using WA technology. These systems work well for specific applications. However, with the wide application of service integration and virtual storage systems, the application mode of storage has changed significantly. Many different types of applications run concurrently on the same storage platform. In the prior art, a snapshot system only adopts one snapshot implementation technology, which is likely to bring defects of low snapshot operation implementation efficiency and poor system performance.

Contents of the invention

Therefore, the object of the present invention is to overcome the defect that the snapshot technology adopted by the existing snapshot products is single and not universal, so as to provide a snapshot system capable of simultaneously supporting both COW and ROW snapshot technologies.

In order to achieve the above object, the present invention provides a snapshot system, including a device management tool set, a snapshot implementation module, and a physical device; it is characterized in that, the snapshot implementation module includes a snapshot mechanism layer for data search and time point management , and a physical resource management layer for allocation and reclaim management of the physical device, the snapshot mechanism layer includes at least one snapshot group, and different snapshot groups can adopt different snapshot technologies; the physical resources The management layer includes metadata devices and data devices corresponding to the snapshot group.

In the above technical solution, in the snapshot group, the adopted snapshot technology is copy-on-write technology or redirect-on-write technology.

In the above technical solution, metadata is stored in the metadata device, and the metadata is used to indicate the increment of data and the storage location of the data in the snapshot version of the snapshot group on the data device; the Metadata is recorded in the form of bitmaps or maps.

In the above technical solution, the bitmap is divided into multi-level structures, and the value of a certain bit in the upper-level bitmap is 1 to indicate that there is a bit with a value of 1 in the corresponding lower-level bitmap. If it is 0, It means that there is no bit with a value of 1 in the corresponding next-level bitmap; on the lowest-level bitmap, a bit with a value of 1 is used to indicate that the data on the corresponding data block has changed. If it is 0, then The data on the corresponding data block does not change.

In the above technical solution, all the snapshot groups use the same memory structure and have four pointers: including the physical device r0 and r1 pointers that need to be accessed when corresponding to the read operation, and the physical device that needs to be accessed when corresponding to the write operation. w0, w1 pointers.

In the above technical solution, in the snapshot group, there is a corresponding snapshot device at a certain time point, and the snapshot devices at each time point are organized in a chain structure.

The present invention also provides a method for using the snapshot system, comprising the following steps:

Step 1), the user issues a command to create a snapshot through the device management tool set, and decides which snapshot technology to choose to create a snapshot in the command;

Step 2), the snapshot mechanism layer in the snapshot implementation module creates a snapshot according to the snapshot technology selected in step 1);

Step 3), perform relevant operations on the snapshot according to user requirements, and the operations include reading, writing and deleting the snapshot.

In the above-mentioned technical solution, in the described step 2), the described creation of the snapshot also includes the following steps:

Step 2-1-1), create a source device, point all the snapshot relationship pointers of the source device to the physical device of the source device;

Step 2-1-2), creating a snapshot at a subsequent point in time, and inserting the created snapshot into the snapshot set linked list.

In the above technical solution, if the copy-on-write technology is used when creating a snapshot, then when inserting the created snapshot into the snapshot set linked list, the w0 pointer of the source device is pointed to the newly created snapshot device, and the newly created The r0 pointer of the snapshot device points to the source device. If there is a snapshot device with an earlier time point in the snapshot set linked list, the snapshot device with an earlier time point and the closest time point will be compared with the newly created snapshot device The r0 pointer points to the newly created snapshot device.

In the above technical solution, if redirection-on-write technology is used when creating a snapshot, when inserting the created snapshot into the snapshot set linked list, the r0 pointer of the source device is pointed to the newly created snapshot device, and the The r0 pointer of the newly created snapshot device points to the snapshot device with an earlier time point and the closest time point; if there is no snapshot device with an earlier time point in the snapshot set linked list, the r0 pointer of the newly created snapshot device points to Zero equipment.

In the above technical solution, in step 3), the reading of the data in the snapshot includes reading the data in the source device and reading the data in the snapshot device.

In the above technical solution, the reading of the data in the source device also includes the following steps:

Step 3-1-1), query the corresponding bit in the bitmap according to the address of the data to be read, if the value of the bit is 1, then directly read locally, and return the read data value, If the value of this bit is 0, execute the next step;

Step 3-1-2), determine whether there is a corresponding snapshot for the data in the source device, if there is no snapshot, then return all zero data, if there is a snapshot, forward the read data request according to the snapshot set linked list, pass the snapshot The data reading completes the data reading process.

In the above technical solution, when judging whether there is a corresponding snapshot for the data in the source device, if the r0 pointer in the source device points to a physical device, there is no snapshot; if the r0 pointer in the source device points to a logical device, there is a snapshot.

In the above technical solution, when the snapshot is created using the copy-on-write technology, step 3) further includes the following steps for reading data from the snapshot:

Step 3-2-1), determine the physical device where the data is stored according to the value of the bitmap, during the determination process, traverse forward from the tail of the snapshot set linked list, if the bitmap value on a certain snapshot S _k is 1, Then read data on the data device of its previous snapshot S _k-1 , if the bitmap value is 0, then forward the read data request to the previous snapshot along the snapshot set linked list until the bitmap value of 1 is found snapshot;

Step 3-2-2), if no snapshot with a bitmap value of 1 is found in the snapshots of the snapshot set linked list, the read data request reaches the source device. If the bitmap value of the source device is 1, it is adjacent to the source device Read data on the data device of the snapshot, otherwise read data on the source data device;

When the snapshot is created using redirection-on-write technology, step 3) also includes the following steps for reading data from the snapshot:

First, determine the physical device where the data is stored according to the value of the bitmap, and then read the data from the physical device where the data is stored; when determining the physical device where the data is stored, it traverses forward from the tail of the snapshot set linked list. A bitmap value on the snapshot S _k is 1, then read on the local data device, and return corresponding data, if the bitmap value is 0, then judge whether the current snapshot is the oldest snapshot in the snapshot set linked list, That is, whether it is a snapshot at the end of the snapshot linked list, if so, return all 0, otherwise, forward the read data request to the previous snapshot in the snapshot linked list.

In the above technical solution, if the write data operation is completed in the snapshot, the step 3) also includes the following steps:

First, determine the physical device where the data is stored according to the value of the bitmap; then write data in the determined physical device.

In the above technical solution, when the snapshot is created using copy-on-write technology, step 3) also includes the following steps for writing the snapshot data:

Judge the bitmap value, if the bitmap value is 1, write it directly into the source physical device; if the bitmap value is 0, write it into the source physical device after the copy-on-write operation is completed.

In the above technical solution, when the snapshot is created using redirection-on-write technology, step 3) also includes the following steps for writing the snapshot data:

The bitmap value is judged, if the bitmap value is 1, it is directly written into the source physical device; if the bitmap value is 0, the data of the data block is merged and then written into the source physical device.

In the above technical solution, in step 3), when the snapshot adopts the copy-on-write technology, only the snapshot with the earliest time point can be deleted during the process of deleting the snapshot.

In the above technical solution, in step 3), when the snapshot adopts redirection-on-write technology, the data to be triggered by deletion of the snapshot is merged before the delete operation is performed on the snapshot to be deleted.

In the above technical solution, the data merging includes the following steps:

Step 3-3-1), sequentially read the corresponding bit of each data block in the snapshot device S2 in the bitmap; wherein, S2 is the adjacent snapshot device of the snapshot device S1 to be deleted;

Step 3-3-2), the bitmap bit in the snapshot device S2 is combined with the corresponding bitmap bit of the snapshot device S1, four situations can be obtained:

S2 S1

a.0 0

b. 0 1

c, 1 0

d, 1 1

When the bitmap bit of S2 is 1, it means that new data has been written, so only the data block whose bitmap bit of S2 is 1 needs to be copied back to the data device of S1 during data merging;

Step 3-3-3), execute the data move back operation, copy the data in the data device of S2 to the data device of S1;

Step 3-3-4), after the data transfer is completed, set the corresponding bit of the bitmap of S2 to be 0;

Step 3-3-5), if all data blocks have been processed, end the data merging operation, otherwise re-execute step 3-3-1), and continue to process the next data block.

The advantages of the present invention are:

1. The general snapshot system of the present invention can support two snapshot technologies of COW and ROW at the same time, and users can flexibly choose according to actual needs, so as to provide optimal support for different applications.

2. While supporting COW and ROW read-only snapshot devices, the present invention provides writable snapshot support, expanding the application of snapshot technology in new fields such as service deployment and electronic classrooms.

3. The present invention adopts a chained device organization structure. At a specific time point, only one copy of data needs to be saved on the data device, which does not increase with the increase in the number of snapshots. In the COW mode of the star structure, storage resources The usage of the snapshot device increases linearly with the increase of the number of snapshot devices, so the present invention can effectively save storage space

4. The present invention adopts a chained device organization structure and a separation structure between metadata devices and data devices, and has good scalability in terms of performance and capacity.

5. The present invention uses an independent device to store metadata and does not reserve space on the data device, avoiding the defect of too large or too small reserved space, and the system scale is not limited by the capacity of the metadata reserved area.

6. The present invention uses an independent device to store metadata, so that the operation of the metadata will not cause the head of the data device to vibrate, and will not affect the performance of user data access.

7. The present invention adopts a three-level bitmap memory structure when storing metadata, provides third-level bitmap swap-in and swap-out operations, and occupies less memory resources while maintaining high-efficiency metadata operations.

8. The present invention adopts a chain organization, only two data devices are involved when reading and writing data, and the performance does not decrease with the increase of the number of snapshots.

Description of drawings

Hereinafter, embodiments of the present invention will be described in detail in conjunction with the accompanying drawings, wherein:

Fig. 1 is the organizational chart of the snapshot system of the present invention;

Fig. 2 is a logical structural diagram of the snapshot system of the present invention;

Fig. 3 is the snapshot device relationship diagram adopting COW technology in the snapshot system of the present invention;

Fig. 4 is a snapshot device relationship diagram using ROW technology in the snapshot system of the present invention;

Fig. 5 is the schematic diagram of the tertiary structure bitmap adopted in the snapshot system of the present invention;

Fig. 6 is the schematic diagram that adopts COW technology to create snapshot in the snapshot system of the present invention;

FIG. 7 is a schematic diagram of creating a snapshot using ROW technology in the snapshot system of the present invention;

FIG. 8 is a schematic diagram of the corresponding change relationship between metadata and data when the snapshot system of the present invention uses ROW technology to create a snapshot in one embodiment;

FIG. 9 is a schematic diagram of the corresponding change relationship between metadata and data when the snapshot system of the present invention adopts COW technology to create a snapshot in one embodiment;

FIG. 10 is a schematic diagram of the usage method of the snapshot system of the present invention.

Detailed ways

The snapshot system of the present invention will be described below with reference to the accompanying drawings and specific embodiments.

Before the present invention is described in detail, at first carry out relevant description to the definition of part noun involved in the present invention:

Logical device: The device created by the snapshot system of the present invention is called a logical device, which is divided into two types: snapshot device and source device.

Physical device: the underlying device used by logical devices, and the device used to store data is called a physical device.

Metadata device: The physical device that stores metadata is called a metadata device.

Data device: The physical device that stores user data is called a data device

Zero device: is a virtual device that returns zero data when read.

Data block: The minimum granularity of I/O request access, generally 4K bytes.

Snapshot group: A group of devices consisting of a source device and multiple snapshot devices derived from the source device

After explaining the above-mentioned words that are easily confused in the present invention, the general snapshot system of the present invention and the corresponding methods involved in the system will be described below.

The general snapshot system of the present invention is a device-level snapshot system, and the snapshot system can simultaneously support a copy-on-write (COW) snapshot technology and a write-time redirection (ROW) snapshot technology. As shown in Figure 2, a complete general snapshot system includes a device management tool set, a snapshot implementation module and physical devices. Wherein, the device management tool set is located at the top of the general snapshot system, and is an interactive interface between the system and the user, and the snapshot implementation module is located at the next layer of the device management tool set, which belongs to the core module of the system and is used to realize For various operations of the snapshot, the physical device is the bottom layer of the general snapshot system of the present invention, and is used for storing various data related to the snapshot. The composition and function of them will be further explained below.

The device management toolset consists of a set of user programs that provide management functions for creating/deleting/modifying snapshot devices. When the general snapshot system of the present invention creates a snapshot, it needs to select which snapshot technology to use to create the snapshot, and this selection process is also realized by the device management tool set. The specific implementation of the device management tool set is similar to the implementation of the corresponding functional modules of the snapshot system in the prior art, therefore, repeated description is not made in the present invention.

The snapshot implementation module is the main part of the general snapshot system. Logically, it can be divided into two layers: the snapshot mechanism layer and the physical resource management layer. The snapshot mechanism layer provides the relationship between snapshot devices for data lookup and point-in-time management. The physical resource management layer provides allocation and reclamation management of the underlying physical space. The composition and functions of the snapshot mechanism layer and the physical resource management layer are described below.

The snapshot mechanism layer includes multiple snapshot groups. The snapshot technology used in a snapshot group is the same, but different snapshot groups can use different snapshot technologies. That is, in the present invention, a snapshot group can only select one of COW and ROW snapshot implementation technologies, but the snapshot implementation technologies of different snapshot groups do not have to be the same.

In a snapshot group, multiple snapshot devices are distinguished according to time points, and each snapshot device is organized in an incremental mode and in series. The relationship between the source device and the snapshot device in a snapshot group is described in FIG. 3 and FIG. 4 . Figure 3 shows the relationship between the source device and the snapshot device under the COW snapshot implementation technology. Figure 4 shows the relationship between the source device and the snapshot device under the ROW snapshot implementation technology. In the ROW device group, the dependency between devices is from the source device to the earliest created snapshot device in turn. In the COW device group, the opposite is true, and the dependency is from the earliest created snapshot device to the source device.

Although in the above-mentioned snapshot group, the snapshot group may adopt COW technology or ROW, but in order to simplify the design and integrate two different technologies of COW and ROW, the present invention adopts a general memory structure of four pointers. In Fig. 6, the general memory structure of four pointers is described, wherein, pointers r0 and r1 correspond to physical devices that need to be accessed during read operations, and w0 and w1 correspond to physical devices that need to be accessed during write operations.

The snapshots provided by the general snapshot system of the present invention can be classified into read-only snapshots and writable snapshots according to types. Generally, snapshots are read-only. The source device and read-only snapshots created at different times form a chain structure. The source device is at the head of the chain, serving as the root node, and the snapshot device with the earliest time is at the end of the chain. Each snapshot device records incremental data changed at adjacent snapshot time points, providing data version management applications with different versions of data on the source volume at multiple time points. In some specific applications, the system also provides writable snapshots. It creates a new snapshot device for read-only snapshots to obtain a writable copy of read-only snapshots, and realizes writable read-only snapshots. After creating a writable snapshot for a read-only snapshot, the original read-only snapshot is still visible to the outside world, and the writable snapshot created for the read-only snapshot appears as a brand new snapshot device, which is equivalent to a new source device. A read-only snapshot can be created for a writable snapshot, and a writable snapshot and its read-only snapshot also form a chain. Write operations to writable snapshots are written directly to the new snapshot device.

When performing a read operation on a writable snapshot, first search for data in the chain where the writable snapshot is located, and if it cannot be found, forward it to the read-only snapshot it depends on for reading. As shown in Figures 3 and 4, g0 is a snapshot group, and a writable snapshot is made to the snapshot device snap5 in it, and the snap5' snapshot device is generated, and the read-only snapshot is continued to snap5' to form a g2 snapshot group. The snapshot group g2 is connected with the snapshot group g0 through the parent pointer of its writable device snap5'. Similarly, take a writable snapshot of snap5'2 in the snapshot group g2, and finally form the g3 snapshot group. When reading the data in the snap5'1 device in the g2 group, first search in the g2 group, if the chain head device snap5' has not found the required data, then continue to search in the g0 group through the parent pointer of snap5' . until the desired data is found.

The physical resource management layer provides management of physical devices used by logical devices, including opening, closing, adding, deleting, and legality checks of physical devices. In a snapshot group, both the source device and the snapshot device can be called logical devices, and one logical device corresponds to two physical devices at the physical resource layer, as shown in Figure 1, where one physical device is a metadata device and the other is a data device equipment.

On the metadata device, the metadata identifying the management and use of physical resources such as physical devices are stored. Specifically, the metadata is used to indicate the increment of data and the storage location of the data in the snapshot version on the data device, which reflects the data mapping relationship of the snapshot version at a certain point in time. For each data block (Chunk) in the data device, there is a corresponding item in the metadata table of the metadata device, which is used to record whether the data block has been modified and the storage location of the data block. A metadata item must include information such as time, data location, and data modification status. Metadata can be recorded in various forms such as bitmaps and mapping tables. In this embodiment, metadata is recorded in a bitmap manner. When recording specifically, each logical device has a bitmap, and each bit in the bitmap represents an address range of a data block size. Locating operations on user data.

The bitmap used to record metadata is stored in the metadata device. Considering the storage space and access efficiency of metadata, the bitmap can be divided into multi-level structures. The divided bitmap structure is consistent with the data in the data device. Block size is closely related. In the present embodiment, the data block size is set to 4096 bytes, as shown in Figure 5, the bitmap uses the following three-level structure: each bit in the first-level bitmap represents a logical space of 4T size, occupying 4k Byte (1 page), each bit in the second-level bitmap represents a 128M address space, and each bit in the third-level bitmap represents a 4k address space. The first-level bitmap is resident in memory, and the second and third levels adopt a swap-in and swap-out mechanism to ensure that the metadata occupies a small space. The relationship between the three-level bitmaps is: if a bit of the first-level bitmap is 1, it means that there is a 1 in the corresponding second-level bitmap, similarly, if there is a 1 in the second-level bitmap, it means 1 exists in its corresponding third-level bitmap. Through the three-level bitmap structure, the bitmap traversal time can be reduced.

In the general snapshot system of the present invention, when data changes between logical devices, metadata is used to reflect the above changes, that is, metadata has the function of indicating data increment. When the metadata recorded in bitmap mode represents the increment between two snapshots at different time points, its specific meaning is as follows:

The bitmap value is 1: it means that there are write operations between the time points of these two snapshots;

A bitmap value of 0: indicates that there were no write operations between the two snapshot points in time.

Data in logical devices can be read through metadata. However, in the general snapshot system of the present invention, the snapshot implementation technology adopted in the snapshot group may be COW snapshot implementation technology or ROW snapshot implementation technology. Therefore, the metadata recorded in bitmap mode has its own specific meaning in the two snapshot implementation technologies, and the internal semantics are different when metadata is used to read data. The different internal semantics involved here will be involved in detail when using the general snapshot system of the present invention to create and delete snapshots and read snapshot data, which will be described in detail below.

After the above description of the structure of the general snapshot system of the present invention, the method of using the general snapshot system of the present invention will be further described, including the following steps:

Step 1. The user issues a command to create a snapshot through the device management tool set, and decides which snapshot technology to choose to create a snapshot in the command;

Step 2, the snapshot mechanism layer in the snapshot implementation module creates a snapshot according to the snapshot technology selected in step 1;

Step 3. According to the needs of the user, read the data in the snapshot;

Step 4. Complete the operation of writing data in the snapshot according to the needs of the user;

Step 5, delete the snapshot according to the user's requirement.

In the above steps, the creation of the snapshot, reading of the snapshot data, writing of the snapshot data, and deletion of the snapshot involved may vary due to different snapshot technologies, which will be described separately below.

The process of creating a ROW snapshot is shown in Figure 7. When only the source device exists, the physical device used is physical device 1. After the snapshot is created, the physical device 2 assigned to the snapshot device is handed over to the source device for use, and the source device The used physical device 1 is handed over to the snapshot device.

In the ROW mode, since the metadata device and the data device are exchanged together when creating a snapshot, the 1 in the bitmap has a one-to-one correspondence with the incremental data on the data device. When accessing snapshot data at a certain point in time, the meanings of the values in the bitmap are as follows:

The value of the bitmap is 1: it means that the data of the data block corresponding to the corresponding bit is stored on the data device of the snapshot.

A bitmap value of 0: indicates that the data of the data block corresponding to the corresponding bit is not on the data device of the snapshot.

As shown in Figure 8, there is a one-to-one correspondence between the ROW bitmap and the data location. If the bitmap is 1, it means that the data is on the device, and if it is 0, it needs to search backward. For example, in Fig. 8, to read the A data of state 6, it is necessary to search backward to the s1 time point to determine the data position.

The COW snapshot creation process is shown in Figure 6. In the COW mode creation process, only metadata devices are exchanged, and data devices are not exchanged. The dependency relationship among logical devices of a device group is different from the ROW method. For example, the source device corresponds to physical device 1. After the snapshot is created, the source device still corresponds to physical device 1, and snapshot s1 corresponds to physical device 2.

In COW mode, writes that occur after the snapshot is created cause the COW process to copy data into the snapshot. Therefore, before COW is performed, 1 in the metadata of the snapshot device cannot indicate that the data is stored on the data device of the snapshot. At this time, 1 only indicates that a COW operation was performed on the corresponding data block at that time. In conjunction with Figure 6, the method for determining the data location in the COW mode is described as follows:

When reading snapshot S1, read the bitmap of snapshot S2 at the next point in time. If the value of the bitmap is 1, it means that the data of the corresponding data block is stored on the data device of snapshot S1; if the value of the bitmap is 0 , then continue to search along the chain to the snapshot at the later point in time, if the bitmap value of the snapshot S _k is 1, it means that the data of the corresponding data block is stored on the data device of the snapshot S _k-1 ; if in the search process When the source device is encountered in , and the bitmap value of the source device is 1, it means that the data of the data block is stored on the data device of the adjacent snapshot of the source device; if the bitmap value of the source device is 0, it means that the data block The data is stored on the source data device.

As shown in Figure 9, the 1 state is that the s1 snapshot device is created after the source writes four blocks of ABCD. At this time, s1 holds the bitmap of the source in the s0 state, and the bitmap of the source is cleared. In state 2, write blocks 1 and 3. At this time, the COW operation will be triggered to copy the data from the source device to the s1 snapshot device. The corresponding bits of the bitmap of the source are set, and the bitmap of the snapshot is unchanged. In state 4, write blocks 2 and 3 of the source. At this time, the COW operation is triggered to copy the data from the source device to the s2 snapshot device, and so on. When reading the data at the time point of s1, it is necessary to look forward. For example, in state 6, when reading D data, it is necessary to look forward to the s3 device to finally determine the location of the data.

When metadata is stored in a physical device, it can be stored together with the data on the same physical medium for mixed storage, or it can be stored on an independent physical device and isolated from the physical medium where the data resides. In this embodiment, considering efficiency and scalability issues, a scheme of separating metadata devices is finally adopted.

After explaining the basic composition of the general snapshot system of the present invention, the snapshot system creates snapshots, deletes snapshots, reads data, and migrates data.

Figure 6 and Figure 7 are the corresponding descriptions of the snapshot creation process, in which Figure 6 is the process of creating a snapshot using the COW snapshot implementation technology, and Figure 7 is the process of creating a snapshot using the ROW snapshot implementation technology.

In Figure 6, there are four snapshot relationship pointers in both the snapshot device and the source device, respectively pointing to the physical devices that need to be accessed by their read and write operations, and recording the dependencies between snapshots and between snapshots and source devices. The snapshot creation process in COW mode is as follows: first create a source device, when creating, point all the snapshot relationship pointers of the source device to the physical device of the source device, and at this time only the source device itself is in the snapshot list; then at the next time Click to create the first snapshot. During the process of inserting the snapshot into the snapshot linked list, modify the w0 pointer of the source device to point to the snapshot, and the r0 pointer of the snapshot to point to the source. If the created snapshot is a read-only snapshot, you need to set w0 and The w1 device is modified to a null pointer; when creating a second snapshot for the source device, when inserting the snapshot device 2 into the linked list, modify the w0 pointer of the source device to point to the snapshot device 2, the r0 pointer of the snapshot device 2 to point to the source, and the snapshot device 1’s The r0 pointer points to the snapshot device 2. If the snapshot is a read-only snapshot, the w0 and w1 devices of the snapshot device 2 also need to be modified to null pointers. Through the above process, the dependencies between snapshot devices at different time points are established.

Figure 7 is similar to the snapshot creation process in Figure 6, except that the four pointers are different. In the ROW mode, the r0 pointer of the source device points to the snapshot device, and the snapshot device does not point back to the source device. When there are multiple snapshot devices, the snapshot chain of ROW is from the snapshot at the latest point in time to the snapshot at the oldest point in time. As shown in Figure 7, s1 is the earliest snapshot device created, and its time point is the oldest, and s2 is the last The created snapshot has the latest time point. The snapshot chain is directed from the source device to the s2 and s1 devices in turn, which is just the opposite of the COW method.

When reading data in the general snapshot system, it can be divided into reading data in the source device and reading data in the snapshot device.

When reading data in the source device, no matter it is in COW mode or ROW mode, the process of reading data is the same, including the following steps:

Step A1. Query the corresponding bit in the bitmap according to the address of the data to be read. If the value of the bit is 1, read it locally and return the read data value. If the value of the bit is 0 , then go to the next step;

Step A2. Determine whether there is a corresponding snapshot for the data in the source device. The determination algorithm is: if the r0 pointer points to a physical device, there is no snapshot; if it is a logical device, there is a snapshot. If there is no snapshot, all zero data will be returned. If there is a snapshot, the read data request will be forwarded according to the snapshot chain, and the snapshot data reading process will be entered.

In the process of reading snapshot data, depending on the snapshot implementation technology, there are obvious differences in reading, which are explained below.

In COW mode, the process of reading snapshot data is as follows:

Step B1, determine the physical device where the data is stored according to the value of the bitmap; the judgment algorithm is: traverse forward from the tail of the snapshot linked list, if the bitmap value on a certain snapshot S _k is 1, then in the previous snapshot S k Read data on the _k-1 data device, if the bitmap value is 0, then forward the read data request to the previous snapshot along the snapshot list until a snapshot with a bitmap value of 1 is found;

Step B2. No snapshot with a bitmap value of 1 is found in the snapshots of the snapshot linked list, and the read data request reaches the source device. If the bitmap value of the source device is 1, read on the data device of the snapshot adjacent to the source device data, otherwise read the data on the source's data device. For the specific implementation of this step, refer to the description of FIG. 9 above.

In ROW mode, the process of reading snapshot data is as follows:

Determine the physical device where the data is stored according to the value of the bitmap, and then read the data from the physical device where the data is stored. In ROW mode, the judgment algorithm for determining the data storage location is: traverse forward from the tail of the snapshot linked list, if the bitmap value on a certain snapshot S _k is 1, read it on the local data device and return Corresponding data, if the bitmap value is 0, then judge whether the current snapshot is the oldest snapshot in the snapshot linked list, that is, whether it is the snapshot at the end of the snapshot linked list, if so, return all 0, otherwise, forward the read data request to the snapshot linked list In the previous snapshot in . For the specific implementation of this step, refer to the description of FIG. 8 above.

In the process of writing data to the source, depending on the snapshot implementation technology, there are also differences when writing data, which will be explained separately below.

In COW mode, the process of writing data is as follows:

Step a1, determine the physical device where the data is stored according to the value of the bitmap;

Step a2, write data according to the physical device determined in step a1.

The specific implementation method is: if the value of the bitmap is 1, write it directly into the source physical device; if the value of the bitmap is 0, write it into the source physical device after completing the COW operation.

In ROW mode, the process of writing data is as follows:

Step b1, determining the physical device where the data is stored according to the value of the bitmap;

Step b2, writing data according to the physical device determined in step b1.

The specific implementation method is: if the value of the bitmap is 1, it is directly written into the source physical device; if the value of the bitmap is 0, the data of the data block is merged and then written into the source physical device.

Similarly, in the general snapshot system, there are differences in deletion of snapshots due to different snapshot technologies. The following describes the deletion process of snapshots in different situations.

In the COW method, when there are multiple time snapshots, in order to simplify the design and implementation, we stipulate that only the oldest point-in-time snapshot can be deleted to avoid data migration operations.

In the ROW mode, there is a need to delete the newly created snapshot. From the relationship in Figure 7, it can be seen that snapshots at different time points have dependencies, and data at the same time point may be distributed on different physical devices. Therefore, it is necessary to The data is merged. Taking FIG. 7 as an example, the data merging method triggered when the snapshot device is deleted is described.

In Figure 7, S1 and S2 are snapshot devices at two adjacent time points. Assuming that snapshot device S1 is to be deleted, the specific data merging process includes:

Step 10, sequentially read the corresponding bit of each data block in the bitmap in the snapshot device S2;

Step 20. Combining the bitmap bits in the snapshot device S2 with the corresponding bitmap bits in the snapshot device S1, four situations can be obtained:

S2 S1

a.0 0

b. 0 1

c, 1 0

d, 1 1

When the bitmap bit of S2 is 1, it means that new data has been written. Therefore, only the data block whose bitmap bit of S2 is 1 needs to be copied back to the data device of S1, regardless of the bitmap bit of S1. What is the value. This copying process is called a data move back operation.

Step 30, perform a data move back operation, and copy the data in the data device of S2 to the data device of S1.

Step 40, after the migration is completed, set the corresponding bit of the bitmap of S2 to 0;

Step 50, if all data blocks have been processed, execute the next step; otherwise, re-execute step 10, and continue to process the next data block.

Step 60, lock the two logical devices, change the value of the bitmap and r1 pointer at the same time, and exchange the two physical devices.

Step 70, releasing resources such as memory and physical devices occupied by the deleted device.

Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention rather than limit them. Although the present invention has been described in detail with reference to the embodiments, those skilled in the art should understand that modifications or equivalent replacements to the technical solutions of the present invention do not depart from the spirit and scope of the technical solutions of the present invention, and all of them should be included in the scope of the present invention. within the scope of the claims.

Claims (17)

1. A snapshot system, comprising a device management tool set, a snapshot implementation module, and a physical device; it is characterized in that, the snapshot implementation module includes a snapshot mechanism layer for data search and point-in-time management, and is used for The physical resource management layer for allocation and recovery management of physical equipment; The snapshot mechanism layer includes at least one snapshot group, which is created by copy-on-write technology or redirection-on-write technology; the snapshot group includes physical devices r0 and r1 that need to be accessed during corresponding read operations Pointers, and w0 and w1 pointers used to correspond to physical devices that need to be accessed during write operations; in the snapshot group, there are corresponding snapshot devices at a certain point in time, and a chain structure is adopted between snapshot devices at each point in time to organize; The physical resource management layer includes metadata devices and data devices corresponding to the snapshot group; wherein, The device management tool set is used to issue a command to create a snapshot, and the command includes information about which snapshot technology to choose to create a snapshot; the snapshot mechanism layer in the snapshot implementation module creates a source according to the command to create a snapshot device, pointing all the snapshot relationship pointers of the source device to the physical device of the source device, then creating a snapshot at a subsequent point in time, and inserting the created snapshot into the snapshot set linked list; When the command uses copy-on-write technology to create a snapshot, inserting the created snapshot into the snapshot set linked list includes pointing the w0 pointer of the source device to the newly created snapshot device, and setting the r0 pointer of the newly created snapshot device Pointing to the source device, if there is a snapshot device with an earlier time point in the snapshot set linked list, the r0 pointer of the snapshot device with an earlier and closest time point than the newly created snapshot device points to the new Created snapshot device; When the command uses redirection-on-write technology to create a snapshot, inserting the created snapshot into the snapshot set linked list includes pointing the r0 pointer of the source device to the newly created snapshot device, and inserting the newly created snapshot The r0 pointer of the device points to the snapshot device with an earlier time point and the closest time point; if there is no snapshot device with an earlier time point in the snapshot set linked list, the r0 pointer of the newly created snapshot device points to a zero device. 2. The snapshot system according to claim 1, wherein metadata is stored in the metadata device, and the metadata is used to indicate the increment of data and the data in the snapshot version of the snapshot group. The storage location on the data device; the metadata is recorded in the form of a bitmap or a mapping table. 3. The snapshot system according to claim 2, wherein the bitmap adopts a multi-level structure division, and the value of a certain bit of the upper-level bitmap is 1 to indicate the corresponding lower-level bitmap There is a bit with a value of 1, and if it is 0, it means that there is no bit with a value of 1 in the corresponding next-level bitmap; on the lowest-level bitmap, a bit with a value of 1 is used to indicate the corresponding data block The data on the block changes, if it is 0, the data on the corresponding data block does not change. 4. A method for using the snapshot system according to any one of claims 1-3, comprising the following steps: Step 1), the user issues a command to create a snapshot through the device management tool set, and decides which snapshot technology to choose to create a snapshot in the command; Step 2), the snapshot mechanism layer in the snapshot implementation module creates a snapshot according to the snapshot technology selected in step 1); Step 3), perform relevant operations on the snapshot according to user requirements, and the operations include reading, writing and deleting the snapshot. 5. The method for using the snapshot system according to claim 4, characterized in that, in said step 2), said creating a snapshot further includes the following steps: Step 2-1-1), create a source device, point all the snapshot relationship pointers of the source device to the physical device of the source device; Step 2-1-2), creating a snapshot at a subsequent point in time, and inserting the created snapshot into the snapshot set linked list. 6. The method for using the snapshot system according to claim 5, wherein if the copy-on-write technology is used when creating the snapshot, when inserting the created snapshot into the snapshot set linked list, the source device The w0 pointer of the newly created snapshot device points to the newly created snapshot device, and the r0 pointer of the newly created snapshot device points to the source device. The r0 pointer of the snapshot device whose device is earlier than the time point and whose time is closest points to the newly created snapshot device. 7. The method for using the snapshot system according to claim 5, wherein if the redirection-on-write technology is used when creating the snapshot, when inserting the created snapshot into the snapshot set linked list, the source The r0 pointer of the device points to the newly created snapshot device, and the r0 pointer of the newly created snapshot device points to the snapshot device with an earlier time point and the closest time point; if there is no snapshot with an earlier time point in the snapshot set linked list device, the r0 pointer of the newly created snapshot device points to the zero device. 8. The method for using the snapshot system according to claim 4, characterized in that, in said step 3), said reading the data in the snapshot includes reading the data in the source device and reading the snapshot device Reading of data in. 9. The method for using the snapshot system according to claim 8, wherein the reading of the data in the source device further comprises the following steps: Step 3-1-1), query the corresponding bit in the bitmap according to the address of the data to be read, if the value of the bit is 1, then directly read locally, and return the read data value, If the value of this bit is 0, execute the next step; Step 3-1-2), determine whether there is a corresponding snapshot for the data in the source device, if there is no snapshot, then return all zero data, if there is a snapshot, forward the read data request according to the snapshot set linked list, pass the snapshot The data reading completes the data reading process. 10. The method for using the snapshot system according to claim 9, wherein when judging whether there is a corresponding snapshot for the data in the source device, if the r0 pointer in the source device points to a physical device, Then there is no snapshot; if the r0 pointer in the source device points to a logical device, then there is a snapshot. 11. The method for using the snapshot system according to claim 8, wherein when the snapshot is created using the copy-on-write technology, step 3) further includes the following steps for reading data from the snapshot: Step 3-2-1), determine the physical device where the data is stored according to the value of the bitmap, during the determination process, traverse forward from the tail of the snapshot set linked list, if the bitmap value on a certain snapshot S _k is 1, Then read data on the data device of its previous snapshot S _k-1 , if the bitmap value is 0, then forward the read data request to the previous snapshot along the snapshot set linked list until the bitmap value of 1 is found snapshot; Step 3-2-2), if no snapshot with a bitmap value of 1 is found in the snapshots of the snapshot set linked list, the read data request reaches the source device. If the bitmap value of the source device is 1, it is adjacent to the source device Read data on the data device of the snapshot, otherwise read data on the source data device; When the snapshot is created using redirection-on-write technology, step 3) also includes the following steps for reading data from the snapshot: First, determine the physical device where the data is stored according to the value of the bitmap, and then read the data from the physical device where the data is stored; when determining the physical device where the data is stored, it traverses forward from the tail of the snapshot set linked list. A bitmap value on the snapshot S _k is 1, then read on the local data device, and return corresponding data, if the bitmap value is 0, then judge whether the current snapshot is the oldest snapshot in the snapshot set linked list, That is, whether it is a snapshot at the end of the snapshot linked list, if so, return all 0, otherwise, forward the read data request to the previous snapshot in the snapshot linked list. 12. The method for using the snapshot system according to claim 4, wherein if the data writing operation is completed in the snapshot, the step 3) further includes the following steps: First, determine the physical device where the data is stored according to the value of the bitmap; then write data in the determined physical device. 13. The method for using the snapshot system according to claim 12, wherein when the snapshot is created using the copy-on-write technology, step 3) further includes the following steps for writing the snapshot data: Judge the bitmap value, if the bitmap value is 1, write it directly into the source physical device; if the bitmap value is 0, write it into the source physical device after the copy-on-write operation is completed. 14. The method for using the snapshot system according to claim 12, wherein when the snapshot is created using redirection-on-write technology, step 3) further includes the following steps for writing the snapshot data: The bitmap value is judged, if the bitmap value is 1, it is directly written into the source physical device; if the bitmap value is 0, the data of the data block is merged and then written into the source physical device. 15. The method for using the snapshot system according to claim 4, characterized in that, in the step 3), when the snapshot adopts the copy-on-write technology, only time can be deleted in the process of deleting the snapshot Click on the oldest snapshot. 16. The method for using the snapshot system according to claim 4, characterized in that, in the step 3), when the snapshot adopts redirect-on-write technology, before performing the delete operation on the snapshot to be deleted , to merge the data to be triggered by deleting the snapshot. 17. The method for using the snapshot system according to claim 16, characterized in that said data merging comprises the following steps: Step 3-3-1), sequentially read the corresponding bit of each data block in the snapshot device S2 in the bitmap; wherein, S2 is the adjacent snapshot device of the snapshot device S1 to be deleted; Step 3-3-2), the bitmap bit in the snapshot device S2 is combined with the corresponding bitmap bit of the snapshot device S1, four situations can be obtained: S2 S1 a. 0 0 b. 0 1 c, 1 0 d. 1 1 When the bitmap bit of S2 is 1, it means that new data has been written, so only the data block whose bitmap bit of S2 is 1 needs to be copied back to the data device of S1 during data merging; Step 3-3-3), execute the data move back operation, copy the data in the data device of S2 to the data device of S1; Step 3-3-4), after the data transfer is completed, set the corresponding bit of the bitmap of S2 to be 0; Step 3-3-5), if all data blocks have been processed, end the data merging operation, otherwise re-execute step 3-3-1), and continue to process the next data block.

Images