CN102841758B - High-performance virtual disk management system - Google Patents

Abstract

The invention discloses a high-efficiency virtual disk management system, which is provided with a client computer, wherein the client computer is connected with a storage server through a network, the storage server is connected with a block-level storage medium through a virtual disk management system, the virtual disk management system is used for establishing and accessing a plurality of virtual disks with dynamic space configuration effects on the storage medium, and the virtual disk management system uses a management and control information packet comprising an entity section index table, a storage medium group record table, a virtual disk entity section occupation table and a difference disk relation table, so that the address conversion of an entity space and a virtual space can be calculated at one time, and the access efficiency of the virtual disks can be improved.

Description

High-performance virtual disk management system

technical field

The present invention relates to a high-efficiency virtual disk management system, especially capable of one-time calculation of entity and virtual space address conversion, so as to improve the performance of virtual disk access, and to realize block-level storage media like virtual disks. A high-performance virtual disk management system with the utility of dynamic space allocation.

Background technique

US Patent No. US 7,107,385B2 proposes the invention of Storage Virtualization by LayeringVirtual Disk Objects on a File System, the content of which is to use the file system of the operating system to support a wide range of storage media, and to store different types of storage media, such as extended disks or network storage media , the volume section (Volume) of the file system is assembled to store the virtual disk.

Patent No. I307026 of Taiwan Province of China proposes a storage management system and method. Its invention uses a file system server, a file information (Metadata) server and an object storage device to achieve a virtual partition for file access. The file information server stores the file management and index information, and then the object storage device accesses the file content data. Because the file management and index information and the real file content are stored separately on two storage servers, the virtual partition space can be dynamically grown by adding storage units of the object storage device.

The above existing virtual disk technologies all intend to provide flexible correspondence between physical disk space and virtual disk space, and achieve this by using the file system management capability of the operating system or separating file management and index information from the real file content. However, these technologies require a certain degree of additional computing consumption to achieve, that is, the access of a file will go through at least two layers of address translation, resulting in a performance bottleneck for applications that require high-speed virtual disk access.

Contents of the invention

The main purpose of the present invention is to provide a high-performance virtual disk management system, which uses the virtual disk management system to plan block-level storage media into a set of physical sections with dynamic sizes, and then directly writes virtual disk segments to multiple A physical segment, to avoid the address conversion required by the file system of the operating system in the prior art, and then become a one-time calculation of the physical and virtual space address conversion, so as to improve the performance of virtual disk access and enable storage media to be realized. The utility of dynamic space allocation like virtual disks.

In order to achieve the above purpose, the present invention is to connect a virtual disk management system between the storage server and the storage medium, and the storage server is connected to the client computer through the network, and the virtual disk management system is provided with a disk segment planner, a control area segment processor, disk access and scheduler, virtual disk driver, disk segment allocation and reclaimer, virtual disk storage segment table register, virtual disk write protection Disk data cache and buffer, where:

The disk segment planner is used to plan the storage medium into a plurality of physical segments with a given fixed size in advance. A physical segment is the smallest unit of use. After the disk segment planner plans the storage medium, it will generate A physical segment index table stores the status of the physical segment of the storage medium being used. If it is not configured, it is represented by 0; otherwise, it is represented by 1. In the initial state, the values in the index table are all 0; moreover, If multiple storage media are connected to the virtual disk management system, the disk segment planner will use a consistent planning method for each physical storage media, and use a storage media group record table to store all available storage media information , each storage medium will be assigned a unique identification code.

The control section processor will select multiple unconfigured physical sections in each storage medium as the control section for management purposes. One storage medium will store multiple control sections, but only one is the main one. Others are used for backup and inspection to increase the security of important data, and the control section processor will write the entity section index table and storage medium group record table into the control section in the storage medium.

The disk access and scheduler is a drive component for accessing and scheduling storage media, so that data commands for access management purposes have a higher priority execution order than data access commands for virtual disks.

The virtual disk driver is a controller for generating, deleting and mounting virtual disks on the storage medium. When the client computer issues a command to create, delete or mount a virtual disk to the storage server, the storage server forwards the command to A virtual disk driver, the virtual disk driver obtains a new physical section through disk section allocation and reclaimer, reclaims a physical section occupied by a certain virtual disk, or queries all physical sections occupied by the virtual disk.

The disk segment allocation and collector is used to execute and complete the addition, deletion and query of the physical segment occupied by a virtual disk issued by the virtual disk driver. When the disk segment allocation and collector executes the command, it will First, the management and control section processor obtains the latest physical section index table to know which physical sections can be allocated or have been occupied; moreover, for the instruction of adding a new virtual disk or requiring configuration of the physical section For the request, this disk segment allocation and collector will select the physical segment allocation from the available physical segments based on the principle of optimizing access performance. This optimal access performance can be selected by preferentially selecting adjacent The physical section is reached, and a virtual disk physical section occupation table is generated. The index of the virtual disk physical section occupation table is the unique identification code of the virtual disk in the virtual disk management system. The value corresponding to this identification code is a binary string The elements of each group of data in the string are composed of the identification code of the storage media group record table and the entity segment index of the entity segment index table, and if the disk segment allocation and recycler execute new and delete virtual The command of the disk causes the change of the virtual disk physical segment occupation table or the physical segment index table, and the updated virtual disk physical segment occupation table or physical segment index table will be sent to the control section processor to update the control section record of.

The virtual disk storage segment table temporary register is a cache register that speeds up addressing the virtual disk at the location of the storage medium. If there is enough space, it will save the virtual disk entity segment of the virtual disk in use The contents of the occupancy table are stored in a temporary register that can be accessed at a high speed (compared to disk access). This enables the virtual disk write protection transponder to quickly complete the calculation of the physical segments that can be accessed by the virtual disk. When any virtual disk physical segment occupancy table changes, the control segment processor will update the virtual disk. Store the virtual disk physical segment occupancy table in the segment table temporary register to keep the data consistent.

The virtual disk write-protection transponder is responsible for the address translation and checking of the virtual disk’s physically accessible physical segments; The virtual disk is write-protected to the indexer to calculate the physical address of the virtual disk stored in the storage medium, and then the data access of the virtual disk is performed. However, any access command to a virtual disk should not exceed the virtual disk The physical section occupation table records all the physical section ranges that can be accessed by the virtual disk to protect the data of multiple virtual disks and avoid mutual reading and writing of data, but the exception is when the virtual disk is going to a certain When new data is written to the legal disk virtual location (not exceeding the capacity range of the virtual disk) that is configured but will soon obtain the configured physical storage space, it will request the disk segment allocation and recycler to configure a new physical segment. When adding a new After the physical section, the original access command will meet the protection conditions and can be executed normally. Furthermore, if the legal virtual location of an unconfigured physical storage space on the disk is read, that is, the read command reads For addresses that have not been written into data, it is only necessary to return blank data, and there is no need to configure a new physical segment. In addition, the configuration of discontinuous physical sections may cause access instructions to operate multiple physical sections. Such cross-physical section operations need to decompose an instruction into multiple instructions for accessing data in a single physical section. After the instruction is completed, the original instruction is considered to be completed; further, functions similar to snapshot disks or differential disks (Snapshot or Differential Disks) defined by traditional virtual disk technology are also taken care of by the virtual disk write protection transponder, which is collectively referred to as differential disks , the difference disk is also regarded as a virtual disk in essence, assuming that a virtual disk A already exists in the system and the user creates a difference disk B for virtual disk A, at this time, writing data to virtual disk A will be transferred Save to differential disk B; if data is read from virtual disk A, there are two possibilities: first, if the data written in differential disk B is to be read, it will be read from differential disk B; second , if it is to read the data that has not been written in the difference disk B, it will be read directly from the virtual disk A. Therefore, the difference disk will make its corresponding virtual disk into a read-only state; in order to achieve the above functions, the virtual disk The write protection indexer will create a difference disk relationship table. The index field of the difference disk relationship table is the unique number of a difference disk, and its corresponding value is the number of the virtual disk connected to the difference disk. If there are multiple identical virtual disks numbered record, it means that a certain virtual disk has created multiple differencing disks. If you want to delete a difference disk, you only need to delete the records in the difference disk relationship table, then reset the corresponding position of the entity section index table to release the entity section occupied by it, and finally update the content of the control section . If a virtual disk is deleted, all the difference disk records created on the virtual disk in the difference disk relationship table must be deleted synchronously, and the contents of the control section must be updated synchronously; finally, if it is a multi-level difference disk relationship, Just fill in a certain difference disk number in the virtual disk field in the difference disk relationship table.

The virtual disk accessor, which is an implementation component of traditional virtual disk technology, is responsible for writing the (virtual) disk obtained by the client in the traditional virtual disk format, such as VMDK, VHD, and VDI, into the storage medium. When calculating the actual access storage medium address of the traditional virtual disk format, it is only necessary to add the offset of the physical segment obtained from the lookup table of the virtual disk write protection indexer, that is, data access only needs one time The calculation is done.

The virtual disk data cache and buffer are used to speed up the virtual disk data access cache and buffer to reduce the number of reads to the storage medium, and allow the operation of writing data to be completed asynchronously, reducing writing command response time; the difference between this virtual disk data cache and buffer and traditional data cache and buffer is that the virtual disk write protection indexer provides a differential disk function, resulting in a virtual disk and one or more associated Differential disks have different access frequencies. Suppose the same address is read from multiple differentiating disks associated with a certain virtual disk. If most of the differentiating disks have not written to this address, the actual read Fetching the address will return to reading the same address on the virtual disk. Based on this feature, caching the data of the associated virtual disk is faster than fetching the data of its different disks, in terms of cache space utilization and hit rate It is more efficient; similarly, in terms of the locality (locality) of data access with region and time, the data that has just been written may be referenced immediately. Therefore, writing to the virtual disk data cache and buffer can not only Completing the write command in an asynchronous manner like the traditional method not only reduces the response time, but also allows the newly written data to have a higher chance of being cached, especially the data written to the difference disk; simply put, the virtual disk data is fast For read commands, the fetch and buffer will enhance the data cache of the associated virtual disk, and for write commands, it will enhance the data cache of different disks.

Description of drawings

FIG. 1 is a schematic diagram of the present invention connecting to a storage server.

Figure 2 is a block diagram of the present invention.

Fig. 3 is the content of the control section of the present invention.

FIG. 4 is a flow chart of the present invention for initializing storage media.

FIG. 5 is a flow chart of the present invention in creating, deleting and mounting virtual disks.

FIG. 6 is a flow chart of the present invention in processing access commands.

Explanation of reference signs: 1. client computer; 2. network; 3. storage server; 4. virtual disk management system; 41. disk segment planner; 42. control segment processor; 43. disk access and Scheduler; 44. Virtual disk driver; 45. Disk segment allocation and reclaimer; 46. Virtual disk storage segment table temporary register; 47. Virtual disk write protection redirector; 48. Virtual disk access device ;49. Virtual disk data cache and buffer; 5. Storage medium; 51. First storage medium; 511. Entity section; 512. Control section; 5121. Storage medium group record table; 5122. First entity area Segment index table; 5123, second entity segment index table; 5124, difference disk relationship table; 5125, virtual disk entity segment occupancy table; 52, second storage medium; 521, entity segment.

Detailed ways

Please refer to Fig. 1 and Fig. 2, it can be clearly seen from the figure that the high-efficiency virtual disk management system of the present invention is used in that the client computer 1 is connected to one or more storage servers 3 through the network 2, and the storage The server 3 is connected to one or more storage media 5 through the virtual disk management system 4, and the virtual disk management system 4 is provided with a disk segment planner 41, a management and control segment processor 42, a disk access and scheduler 43, A virtual disk driver 44, a disk segment allocation and reclaimer 45, a virtual disk storage segment table register 46, a virtual disk write protection redirector 47, a virtual disk access device 48, and a virtual disk data cache and buffer 49.

Please refer to FIG. 1 to FIG. 4, when the storage medium 5 is performing the initialization process, it is completed by the joint efforts of the disk segment planner 41, the management and control segment processor 42, and the disk access and scheduler 43. The disk access and scheduler 43 first detects the storage media 5 connected to the high-performance virtual disk management system, and sends the quantity of the storage media 5 to the disk segment planner 41; in this embodiment, two sets of storage media are used 5 for illustration, so that the disk access and scheduler 43 detects all connected and usable storage media 5, which are two groups of the first storage media 51 and the second storage media 52, represented by numbers 0 and 1 respectively. At this time, the disk segment planner 41 creates a storage medium group record table 5121 (as shown in FIG. 3 ) according to the number of storage media 5 detected by the disk access and scheduler 43, and records the storage medium group The table 5121 is handed over to the management and control section processor 42, and after the disk section planner 41 knows the number of storage media 5, the disk section planner 41 will use a predetermined fixed section capacity value to allocate the first storage medium 51 and the second storage medium 52 are planned into a plurality of physical sections 511 and 512 in a consistent planning manner, and generate a first physical section index table 5122 and a second physical section index table 5122 for the first storage medium 51 and the second storage medium 52 respectively. Physical segment index table 5123; in this embodiment, when the capacity of the first storage medium 51 is 1 Terabyte, it is planned to be 1024 physical segments 511 with a fixed capacity of 1 Gigabyte (GB), each The physical section 511 is the smallest unit of use; the actual address of a certain section in the first storage medium 51 can be quickly addressed by using the section index value, for example: the i-th section starts at the address of i x 1GB , terminated at the address of (i+1)x 1 GB 1, after determining the capacity of the physical segment 51, the disk segment planner 41 then generates a first physical segment index table 5122 to store the used physical segment 511 If it is not configured, it is represented by 0; otherwise, it is represented by 1. In the initial state, the values in the index table are all 0. Then let the disk segment planner 41 hand over the first physical segment index table 5122 and the second physical segment index table 5123 to the management and control segment processor 42, and the management and control segment processor 42 will select for each storage medium 5 Determine a plurality of (in this example, the 0th and the last one of the first storage medium 51) entity sections 511 as primary and backup management and control sections 512, and set the 0th and last entities in the first entity section index table 5122 Fill in the segment index with 1, which means it has been allocated and used, and let the disk segment planner 41 write the first physical segment index table 5122, the second physical segment index table 5123 and the storage media group record table 5121 into the control Section 512.

Please refer to FIGS. 1 to 3 and 5. After the work of initializing the storage medium 5 is completed, the virtual disk management system 4 can handle the creation, deletion and mounting of the virtual disk from the client computer 1 through the storage server 3. As shown in Figure 5, this type of operation is the same as the action of a traditional network hard disk being mounted as an operable storage space on the file system. For example, the storage server 3 can be an iSCSITarget end, and the user The terminal computer 1 may be an iSCSI Initiator terminal. The virtual disk driver 44, disk segment allocation and reclaimer 45, management and control segment processor 42, and disk access and scheduler 43 of the virtual disk management system 4 are used to perform the creation, deletion, and mounting of virtual disks. Work together. When the client computer 1 sends an operation instruction through the storage server 3, the storage server 3 will transmit the operation instruction to the virtual disk management system 4. At this time, the virtual disk driver 44 first judges the type of the operation instruction, if it is a new virtual disk command, then the disk segment allocation and reclaimer 45 will be requested to configure a new physical segment 511 or a new physical segment 521. This embodiment is illustrated with the No. 1 and No. 52 physical segments 51 of the first storage medium 51, and will The newly created virtual disk is given a unique serial number, assuming it is number 0, and the index value of the newly allocated physical section 511 is recorded in the virtual disk physical section occupation table 5125, and the size of the newly created disk is filled in the command value, that is, the maximum effective addressing space of the disk (in this example, 2GB, as shown in Figure 3), and the capacity of the virtual disk can exceed the actual allocated capacity, as long as the expansion of the actual physical section 511 is completed before use That's it. Next, the management and control section processor 42 updates the relative position of the first physical section index table 5122 to be occupied, that is, updated to 1. Furthermore, if it is an instruction to create a new differential disk, assuming that the newly created virtual disk 1 is the differential disk of virtual disk 0, then it is necessary to add the association record between the new disk and the virtual disk 0 specified by the parameter in the differential disk relationship table 5124 (such as As shown in FIG. 3 ), the capacity value of the created difference disk is equal to its associated virtual disk, and the difference disk relationship table 5124 will be written by the management and control section processor 42 and the disk access and scheduler 43 after being updated. The process of the management and control section 512 is the same as the above-mentioned initialization of the storage medium 5 and writing into the management and control section 512 , so it will not be repeated hereafter. If the virtual disk driver 44 receives an instruction to delete a virtual disk, the number of the target disk to be deleted must be specified as a parameter in the instruction, first release the occupied physical section 511 or physical section 521, and update the relevant first physical section The index table 5122 or the second physical section index table 5123 is in an unconfigured state, and then, only need to find and delete the record in the virtual disk physical section occupation table 5125, and the virtual disk can be eliminated; if the deleted object is a difference disk, the record of the difference disk needs to be deleted from the difference disk relationship table 5124; if a virtual disk with a difference disk has been created, all the difference disks associated with it also need to be deleted together. Also, for the command to mount a virtual disk, the virtual disk driver 44 only needs to check whether the target virtual disk has already existed in the virtual disk physical section occupation table 5125 through the control section processor 42, and check whether there is a correlation in the difference disk relationship table 5124. Connected differential disks, and then load the physical sections 511 and 521 occupied by them into the system. If the commands received by the virtual disk drive 44 are not of the above types, for example, they may be status inquiry commands of the storage medium 5, these commands will be consistent with the traditional standard component operation flow, so they will not be emphasized in the present invention. Some functions required for management purposes, such as enumerating all registered virtual disks or differential disks, adding or deleting new storage media 5, etc., although not described in detail, can be easily deduced from the above content, and are not emphasized in the present invention .

Please refer to FIG. 1 to FIG. 3 and FIG. 6 , when the virtual disk is mounted by the client computer 1 , then the access command of the virtual disk can be served. This workflow is completed by the management and control section processor 42, disk access and scheduler 43, disk section allocation and reclaimer 45, virtual disk storage section table temporary register 46 and virtual disk write protection indexer 47 . After the virtual disk driver 44 receives the instruction of virtual disk access data, it is transferred to the virtual disk write protection indexer 47 for processing (as shown in Figure 5), and the virtual disk write protection indexer 47 can be used by the virtual disk storage area Segment table temporary register 46 (faster) or disk segment allocation and reclaimer 45 (slower, because it needs to be read by the management and control area 512 segment on the storage medium 5) knows the virtual disk entity segment occupancy table 5125, Get all the physical sections 511, 521 of the storage medium 5 that the virtual disk can access, and then do a legality check first, any access command for the virtual disk should not exceed these ranges, otherwise the command should not be Execution, when the virtual disk is about to write new data on a legal virtual location (not exceeding the capacity range of the virtual disk) of a certain unconfigured physical storage space, it will first request to the disk segment allocation and collector 45 to configure the new data The physical sections 511, 521 of the system, and update the data in the control section 512, so that the originally accessed instructions still meet the protection conditions and are executed normally. The above-mentioned actions can calculate the physical address of the virtual disk stored in the storage medium 5, and then carry out real data access on the storage medium 5 by the virtual disk accessor 48 according to the virtual disk format; this virtual disk format is different from the traditional one. The practice is the same, so it is not shown in the flow chart. Furthermore, access commands are handled differently depending on whether the target disk is a differential disk. When a virtual disk is created as a differential disk, the virtual disk is in a read-only state and is protected from writing data. The data will be dumped to the difference disk; however, there are two possibilities for reading the data: if it is to read the data that has been written on the difference disk, it will be read from the difference disk; if it is to read the data that has not been Data written to the differencing disk will be read directly from the virtual disk.

Finally, the virtual disk data cache and buffer 49 will not be described in detail in the part where the virtual disk access operation principle is the same as the traditional cache method, and the virtual disk data cache and buffer 49 are characterized by their operation For the command, the virtual disk data that has been created as a different disk will be cached first; for the write command, the data written to the different disk will be cached first. If the cache space is insufficient, the data obtained by the read command will be cached first. Takes precedence over data that has just been written.

To sum up, the high-performance virtual disk management system of the present invention can realize the effect of dynamic space allocation on multiple block-level storage media 5 without the support of the file system of the operating system to store and manage multiple virtual disk. And because the present invention can avoid the calculation performance loss of running the operating system and its file system, and reduce the calculation consumption of an expensive file system address conversion, it has high-speed access performance. Therefore, using the technology of the present invention, the utilization rate of the storage space of the traditional storage medium 5 technology can be improved without causing a large loss of access performance.

The above description is only illustrative, rather than restrictive, to the present invention. Those of ordinary skill in the art understand that many modifications and changes can be made without departing from the spirit and scope defined by the following appended claims. Or equivalent, but all will fall within the protection scope of the present invention.

Claims (2)

1. A high-efficiency virtual disk management system, characterized in that it is provided with a client computer, the client computer is connected to a storage server through a network, and the storage server is connected to a block-level storage medium through the virtual disk management system, The virtual disk management system is used to create and access multiple virtual disks with dynamic space allocation effects on the storage medium, and the virtual disk management system uses information packages for management and control, which include physical segment index tables, storage medium groups Record table, virtual disk physical section occupation table and differential disk relationship table, among which: The physical segment index table is used by the virtual disk management system to plan each storage medium into a plurality of physical segments in a consistent planning manner with a predetermined fixed segment capacity value, and then generate a physical segment index table; The storage media group record table is that after the virtual disk management system detects the number of all connected and usable storage media in the virtual disk management system, it will assign a unique identification code to each storage medium, thereby creating a storage media group record table ; The virtual disk physical section occupation table includes the virtual disk number and the physical section index. The virtual disk created by the system is generated by giving a unique sequence code, and the entity segment index is used to record the new entity segment sequence configured by the virtual disk management system; The difference disk relationship table is when the virtual disk management system processes the creation operation instruction of the difference disk from the client computer through the storage server, the virtual disk management system will give the created difference disk a unique sequence code, and will associate with it The serial code of the associated virtual disk and the serial code of the difference disk are recorded in the difference disk relationship table; The virtual disk management system selects a primary and a plurality of secondary physical sections on the storage medium as control sections, and stores the physical section index table, storage medium group record table, virtual disk physical section occupation table and differences disk relationship table, The virtual disk management system obtains the physical section index table from the control section to know which physical sections can be allocated or have been occupied. When the virtual disk management system executes the instructions of creating, deleting and mounting virtual disks, It can allocate the physical section to the virtual disk, reclaim the physical section occupied by the virtual disk, or query the physical section occupied by the virtual disk. 2. A high-efficiency virtual disk management system, characterized in that it is provided with a client computer, the client computer is connected to a storage server through a network, and the storage server is connected to a block-level storage medium through the virtual disk management system, The virtual disk management system is used to create and access multiple virtual disks with dynamic space allocation effects on the storage medium, and the virtual disk management system uses information packages for management and control, which include physical segment index tables, storage medium groups Record table, virtual disk physical section occupation table and differential disk relationship table, among which: The physical segment index table is used by the virtual disk management system to plan each storage medium into a plurality of physical segments in a consistent planning manner with a predetermined fixed segment capacity value, and then generate a physical segment index table; The storage media group record table is that after the virtual disk management system detects the number of all connected and usable storage media in the virtual disk management system, it will assign a unique identification code to each storage medium, thereby creating a storage media group record table ; The virtual disk physical section occupation table includes the virtual disk number and the physical section index. The virtual disk created by the system is generated by giving a unique sequence code, and the entity segment index is used to record the new entity segment sequence configured by the virtual disk management system; The difference disk relationship table is when the virtual disk management system processes the creation operation instruction of the difference disk from the client computer through the storage server, the virtual disk management system will give the created difference disk a unique sequence code, and will associate with it The serial code of the associated virtual disk and the serial code of the difference disk are recorded in the difference disk relationship table; The virtual disk management system selects a primary and a plurality of secondary physical sections on the storage medium as control sections, and stores the physical section index table, storage medium group record table, virtual disk physical section occupation table and differences disk relationship table, The virtual disk management system will store the record content of the virtual disk physical segment occupation table of the virtual disk in use in the high-speed access register, and when any event of virtual disk physical segment occupation table changes occurs, it will Simultaneously update the physical section occupancy table of the virtual disk in the control section and the scratchpad.