HK1206839B - Method for controlling hybrid storage and hybrid storage system - Google Patents

Description

Hybrid storage control method and hybrid storage system

Technical Field

The present application relates to the field of data storage, and in particular to a hybrid storage control method and a hybrid storage system.

Background Art

Hybrid storage technology has become a trend. Hybrid storage refers to a storage system composed of multiple different storage devices according to specific strategies based on their distinct characteristics. Using a hybrid storage system generally results in better system performance than using a single storage system. For example, the random access memory portion of a CPU consists of a level 1 cache (L1Cache), a level 2 cache (L2Cache), a level 3 cache (L3Cache), and main memory. While the data access speed of these storage devices gradually decreases, their capacity gradually increases. Using a specific cache consistency mechanism, large amounts of data can be accessed from a smaller, faster cache, improving overall system performance.

Existing hybrid storage technologies mainly include hybrid hard disk technology, flash cache technology, etc.

Hybrid hard drive technology is a hybrid HDD technology that directly integrates a certain size SSD (solid-state drive) inside the hard drive and controls the SSD and HDD fusion technology. HDDs generally have large capacity but slightly slower data access speeds, while SSDs generally have small capacity and fast data access speeds. The ratio of SSD to HDD is generally fixed, which cannot effectively adapt to different scenarios to achieve the best cost-effectiveness.

Flashcache technology caches popular data by adding a new caching layer between the file system (VFS) and the device driver. Flashcache typically uses an SSD as the cache medium (compared to traditional caches that use memory). By caching popular data on traditional hard drives to SSDs, it leverages the SSD's superior read performance to accelerate the system.

Hybrid hard drive technology and FlashCache technology are both based on standalone hybrid mechanisms. These standalone devices are complex to configure and lack flexibility. However, with the development of distributed storage technology, hybrid storage technology is also required at the cluster level. However, existing hybrid storage technologies are not yet feasible at the cluster level.

Summary of the Invention

The main purpose of this application is to provide a hybrid storage control method and a hybrid storage system to solve the above-mentioned problems existing in the prior art, wherein:

The present application provides a hybrid storage control method, which is applied to each HDD-type storage device and each SSD-type storage device in a storage system having one or more HDD-type storage devices and one or more SSD-type storage devices, each HDD-type storage device and each SSD-type storage device in the storage system being connected, and each HDD-type storage device and each SSD-type storage device respectively storing one or more data blocks. The method includes: periodically obtaining access information of each data block stored in the storage device; and adjusting the storage position of each data block in the storage system based on the obtained access information of each data block.

According to an embodiment of the present application, the method further includes: recording access information of each stored data block, wherein the access information includes at least: the number of times the data block has been accessed.

According to an embodiment of the present application, in this method, the storage position of each data block in the storage system is adjusted based on the access information of each data block obtained, including: migrating the data blocks stored in the HDD type storage device and accessed more than a first predetermined number of times to any SSD type storage device in the storage system for storage; or migrating the data blocks stored in the SSD type storage device and accessed less than a second predetermined number of times to any HDD type storage device in the storage system for storage.

According to an embodiment of the present application, in the method, data blocks stored in an HDD-type storage device and accessed more than a first predetermined number of times are migrated to any SSD-type storage device in the storage system for storage, further including: copying data blocks stored in the HDD-type storage device and accessed more than the first predetermined number of times to any SSD-type storage device in the storage system, and deleting the data blocks from the HDD-type storage device that originally stored the data blocks; data blocks stored in an SSD-type storage device and accessed less than a second predetermined number of times are migrated to any HDD-type storage device in the storage system for storage, further including: copying data blocks stored in the SSD-type storage device and accessed less than the second predetermined number of times to any HDD-type storage device in the storage system, and deleting the data blocks from the SSD-type storage device that originally stored the data blocks.

The present application also provides a hybrid storage system, comprising: one or more HDD-type storage devices and one or more SSD-type storage devices, each of the HDD-type storage devices comprising: an HDD-type storage unit and a control unit; each of the SSD-type storage devices comprising: an SSD-type storage unit and a control unit; the HDD-type storage unit and the SSD-type storage unit are used to store one or more data blocks; the control unit comprises: an access information acquisition module, used to periodically acquire access information of each data block stored in the HDD-type storage unit or the SSD-type storage unit; and a storage position adjustment module, used to adjust the storage position of each data block in the storage system according to the acquired access information of each data block.

According to an embodiment of the present application, in the system, the control unit further includes: a recording module for recording access information of each data block stored in the HDD-type storage unit or the SSD-type storage unit, wherein the access information includes at least: the number of times the data block is accessed.

According to an embodiment of the present application, in the system, the storage position adjustment module is further used to: migrate data blocks stored in the HDD type storage unit of the HDD type storage device and accessed more than a first predetermined number of times to the SSD type storage unit of any SSD type storage device in the storage system for storage; or further used to: migrate data blocks stored in the SSD type storage unit of the SSD type storage device and accessed less than a second predetermined number of times to the HDD type storage unit of any HDD type storage device in the storage system for storage.

According to an embodiment of the present application, in the system, the storage position adjustment module is further used to: copy a data block stored in an HDD-type storage unit of an HDD-type storage device and accessed more than a first predetermined number of times to an SSD-type storage unit of any SSD-type storage device in the storage system, and delete the data block from the HDD-type storage unit of the HDD-type storage device that originally stored the data block; or further used to: copy a data block stored in an SSD-type storage unit of an SSD-type storage device and accessed less than a second predetermined number of times to an HDD-type storage unit of any HDD-type storage device in the storage system, and delete the data block from the SSD-type storage unit of the SSD-type storage device that originally stored the data block.

Compared with the prior art, the technical solution of the present application can fully utilize the advantages of different storage devices and dynamically configure the storage location of data blocks according to the access frequency to achieve performance optimization in terms of storage capacity and data operation speed.

BRIEF DESCRIPTION OF THE DRAWINGS

The drawings described herein are used to provide a further understanding of the present application and constitute a part of the present application. The illustrative embodiments of the present application and their descriptions are used to explain the present application and do not constitute an improper limitation on the present application. In the drawings:

FIG1 is a flow chart of a method for controlling hybrid storage according to an embodiment of the present application;

FIG2 is a structural block diagram of a hybrid storage system according to an embodiment of the present application;

FIG3 a is a structural block diagram of each HDD-type storage device in a hybrid storage system according to an embodiment of the present application; and

FIG3 b is a structural block diagram of each SSD-type storage device in a hybrid storage system according to an embodiment of the present application.

DETAILED DESCRIPTION

The main idea of this application is to implement and optimize hybrid storage of data by configuring only two types of storage devices, HDD and SSD.

To make the purpose, technical solutions, and advantages of this application more clear, the technical solutions of this application will be clearly and completely described below in conjunction with the specific embodiments of this application and the corresponding drawings. Obviously, the embodiments described are only part of the embodiments of this application, not all of them. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of this application.

According to an embodiment of the present application, a hybrid storage control method is provided. This method can be applied to each HDD-type storage device and each SSD-type storage device in a storage system having one or more HDD-type storage devices and one or more SSD-type storage devices. All HDD-type storage devices and SSD-type storage devices in the storage system are connected, and each HDD-type storage device and each SSD-type storage device respectively stores one or more data blocks. A data block is a unit of data transmitted between storage devices.

It should be understood that in the storage system, the HDD-type storage device and the SSD-type storage device are each separate computing nodes. That is, each computing node contains only one type of storage medium. For example, if a computing node (storage device) contains an HDD-type storage unit, then it will not contain an SSD-type storage unit, and thus the computing node can be an HDD-type storage device. Similarly, if a computing node (storage device) contains an SSD-type storage unit, then it will not contain an HDD-type storage unit, and thus the computing node can be an SSD-type storage device.

That is, HDD-type storage devices only include HDDs (hard disk drives), and SSD-type storage devices only include SSDs (solid-state drives). The one or more HDD-type storage devices can be HDD-type storage devices of the same size, and the one or more SSD-type storage devices can be SSD-type storage devices of the same size. That is, each HDD-type storage device can be identical, and each SSD-type storage device can also be identical. Of course, it should also be understood that the one or more HDD-type storage devices can also be HDD-type storage devices of different sizes, and the one or more SSD-type storage devices can also be SSD-type storage devices of different sizes.

Referring to FIG1 , FIG1 is a flow chart of a hybrid storage control method according to an embodiment of the present application. As shown in FIG1 , for each storage device in the storage system:

In step S101, access information for each data block stored in the storage device is periodically obtained. The access information includes at least the number of times the data block has been accessed. That is, the number of times each data block stored in the HDD or SSD storage device has been accessed is obtained at predetermined intervals. The data blocks can be divided into megabyte (MB)-level blocks, for example, blocks of several megabytes, to prevent excessive metadata and conserve storage space. The access information may also include metadata about the data blocks, such as the storage address of the data blocks.

According to one embodiment of the present application, the method further includes the step of recording access information of each stored data block, that is, counting and recording the number of times each stored data block has been accessed. The access information of each data block can be recorded asynchronously, that is, the access information of two or more data blocks can be recorded simultaneously. For example, when recording the access information of one data block, if another data block is also accessed, the access information of the other data block can be counted and recorded simultaneously.

In step S102, the storage location of each data block in the storage system is adjusted according to the acquired access information of each data block.

Step S102 may include: migrating data blocks stored in an HDD-type storage device and having been accessed more than a first predetermined number of times to any SSD-type storage device in the storage system for storage; or migrating data blocks stored in an SSD-type storage device and having been accessed less than a second predetermined number of times to any HDD-type storage device in the storage system for storage.

In other words, since HDD-type devices generally have larger capacity but slower data access speeds, SSD devices generally have smaller capacity but faster data access speeds. Therefore, if the storage device is an HDD-type device, data blocks that are accessed more frequently (accessed more than a first predetermined number of times) are migrated to any SSD-type storage device in the storage system for storage, thereby improving the read speed of these more frequently accessed data blocks. Conversely, if the storage device is an SSD-type storage device, data blocks that are accessed less frequently (accessed less than a second predetermined number of times) are migrated to any HDD-type storage device in the storage system for storage. This allows the storage device in the storage system to be adjusted based on the number of times a data block is accessed, thereby maximizing storage system performance.

By adopting the above solution, the storage system only needs to be configured with two types of storage devices: one or more HDD-type storage devices and one or more SSD-type storage devices. In addition, an appropriate number of the two types of storage devices can be configured according to the needs of general scenarios. By periodically counting the number of times the data blocks stored in each storage device are accessed, the data blocks that are accessed more frequently are migrated to the SSD-type storage devices with faster reading speeds for storage, and the data blocks that are accessed less frequently are migrated to the HDD-type storage devices with larger capacity for storage, so as to fully utilize the advantages of different types of devices and achieve a performance balance between storage capacity and data operation speed.

According to an embodiment of the present application, to migrate a data block from one storage device (the original storage device) to another storage device (the target storage device), the data block can be first copied to the target storage device and then deleted from the original storage device to complete the migration.

That is, migrating a data block stored in an HDD-type storage device and accessed more than a first predetermined number of times to any SSD-type storage device in the storage system for storage means copying a data block stored in an HDD-type storage device and accessed more than the first predetermined number of times to any SSD-type storage device in the storage system, and deleting the data block from the HDD-type storage device that originally stored the data block. Migrating a data block stored in an SSD-type storage device and accessed less than a second predetermined number of times to any HDD-type storage device in the storage system for storage means copying a data block stored in an SSD-type storage device and accessed less than the second predetermined number of times to any HDD-type storage device in the storage system, and deleting the data block from the SSD-type storage device that originally stored the data block.

The present application also provides a hybrid storage system.

2 schematically shows a block diagram of a hybrid storage system according to an embodiment of the present application. According to an embodiment of the present application, the hybrid storage system 200 includes: one or more HDD-type storage devices 210 and one or more SSD-type storage devices 220.

To more clearly illustrate the structure of each storage device, Figures 3a and 3b schematically illustrate the structural block diagrams of each HDD-type storage device and SSD-type storage device, respectively, in a hybrid storage system according to an embodiment of the present application. As shown in Figure 3a, each HDD-type storage device 310 includes an HDD-type storage unit 311 and a control unit 312; as shown in Figure 3b, each SSD-type storage device 320 includes an SSD-type storage unit 321 and a control unit 322.

The structures of the HDD type storage device 310 and the SSD type storage device 320 may be as described above, and will not be described again here.

The HDD-type storage unit 311 and the SSD-type storage unit 321 may be used to store one or more data blocks.

The control unit 312 / 322 may include: an access information acquisition module 3121 / 3221 and a storage location adjustment module 3122 / 3222 .

The access information acquisition module 3121 / 3221 may be configured to periodically acquire access information of each data block stored in the HDD storage unit 311 or the SSD storage unit 321 .

The storage position adjustment module 3122 / 3222 may be configured to adjust the storage position of each data block in the storage system according to the acquired access information of each data block.

The control unit 312 / 322 may further include a recording module for recording access information of each data block stored in the HDD storage unit 311 or the SSD storage unit 321 , wherein the access information includes at least the number of times the data block has been accessed.

In the HDD storage unit 311 of the HDD storage device 310, the storage location adjustment module 3122 may be further configured to: migrate data blocks stored in the HDD storage unit 311 of the HDD storage device 310 and accessed more than a first predetermined number of times to an SSD storage unit 321 of any SSD storage device 320 in the storage system for storage. In the SSD storage unit 321 of the SSD storage device 320, the storage location adjustment module 3222 may be further configured to: migrate data blocks stored in the SSD storage unit 321 of the SSD storage device 320 and accessed less than a second predetermined number of times to an HDD storage unit 311 of any HDD storage device 310 in the storage system for storage.

In the HDD storage unit 311 of the HDD storage device 310, the storage position adjustment module 3122 may be further configured to copy a data block stored in the HDD storage unit 311 of the HDD storage device 310 and accessed more than a first predetermined number of times to an SSD storage unit 321 of any SSD storage device 320 in the storage system, and delete the data block from the HDD storage unit 311 of the HDD storage device 310 that originally stored the data block. In the SSD storage unit 321 of the SSD storage device 320, the storage position adjustment module 3222 may be further configured to copy a data block stored in the SSD storage unit 321 of the SSD storage device 320 and accessed less than a second predetermined number of times to an HDD storage unit 311 of any HDD storage device 310 in the storage system, and delete the data block from the SSD storage unit 321 of the SSD storage device 320 that originally stored the data block.

Since the functions implemented by the system of this embodiment basically correspond to the method embodiment shown in FIG1 , for details not fully described in this embodiment, please refer to the relevant descriptions in the aforementioned embodiments and will not be elaborated here.

In a typical configuration, a computing device includes one or more processors (CPUs), input/output interfaces, network interfaces, and memory.

Memory may include non-permanent storage in a computer-readable medium, random access memory (RAM) and/or non-volatile memory in the form of read-only memory (ROM) or flash RAM. Memory is an example of a computer-readable medium.

Computer-readable media include permanent and non-permanent, removable and non-removable media that can be implemented by any method or technology to store information. The information can be computer-readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disc read-only memory (CD-ROM), digital versatile disc (DVD) or other optical storage, magnetic cassettes, magnetic tape, magnetic disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device. As defined herein, computer-readable media does not include non-transitory media such as modulated data signals and carrier waves.

It should also be noted that the terms "comprises," "includes," or any other variations thereof are intended to encompass non-exclusive inclusion, such that a process, method, commodity, or apparatus that includes a list of elements includes not only those elements but also other elements not explicitly listed, or elements inherent to such process, method, commodity, or apparatus. In the absence of further limitations, an element defined by the phrase "comprises a..." does not preclude the presence of additional identical elements in the process, method, commodity, or apparatus that includes the element.

Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems, or computer program products. Therefore, the present application may take the form of an entirely hardware embodiment, an entirely software embodiment, or an embodiment combining software and hardware. Furthermore, the present application may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to magnetic disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

The foregoing is merely an embodiment of the present application and is not intended to limit the present application. Various modifications and variations are possible for those skilled in the art. Any modifications, equivalent substitutions, or improvements made within the spirit and principles of the present application shall be included within the scope of the claims of the present application.

Claims (8)

1. A hybrid storage control method, applied to a storage system having one or more HDD storage devices and one or more SSD storage devices, wherein each HDD storage device and each SSD storage device in the storage system are connected, and each HDD storage device and each SSD storage device respectively stores one or more data blocks; Each HDD storage device and each SSD storage device is a separate computing node; each HDD storage device includes an HDD storage unit and a control unit; each SSD storage device includes an SSD storage unit and a control unit. The method includes: The control unit of any storage device in the storage system periodically obtains access information of the data block of the device, which is stored in the storage unit of the device. Based on the access information of the acquired data blocks, the data blocks of this device are adjusted to other storage devices in the storage system. 2. The method according to claim 1, characterized in that it further comprises: recording access information for each stored data block, wherein the access information includes at least: the number of times the data block has been accessed. 3. The method according to claim 1, characterized in that, adjusting the data blocks of this device to other storage devices in the storage system according to the access information of each data block obtained, includes: HDD type storage device, migrate data blocks that have been accessed more than a first predetermined number of times stored in this device to any SSD type storage device in the storage system for storage; or, An SSD-type storage device may migrate data blocks that have been accessed less than a second predetermined number of times from this device to any HDD-type storage device in the storage system. 4. The method according to claim 3, characterized in that, Migrating data blocks that have been accessed more than a first predetermined number of times from an HDD storage device to any SSD storage device in the storage system further includes: copying the data blocks that have been accessed more than a first predetermined number of times from an HDD storage device to any SSD storage device in the storage system, and deleting the data blocks from the original HDD storage device where the data blocks were stored. Migrating data blocks that have been accessed less than a second predetermined number of times from an SSD storage device to any HDD storage device in the storage system further includes: copying the data blocks that have been accessed less than a second predetermined number of times from an SSD storage device to any HDD storage device in the storage system, and deleting the data blocks from the original SSD storage device where they were stored. 5. A hybrid storage system, comprising: one or more HDD-type storage devices and one or more SSD-type storage devices, wherein each HDD-type storage device and each SSD-type storage device in the storage system are connected to each other, and each HDD-type storage device and each SSD-type storage device respectively stores one or more data blocks; Each HDD storage device and each SSD storage device is a separate computing node; each HDD storage device includes an HDD storage unit and a control unit; each SSD storage device includes an SSD storage unit and a control unit. The control unit includes: The access information acquisition module is used to periodically acquire access information of data blocks of this device, wherein the data blocks of this device are stored in the storage unit of this device; The storage location adjustment module is used to adjust the data blocks of this device to other storage devices in the storage system based on the access information of each data block obtained. 6. The system according to claim 5, wherein the control unit further comprises: a recording module, used to record access information of each data block stored in the HDD-type storage unit or the SSD-type storage unit, wherein the access information includes at least: the number of times the data block has been accessed. 7. The system according to claim 5, characterized in that, The storage location adjustment module of the HDD type storage device is further used to: migrate data blocks that have been accessed more than a first predetermined number of times stored in the HDD type storage unit of the HDD type storage device to any SSD type storage unit of the SSD type storage device in the storage system for storage; The storage location adjustment module for SSD-type storage devices is further used to: migrate data blocks stored in the SSD-type storage cells of the SSD-type storage device that have been accessed less than a second predetermined number of times to any HDD-type storage cell of the HDD-type storage device in the storage system. 8. The system according to claim 7, characterized in that, The storage location adjustment module of the HDD type storage device is further used to: copy the data block stored in the HDD type storage cell of the HDD type storage device that has been accessed more than a first predetermined number of times to the SSD type storage cell of any SSD type storage device in the storage system, and delete the data block from the HDD type storage cell of the HDD type storage device that originally stored the data block; The storage location adjustment module of the SSD type storage device is further configured to: copy data blocks that have been accessed less than a second predetermined number of times from the SSD type storage cell of the SSD type storage device to the HDD type storage cell of any HDD type storage device in the storage system, and delete the data blocks from the SSD type storage cell of the SSD type storage device that originally stored the data blocks.