CN111625203A - A method, system, device and medium for hierarchical storage - Google Patents

Abstract

The invention discloses a hierarchical storage method, system, device and storage medium. The method includes the following steps: in a distributed object storage system, a hot storage pool is set based on a first storage disk, and a cold storage pool is set based on the second storage disk. ; In response to a read operation or a write operation on the file, increase the thermal value of the file, and then decrease the thermal value of the file every preset time; determine whether the thermal value of the file in the hot storage pool is less than the first threshold value, and determine whether the thermal value of the file is cold or not. whether the hotness value of the file in the storage pool is greater than the second threshold; and in response to the hotness value of the file in the hot storage pool being less than the first threshold, transferring the file to the cold storage pool, and in response to the hotness of the file in the cold storage pool If the value is greater than the second threshold, move the file to the hot storage pool. The solution proposed by the present invention improves product competitiveness by effectively utilizing the performance of different media in the distributed storage system, taking into account the performance and cost of the storage system.

Description

A method, system, device and medium for hierarchical storage

technical field

The present invention relates to the field of storage, and more particularly, to a method, system, computer device and readable medium for hierarchical storage.

Background technique

A distributed object storage system is a storage cluster composed of multiple object storage servers. Distributed storage has the advantages of high reliability, high availability, fast access, and easy expansion. At present, HDD mechanical hard disks are still widely used in storage systems. However, with the decline in the prices of new storage media, SSDs and NVME SSDs, storage systems are gradually equipped with some SSD hard disks as storage media. Fast storage media has better performance, but its capacity is often insufficient. HDD hard disks have larger capacity, but their performance is slightly insufficient. How to balance system performance and cost has become a problem to be solved in storage system design.

SUMMARY OF THE INVENTION

In view of this, the purpose of the embodiments of the present invention is to propose a hierarchical storage method, system, computer equipment and computer-readable storage medium, by setting up different storage pools based on different storage disks, effectively utilizing different storage systems in a distributed storage system. The performance of the media, taking into account the performance and cost of the storage system, improve product competitiveness.

Based on the above purpose, an aspect of the embodiments of the present invention provides a method for hierarchical storage, including the following steps: in a distributed object storage system, a hot storage pool is set based on a first storage disk, and a cold storage pool is set based on a second storage disk. pool; in response to a read operation or a write operation on the file, increase the heat value of the file, and then decrease the heat value of the file every preset time; determine whether the heat value of the file in the hot storage pool is smaller than the first threshold, and determine whether the thermal value of the file in the cold storage pool is greater than a second threshold; and in response to the thermal value of the file in the hot storage pool being less than the first threshold, transfer the file to the cold storage pool, and in response to a file in the cold storage pool having a hotness value greater than a second threshold, transferring the file to the hot storage pool.

In some embodiments, the method further includes: establishing a file heat table, recording all files with a heat value greater than a third threshold and their corresponding heat values in the file heat table, and then updating all files at preset time intervals. the heat value.

In some implementations, the method further includes: judging whether the heat value of the file in the cold storage pool is less than a first threshold; in response to the heat value of the file in the cold storage pool being less than the first threshold, performing identification, and determine whether the number of consecutive identifications reaches the fourth threshold.

In some embodiments, the method further includes: in response to the number of consecutive identifications reaching a fourth threshold, deleting the record of the file in the file heat table.

In some embodiments, the subsequently reducing the popularity value of the file every preset time includes: reducing the popularity value of each file in the file popularity table by the same value every preset same time.

In some embodiments, the method further includes: judging whether the transfer of the file from the hot storage pool to the cold storage pool or from the cold storage pool to the hot storage pool exceeds a predetermined time.

In some embodiments, the method further includes: in response to the file being transferred from the hot storage pool to the cold storage pool or from the cold storage pool to the hot storage pool for more than a predetermined time, determining whether the file only exists in the cold storage pool.

Another aspect of the embodiments of the present invention further provides a hierarchical storage system, including: a creation module configured to set a hot storage pool based on a first storage disk and set a hot storage pool based on a second storage disk in a distributed object storage system a cold storage pool; an execution module, configured to increase the heat value of the file in response to a read operation or a write operation on the file, and then decrease the heat value of the file every preset time; a judgment module, configured to judge all determining whether the heat value of the files in the hot storage pool is less than a first threshold, and judging whether the heat value of the files in the cold storage pool is greater than a second threshold; and a transfer module configured to respond to the hot storage pool The heat value of the file is less than the first threshold, the file is transferred to the cold storage pool, and in response to the heat value of the file in the cold storage pool being greater than the second threshold, the file is transferred to the hot storage pool. storage pool.

In yet another aspect of the embodiments of the present invention, there is also provided a computer device, comprising: at least one processor; and a memory, where the memory stores computer instructions that can be executed on the processor, and the instructions are executed by the processor. The processor implements the steps of the above method when executed.

In yet another aspect of the embodiments of the present invention, a computer-readable storage medium is also provided, where the computer-readable storage medium stores a computer program that implements the above method steps when executed by a processor.

The invention has the following beneficial technical effects: by setting up different storage pools based on different storage disks, the performance of different media in the distributed storage system is effectively utilized, the performance and cost of the storage system are taken into consideration, and the product competitiveness is improved.

Description of drawings

In order to explain the embodiments of the present invention or the technical solutions in the prior art more clearly, the following briefly introduces the accompanying drawings that need to be used in the description of the embodiments or the prior art. Obviously, the accompanying drawings in the following description are only These are some embodiments of the present invention. For those of ordinary skill in the art, other embodiments can also be obtained according to these drawings without creative efforts.

1 is a schematic diagram of an embodiment of a method for hierarchical storage provided by the present invention;

FIG. 2 is a schematic diagram of a hardware structure of an embodiment of a computer device with hierarchical storage provided by the present invention.

Detailed ways

In order to make the objectives, technical solutions and advantages of the present invention more clearly understood, the embodiments of the present invention will be further described in detail below with reference to the specific embodiments and the accompanying drawings.

It should be noted that all expressions using "first" and "second" in the embodiments of the present invention are for the purpose of distinguishing two entities with the same name but not the same or non-identical parameters. It can be seen that "first" and "second" It is only for the convenience of expression and should not be construed as a limitation to the embodiments of the present invention, and subsequent embodiments will not describe them one by one.

Based on the above purpose, in the first aspect of the embodiments of the present invention, an embodiment of a method for hierarchical storage is proposed. FIG. 1 shows a schematic diagram of an embodiment of a hierarchical storage method provided by the present invention. As shown in Figure 1, the embodiment of the present invention includes the following steps:

S1. In the distributed object storage system, a hot storage pool is set based on the first storage disk, and a cold storage pool is set based on the second storage disk;

S2, in response to a read operation or a write operation on the file, increase the heat value of the file, and then decrease the heat value of the file every preset time;

S3, judging whether the heat value of the files in the hot storage pool is less than the first threshold, and judging whether the heat value of the files in the cold storage pool is greater than the second threshold; and

S4. In response to the heat value of the file in the hot storage pool being less than the first threshold, transfer the file to the cold storage pool, and in response to the heat value of the file in the cold storage pool being greater than the second threshold, transfer the file to the hot storage pool .

In the distributed object storage system, a hot storage pool is set based on the first storage disk, and a cold storage pool is set based on the second storage disk. The first storage disk may be an SSD fast storage disk, and the second storage disk may be an HDD slow disk, and a hot storage pool may be set according to the fast storage disk, and a cold storage pool may be set according to the slow storage disk.

In response to a read operation or a write operation on the file, the hotness value of the file is increased, and then the hotness value of the file is decreased every preset time. For each file, there is a heat value; read and write operations on the file will increase the heat value of the file. For example, if a file is read once, the heat value increases by 100. The heat value of the file will decay periodically, for example, it can decrease by 50 every minute, until the heat value decays to zero.

It is judged whether the heat value of the files in the hot storage pool is smaller than the first threshold, and whether the heat value of the files in the cold storage pool is greater than the second threshold. Data with a heat value greater than the second threshold may be set as hot data, and data with a heat value less than the first threshold may be set as cold data. The first threshold may be, for example, 0, and the second threshold may be, for example, 300. If the heat value of the file in the hot storage pool is less than 0, it needs to be moved to the cold storage pool. If the heat value of the file in the cold storage pool is greater than 300, it can be moved to the hot storage pool.

The file is transferred to the cold storage pool in response to the hotness value of the file in the hot storage pool being less than the first threshold, and the file is transferred to the hot storage pool in response to the hotness value of the file in the cold storage pool being greater than the second threshold. When the data changes from cold to hot, we migrate the data from the cold storage pool to the hot storage pool, which can improve the read speed of the data. If the data goes from hot to cold, we move the data from the hot storage pool to the cold storage pool to make room for the hot storage pool for new data storage.

In some embodiments, the method further includes: establishing a file heat table, recording all files with a heat value greater than a third threshold and their corresponding heat values in the file heat table, and then updating all files at preset time intervals. the heat value. RGW represents an object storage gateway program, which is used to receive http requests for storing or reading data from the network, and according to the request commands, store the relevant data in the storage server storage medium, or read data from the storage medium, and return it to the user. RGW can create a file heat table to record the heat of currently active files. When each object has a write or read operation, the heat will increase and be added to the heat table. The third threshold may be equal to the first threshold, that is, files with a heat value greater than zero are recorded in the heat table. Of course, the third threshold may not be equal to the first threshold, and the third threshold may be specifically set according to the specific situation.

RGW will start a hot data scanning thread to periodically reduce the temperature of the files in the heat table. When the temperature of the file is less than the first threshold, and in the hot storage pool, the command to migrate the file to the cold storage pool will be put into Migration queue; if the file is in the cold storage pool, but the temperature is greater than the second threshold, the command to transfer the file to the hot storage pool will be placed in the migration queue. The RGW can start a group of object migration threads, read the file migration commands in the migration queue, and call the storage pool migration function to complete the mutual migration of the cold storage pool and the hot storage pool.

In some implementations, the method further includes: judging whether the heat value of the file in the cold storage pool is less than a first threshold; in response to the heat value of the file in the cold storage pool being less than the first threshold, performing identification, and determine whether the number of consecutive identifications reaches the fourth threshold. You can modify the status of files in the cold storage pool whose hotness value is continuously lower than the first threshold to be DELETING.

In some embodiments, the method further includes: in response to the number of consecutive identifications reaching a fourth threshold, deleting the record of the file in the file heat table. The fourth threshold may be, for example, three. When the file is in the DELETING state for three heat scanning cycles, the file object will be deleted from the storage pool heat table object.

In some embodiments, the subsequently reducing the popularity value of the file every preset time includes: reducing the popularity value of each file in the file popularity table by the same value every preset same time.

In some embodiments, the method further includes: judging whether the transfer of the file from the hot storage pool to the cold storage pool or from the cold storage pool to the hot storage pool exceeds a predetermined time. If the file transfer exceeds the predetermined time, it may indicate that an abnormality has occurred. Therefore, it is necessary to determine whether the transfer time exceeds the predetermined time.

In some embodiments, the method further includes: in response to the file being transferred from the hot storage pool to the cold storage pool or from the cold storage pool to the hot storage pool for more than a predetermined time, determining whether the file only exists in the cold storage pool. If the transfer of the file from the hot storage pool to the cold storage pool times out, and the file only exists in the cold storage pool, it means that the transfer has been successful and no further operations are required. If the file exists in both the hot storage pool and the cold storage pool, or only exists in the cold storage pool For hot storage pools, transfer again. If the file does not exist in either the hot storage pool or the cold storage pool, initialize the file directly.

The method of the present invention is described below by taking a file written into the hot storage pool as an example:

At the beginning, the file is in the INITIAL (initial) state. If a file is written to the hot storage pool, the state of the file will change to the HOT state, and the file header is added to the heat table; in this way, the file is included in the management of the heat state machine, and the heat The scan thread is executed periodically, and each execution cycle reduces the heat of the file in the heat table, and then executes the heat state machine function.

If there is no read and write operation, the heat value of the file in the HOT state will change to 0 after several heat scan thread execution cycles. At this time, when the state machine judges, the state machine function will change the data state to MOVING_TO_COLD state, and at the same time The command to transfer this object to the cold storage pool is placed in the migration queue.

The migration thread reads the command to migrate the object from the migration queue, executes the migration function to migrate the object to the cold storage pool, deletes the object from the hot storage pool, and then sets the object to the COLD state; if there is an error in the transfer process, it will set The state is ERR state.

When the file is in the COLD state and the heat is less than 0, the state machine will set the file state to the DELETING state and delete the file in the heat table. When the file is in the DELETING state and is placed on the storage pool heat table object, other RGWs will read the file in the DELETING state from the storage pool heat table object, and delete the file in the heat table.

When the file is in the DELETING state, after three heat scan cycles, the file object will be deleted from the storage pool heat table object, and then the file will return to the initial state.

In this embodiment of the present invention, the heat table is used to record the file name, the file's heat and state information. Every file in the current system with a temperature greater than zero will be added to the heat table and recorded, and every file whose temperature is less than zero and the file status is cold (the file is in the cold storage pool) will be removed from the heat table. Keep the file heat table small to improve search efficiency and reduce memory usage. The heat table adopts the mechanism of periodic disk placement. When the heat scan thread is executed periodically, the heat table information will be read from the storage pool at the beginning of execution. After the heat table is processed, it will be written back to the storage pool. This will prevent the heat meter from being lost due to sudden power failure of the system.

In the embodiment of the present invention, a design of concurrent migration of multiple migration threads is adopted due to the consideration of the requirements of the massive data of the storage system on the migration speed of the storage system. The heat scan thread will put the files that need to be transferred and the direction of migration obtained through the heat state machine processing into the migration queue. The migration thread group we established will periodically cycle to obtain migration commands from the migration queue, complete the migration work, and report the status of the set files.

It should be specially pointed out that each step in each embodiment of the above-mentioned hierarchical storage method can be intersected, replaced, added, and deleted. Therefore, the method for hierarchical storage by these reasonable permutations and combinations should also belong to the present invention. The protection scope of the present invention should not be limited to the embodiments.

Based on the above purpose, in a second aspect of the embodiments of the present invention, a hierarchical storage system is proposed, including: a creation module configured to set a hot storage pool based on a first storage disk in a distributed object storage system, and based on a first storage disk Two storage disks set up a cold storage pool; an execution module, configured to respond to a read operation or a write operation on the file, increase the heat value of the file, and then decrease the heat value of the file every preset time; the judgment module, configured for judging whether the heat value of the files in the hot storage pool is less than a first threshold, and judging whether the heat value of the files in the cold storage pool is greater than a second threshold; and a transfer module, configured to respond to the The file in the hot storage pool has a hotness value less than a first threshold, the file is transferred to the cold storage pool, and in response to the hotness value of the file in the cold storage pool is greater than a second threshold, the file is transferred to the thermal storage pool.

In some embodiments, it further includes: a heat table module, configured to establish a file heat table, and record all files with a heat value greater than a third threshold and the corresponding heat values of the files in the file heat table, and then The heat value is updated every preset time.

In some embodiments, a second judgment module is further included, configured to: judge whether the heat value of the files in the cold storage pool is less than a first threshold; in response to the heat value of the files in the cold storage pool being less than the first threshold A threshold is used to identify the file, and it is determined whether the number of consecutive identifications reaches a fourth threshold.

In some embodiments, the method further includes: a deletion module, configured to delete the record of the file in the file heat table in response to the number of consecutive identifications reaching a fourth threshold.

In some embodiments, the execution module is further configured to: decrease the heat value of each file in the file heat table by the same value every preset same time.

In some embodiments, it further includes: a third judging module configured to judge whether the transfer of the file from the hot storage pool to the cold storage pool or from the cold storage pool to the hot storage pool exceeds a predetermined time .

In some embodiments, it further includes: a fourth judgment module, configured to respond to the file being transferred from the hot storage pool to the cold storage pool or transferred from the cold storage pool to the hot storage pool for more than a predetermined time , and determine whether the file exists only in the cold storage pool.

Based on the above objective, in a third aspect of the embodiments of the present invention, a computer device is provided, including: at least one processor; and a memory, where the memory stores computer instructions that can be executed on the processor, and the instructions are executed by the processor to The following steps are implemented: S1. In the distributed object storage system, a hot storage pool is set based on the first storage disk, and a cold storage pool is set based on the second storage disk; heat value, and then reduce the heat value of the file every preset time; S3, determine whether the heat value of the file in the hot storage pool is less than the first threshold, and judge whether the heat value of the file in the cold storage pool is greater than the second threshold value and S4, in response to the heat value of the file in the hot storage pool being less than the first threshold value, transfer the file to the cold storage pool, and in response to the heat value of the file in the cold storage pool being greater than the second threshold value, transfer the file to the hot storage pool. storage pool.

In some embodiments, the method further includes: establishing a file heat table, recording all files with a heat value greater than a third threshold and their corresponding heat values in the file heat table, and then updating all files at preset time intervals. the heat value.

In some implementations, the method further includes: judging whether the heat value of the file in the cold storage pool is less than a first threshold; in response to the heat value of the file in the cold storage pool being less than the first threshold, performing identification, and determine whether the number of consecutive identifications reaches the fourth threshold.

In some embodiments, the method further includes: in response to the number of consecutive identifications reaching a fourth threshold, deleting the record of the file in the file heat table.

In some embodiments, the subsequently reducing the popularity value of the file every preset time includes: reducing the popularity value of each file in the file popularity table by the same value every preset same time.

In some embodiments, the method further includes: judging whether the transfer of the file from the hot storage pool to the cold storage pool or from the cold storage pool to the hot storage pool exceeds a predetermined time.

In some embodiments, the method further includes: in response to the file being transferred from the hot storage pool to the cold storage pool or from the cold storage pool to the hot storage pool for more than a predetermined time, determining whether the file only exists in the cold storage pool.

As shown in FIG. 2 , it is a schematic diagram of the hardware structure of an embodiment of the above-mentioned hierarchical storage computer device provided by the present invention.

Taking the device shown in FIG. 2 as an example, the device includes a processor 301 and a memory 302 , and may further include an input device 303 and an output device 304 .

The processor 301 , the memory 302 , the input device 303 and the output device 304 may be connected by a bus or in other ways, and the connection by a bus is taken as an example in FIG. 2 .

As a non-volatile computer-readable storage medium, the memory 302 can be used to store non-volatile software programs, non-volatile computer-executable programs and modules, such as programs corresponding to the hierarchical storage method in the embodiments of the present application Directive/Module. The processor 301 executes various functional applications and data processing of the server by running the non-volatile software programs, instructions and modules stored in the memory 302, that is, implementing the hierarchical storage method of the above method embodiments.

The memory 302 may include a storage program area and a storage data area, wherein the storage program area may store an operating system, an application program required for at least one function; the storage data area may store data created according to the use of a hierarchical storage method, and the like. Additionally, memory 302 may include high speed random access memory, and may also include nonvolatile memory, such as at least one magnetic disk storage device, flash memory device, or other nonvolatile solid state storage device. In some embodiments, memory 302 may optionally include memory located remotely from processor 301, which may be connected to local modules via a network. Examples of such networks include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

The input device 303 can receive input information such as user name and password. The output device 304 may include a display device such as a display screen.

The program instructions/modules corresponding to one or more hierarchical storage methods are stored in the memory 302, and when executed by the processor 301, the hierarchical storage method in any of the foregoing method embodiments is executed.

Any embodiment of the computer device that executes the above-mentioned method for hierarchical storage can achieve the same or similar effects as any of the foregoing method embodiments corresponding to it.

The present invention also provides a computer-readable storage medium, where the computer-readable storage medium stores a computer program that executes the above method when executed by a processor.

Finally, it should be noted that those of ordinary skill in the art can understand that all or part of the process in the method of the above-mentioned embodiments can be implemented by instructing the relevant hardware through a computer program, and the program of the hierarchical storage method can be stored in a computer readable In the storage medium, when the program is executed, it may include the processes of the foregoing method embodiments. Wherein, the storage medium of the program may be a magnetic disk, an optical disk, a read only memory (ROM), or a random access memory (RAM) or the like. The above computer program embodiments can achieve the same or similar effects as any of the foregoing method embodiments corresponding thereto.

In addition, the methods disclosed according to the embodiments of the present invention may also be implemented as a computer program executed by a processor, and the computer program may be stored in a computer-readable storage medium. When the computer program is executed by the processor, the above-mentioned functions defined in the methods disclosed in the embodiments of the present invention are executed.

In addition, the above-mentioned method steps and system units can also be implemented by using a controller and a computer-readable storage medium for storing a computer program that enables the controller to implement the functions of the above-mentioned steps or units.

In addition, it should be understood that computer-readable storage media (eg, memory) herein can be volatile memory or non-volatile memory, or can include both volatile and non-volatile memory. By way of example and not limitation, nonvolatile memory may include read only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), or flash memory memory. Volatile memory can include random access memory (RAM), which can act as external cache memory. By way of example and not limitation, RAM is available in various forms such as Synchronous RAM (DRAM), Dynamic RAM (DRAM), Synchronous DRAM (SDRAM), Double Data Rate SDRAM (DDRSDRAM), Enhanced SDRAM (ESDRAM), Synchronous Link DRAM (SLDRAM), and Direct Rambus RAM (DRRAM). The storage devices of the disclosed aspects are intended to include, but not be limited to, these and other suitable types of memory.

Those skilled in the art will also appreciate that the various exemplary logical blocks, modules, circuits, and algorithm steps described in connection with the disclosure herein may be implemented as electronic hardware, computer software, or combinations of both. To clearly illustrate this interchangeability of hardware and software, various illustrative components, blocks, modules, circuits, and steps have been described generally in terms of their functionality. Whether such functionality is implemented as software or hardware depends on the specific application and design constraints imposed on the overall system. Those skilled in the art may implement the functions in various ways for each specific application, but such implementation decisions should not be interpreted as causing a departure from the scope of the disclosed embodiments of the present invention.

The various exemplary logical blocks, modules, and circuits described in connection with the disclosure herein can be implemented or executed using the following components designed to perform the functions herein: general purpose processors, digital signal processors (DSPs), application specific integrated circuits (ASIC), Field Programmable Gate Array (FPGA) or other programmable logic device, discrete gate or transistor logic, discrete hardware components, or any combination of these components. A general-purpose processor may be a microprocessor, but in the alternative, the processor may be any conventional processor, controller, microcontroller, or state machine. A processor may also be implemented as a combination of computing devices, eg, a combination of a DSP and a microprocessor, multiple microprocessors, one or more microprocessors in combination with a DSP, and/or any other such configuration.

The steps of a method or algorithm described in connection with the disclosures herein may be embodied directly in hardware, in a software module executed by a processor, or in a combination of the two. A software module may reside in RAM memory, flash memory, ROM memory, EPROM memory, EEPROM memory, registers, hard disk, removable disk, CD-ROM, or any other form of storage medium known in the art. An exemplary storage medium is coupled to the processor, such that the processor can read information from, and write information to, the storage medium. In an alternative, the storage medium may be integrated with the processor. The processor and storage medium may reside in an ASIC. The ASIC may reside in the user terminal. In an alternative, the processor and storage medium may reside in the user terminal as discrete components.

In one or more exemplary designs, functions may be implemented in hardware, software, firmware, or any combination thereof. If implemented in software, the functions may be stored on or transmitted over as one or more instructions or code on a computer-readable medium. Computer-readable media includes both computer storage media and communication media including any medium that facilitates transfer of a computer program from one place to another. A storage medium can be any available medium that can be accessed by a general purpose or special purpose computer. By way of example and not limitation, the computer-readable medium may include RAM, ROM, EEPROM, CD-ROM or other optical disk storage devices, magnetic disk storage devices or other magnetic storage devices, or may be used to carry or store instructions in the form of or data structures and any other medium that can be accessed by a general purpose or special purpose computer or a general purpose or special purpose processor. Also, any connection is properly termed a computer-readable medium. For example, if coaxial cable, fiber optic cable, twisted pair, digital subscriber line (DSL), or wireless technologies such as infrared, radio, and microwave are used to send software from a website, server, or other remote source, the above coaxial cable Cable, fiber optic cable, twisted pair, DSL or wireless technologies such as infrared, radio and microwave are all included in the definition of medium. As used herein, magnetic disks and optical disks include compact disks (CDs), laser disks, optical disks, digital versatile disks (DVDs), floppy disks, blu-ray disks, where disks usually reproduce data magnetically, while optical disks reproduce data optically with lasers . Combinations of the above should also be included within the scope of computer-readable media.

The above are exemplary embodiments of the present disclosure, but it should be noted that various changes and modifications may be made without departing from the scope of the disclosure of the embodiments of the present invention as defined in the claims. The functions, steps and/or actions of the method claims in accordance with the disclosed embodiments described herein need not be performed in any particular order. Furthermore, although elements disclosed in the embodiments of the present invention may be described or claimed in the singular, unless explicitly limited to the singular, the plural may also be construed.

It should be understood that, as used herein, the singular form "a" is intended to include the plural form as well, unless the context clearly supports an exception. It will also be understood that "and/or" as used herein is meant to include any and all possible combinations of one or more of the associated listed items.

The above-mentioned embodiments of the present invention disclose the serial numbers of the embodiments only for description, and do not represent the advantages and disadvantages of the embodiments.

Those of ordinary skill in the art can understand that all or part of the steps of implementing the above embodiments can be completed by hardware, or can be completed by instructing relevant hardware through a program, and the program can be stored in a computer-readable storage medium. The storage medium can be a read-only memory, a magnetic disk or an optical disk, and the like.

Those of ordinary skill in the art should understand that the discussion of any of the above embodiments is only exemplary, and is not intended to imply that the scope of the disclosure (including the claims) of the embodiments of the present invention is limited to these examples; under the idea of the embodiments of the present invention , the technical features in the above embodiments or different embodiments can also be combined, and there are many other changes in different aspects of the above embodiments of the present invention, which are not provided in detail for the sake of brevity. Therefore, any omission, modification, equivalent replacement, improvement, etc. made within the spirit and principle of the embodiments of the present invention should be included within the protection scope of the embodiments of the present invention.

Claims (9)

1. A method of hierarchical storage, comprising the steps of:

setting a hot storage pool based on a first storage disk and setting a cold storage pool based on a second storage disk in the distributed object storage system;

in response to the read operation or the write operation of the file, increasing the heat value of the file, and then reducing the heat value of the file every preset time;

judging whether the heat value of the file in the hot storage pool is smaller than a first threshold value or not, and judging whether the heat value of the file in the cold storage pool is larger than a second threshold value or not;

transferring the file to the cold storage pool in response to the heat value of the file in the hot storage pool being less than a first threshold, and transferring the file to the hot storage pool in response to the heat value of the file in the cold storage pool being greater than a second threshold; and

establishing a file heat table, recording all files with heat values larger than a third threshold value and corresponding heat values of the files in the file heat table, and then updating the heat values at preset time intervals.

2. The method of claim 1, further comprising:

judging whether the heat value of the file in the cold storage pool is smaller than a first threshold value or not;

and identifying the file in the cold storage pool in response to the fact that the heat value of the file in the cold storage pool is smaller than a first threshold value, and judging whether the continuous identification times reach a fourth threshold value.

3. The method of claim 2, further comprising:

and in response to the number of consecutive identifications reaching a fourth threshold, deleting the record of the file in the file heat table.

4. The method of claim 1, wherein the subsequently reducing the heat value of the file at preset intervals comprises:

and reducing the heat value of each file in the file heat table by the same value at preset same time intervals.

5. The method of claim 1, further comprising:

a determination is made whether a file is transferred from the hot storage pool to the cold storage pool or from the cold storage pool to the hot storage pool for more than a predetermined time.

6. The method of claim 5, further comprising:

in response to a file being transferred from the hot storage pool to the cold storage pool or from the cold storage pool to the hot storage pool for more than a predetermined time, determining whether the file exists only in the cold storage pool.

7. A system for hierarchical storage, comprising:

a creation module configured to set a hot storage pool based on a first storage disk and set a cold storage pool based on a second storage disk in the distributed object storage system;

the execution module is configured to respond to read operation or write operation on a file, increase the heat value of the file and reduce the heat value of the file every preset time;

the judging module is configured to judge whether the heat value of the file in the hot storage pool is smaller than a first threshold value and judge whether the heat value of the file in the cold storage pool is larger than a second threshold value;

a transfer module configured to transfer a file in the hot storage pool to the cold storage pool in response to a heat value of the file being less than a first threshold, and to transfer the file to the hot storage pool in response to a heat value of the file in the cold storage pool being greater than a second threshold; and

and the heat meter module is configured for establishing a file heat meter, recording all files with heat values larger than a third threshold value and corresponding heat values of the files in the file heat meter, and then updating the heat values at preset time intervals.

8. A computer device, comprising:

at least one processor; and

a memory storing computer instructions executable on the processor, the instructions when executed by the processor implementing the steps of the method of any one of claims 1 to 6.

9. A computer-readable storage medium, in which a computer program is stored which, when being executed by a processor, carries out the steps of the method according to any one of claims 1 to 6.

Images