CN107122126B - Data migration method, device and system - Google Patents

Abstract

A method for migrating data in a storage system, the storage system includes a first storage device and a second storage device, the method includes the following steps: acquiring the number of reads and writes per unit time of the second storage device; The number of times of reading and writing is obtained, and the intensity ratio of the second storage device is obtained; the corresponding table of intensity ratio and capacity ratio is inquired, and the capacity ratio corresponding to the intensity ratio is obtained; according to the obtained capacity ratio, the need to migrate from the first storage device to the The number of data blocks of the second storage device; and the corresponding data blocks are migrated from the first storage device to the second storage device according to the number. With this method, the number of data blocks to be migrated from the first storage device to the second storage device can be accurately determined, and while migrating as much data as possible to the second storage device, the performance of the second storage device can be guaranteed not to be affected. impact, improving the performance and efficiency of the storage system.

Description

Data migration method, device and system

technical field

The present application relates to the field of communication technologies, and in particular, to a data migration method, device and system.

Background technique

With the development of the network, more and more data is generated. Different data have different use values. Some data will be accessed frequently, some data will not be accessed for a long time, and some data will be frequently accessed at certain times.

The use value of data has its own life cycle and can be followed by rules. Generally, various types of metadata (for example, some operating system data of virtual machines) are of high use value, are frequently accessed, and require high response speed. The newly generated data is usually accessed more frequently. Over time, the frequency of use of newly generated data will decrease and it will no longer be accessed frequently.

Therefore, the automatic tiered storage technology is produced and is widely used in all walks of life. The automatic hierarchical storage technology stores data on storage devices with different performances according to the characteristics of data access frequency, importance, retention time, etc.; In storage devices, frequently accessed data is migrated to storage devices with high performance.

The data migration strategy includes migrating data that is accessed infrequently to storage devices with lower performance, and data that is frequently accessed to storage devices with high performance. When migrating data, it is preferred to migrate frequently accessed data to high-performance storage devices. The amount of data to be migrated is often determined based on the current available capacity of high-performance storage devices, and is generally performed through IO requests to access data. Monitor, migrate the data whose number of accessed IO requests reaches a certain amount to high-performance storage devices.

During the invention process, the inventor found that the data migration strategy of the current automatic hierarchical storage technology is relatively simple, and the overall performance of the migrated storage system cannot be predicted, which may lead to the problem of overloading of high-performance storage devices.

SUMMARY OF THE INVENTION

Embodiments of the present application provide a method, apparatus, and system for migrating data, which can accurately determine the amount of data that needs to be migrated from a first storage device to a second storage device, and then migrate as much data as possible to the second storage device. At the same time, it can also ensure that the performance of the second storage device is not affected, thereby improving the performance and efficiency of the storage system.

The embodiments of the present application provide the following technical solutions:

A first aspect provides a method for migrating data in a storage system, including a first storage device and a second storage device in the storage system, the method comprising: acquiring the number of reads and writes per unit time of the second storage device; according to the second storage device The number of reads and writes per unit time, to obtain the intensity ratio of the second storage device; query the correspondence table between the intensity ratio and the capacity ratio to obtain the capacity ratio corresponding to the intensity ratio; The number of data blocks migrated from a storage device to the second storage device; and the corresponding data blocks are migrated from the first storage device to the second storage device according to the number.

After the intensity ratio of the second storage device is acquired, the number of data blocks that need to be migrated to the second storage device is obtained according to the correspondence table between the intensity ratio and the capacity ratio. In this way, the specific value of the migrated data can be determined according to the performance of the second storage device, which can ensure that the performance of the second storage device is not affected after the data is migrated, and the performance of the entire storage system is guaranteed.

In a possible design, the intensity ratio and capacity ratio correspondence table is obtained by pre-monitoring and analysis.

In this way, after obtaining the intensity ratio of the second storage device, the corresponding table of the intensity ratio and the capacity ratio can be directly queried to quickly obtain the number of data blocks that need to be migrated from the first storage device to the second storage device.

In a possible design, the intensity ratio and capacity ratio correspondence table is queried, and the capacity ratio corresponding to the intensity ratio is obtained through a fuzzy matching rule.

It is impossible to record all possible data in the correspondence table of intensity ratio and capacity ratio, so when the specific value of the calculated intensity ratio cannot be found in the correspondence table of intensity ratio and capacity ratio, according to the fuzzy matching rule, The corresponding capacity ratio is determined based on the value of the intensity ratio that is one level higher than the calculated intensity ratio in the corresponding table of intensity ratio and capacity ratio, so as to improve the search efficiency.

In a possible design, the number of reads and writes per unit time of each data block in the storage system is acquired, and the acquired numbers of reads and writes per unit time of each data block are in order from high to low arrangement. Obtaining, according to the capacity ratio, the number of data blocks that need to be migrated from the first storage device to the second storage device specifically includes: according to the capacity ratio and the obtained data block per unit time The number of reads and writes determines the data blocks corresponding to the capacity ratio; and confirms the number of data blocks corresponding to the capacity ratio in the first storage device.

In order to determine to migrate data with more access times from the first storage device to the second storage device, the obtained read and write times per unit time of each data block in the storage system are arranged in order from high to low, so that According to the determined number of data blocks that need to be migrated, the data blocks with higher number of reads and writes per unit time can be migrated from the first storage device to the second storage device with higher performance in order to respond to IO requests faster. .

In a possible design, the number of times of reading and writing per unit time of the first storage device is obtained;

The strength ratio of the second storage device is the sum of the number of reads and writes per unit time of the second storage device to the sum of the number of reads and writes per unit time of the first storage device and the number of reads and writes per unit time of the second storage device. proportion.

In a possible design, the number of times of reading and writing per unit time of the first storage device is the product of the number of times of reading and writing per disk in the first storage device per unit time and the number of disks in the first storage device Divide by the first conversion factor, the first conversion factor is related to the ratio of read requests to write requests of the first storage device and the RAID level of the first storage device; The number of writes is the product of the number of times of reading and writing a single disk in the second storage device per unit time and the number of disks in the second storage device, and then divided by a second conversion factor, the second conversion factor and the The ratio of the read request to the write request of the second storage device is related to the RAID level of the second storage device.

In a possible design, the number of reads and writes per disk per unit time in the first storage device is related to the load characteristics and response time of the first storage device; the second storage device in the single disk per unit time The number of times of reading and writing in the second storage device is related to the load characteristics and the response time of the second storage device.

By setting and obtaining the specific values of each parameter, the number of data blocks that need to be migrated from the first storage device to the second storage device can be accurately obtained, and as much hot data can be migrated to the high-performance second storage device as much as possible. At the same time, it can ensure the performance of the storage device and improve the efficiency and performance of the storage system.

In a second aspect, a storage system for implementing data migration is provided. The storage system includes a first storage device, a second storage device, and a processor, and the processor is used for each step in the above method. For the implementation details of each step and the corresponding benefits, please refer to the relevant description in the first aspect.

In a third aspect, a storage system for implementing data migration is provided. The storage system includes a first storage device, a second storage device, and a processor, and the processor includes a data acquisition and analysis module and a data migration module. The data collection and analysis module is used to obtain the read and write times of the second storage device per unit time; obtain the intensity ratio of the second storage device according to the read and write times of the second storage device per unit time; query the ratio of intensity to capacity Correspondence table, obtain the capacity ratio corresponding to the strength ratio; obtain the number of data blocks that need to be migrated from the first storage device to the second storage device according to the capacity ratio; obtain the data blocks that need to be migrated The number sent to the data migration module. The data migration module is used for migrating the obtained number of data blocks from the first storage device to the second storage device.

The data acquisition and analysis module acquires the intensity ratio of the second storage device, and obtains the number of data blocks that need to be migrated to the second storage device according to the correspondence table between the intensity ratio and the capacity ratio. In this way, the specific value of the migrated data can be determined according to the performance of the second storage device, which can ensure that the performance of the second storage device is not affected after the data is migrated, and the performance of the entire storage system is guaranteed.

In a possible design, the data collection and analysis module is further used to: collect and analyze the IO requests of the application to the storage system in advance, and obtain a correspondence table of the intensity ratio and the capacity ratio.

In this way, after obtaining the intensity ratio of the second storage device, the data collection and analysis module can directly query the intensity ratio and capacity ratio correspondence table to quickly obtain the number of data blocks that need to be migrated from the first storage device to the second storage device.

In a possible design, the data acquisition and analysis module is used to query the intensity ratio and capacity ratio correspondence table to obtain the capacity ratio corresponding to the intensity ratio. Specifically, the data acquisition and analysis module is used to query the intensity ratio and capacity ratio correspondence table, The capacity ratio corresponding to the intensity ratio is obtained through fuzzy matching rules.

It is impossible to record all possible data in the correspondence table of intensity ratio and capacity ratio, so when the specific value of the calculated intensity ratio cannot be found in the correspondence table of intensity ratio and capacity ratio, according to the fuzzy matching rule, The corresponding capacity ratio is determined based on the value of the intensity ratio that is one level higher than the calculated intensity ratio in the corresponding table of intensity ratio and capacity ratio, so as to improve the search efficiency.

In a possible design, the data acquisition and analysis module is further configured to obtain the number of reads and writes per unit time of each data block in the storage system, and to obtain the read and write times per unit time of each data block obtained. The times are arranged in order from high to low; the data acquisition and analysis module obtains the number of data blocks that need to be migrated from the first storage device to the second storage device according to the capacity ratio. Specifically: The capacity ratio derives in order the number of data blocks that need to be migrated from the first storage device to the second storage device.

In order to determine to migrate the data with more access times from the first storage device to the second storage device, the data acquisition and analysis module will obtain the read and write times per unit time of each data block in the storage system from high to low by pressing In this way, according to the determined number of data blocks that need to be migrated, the data blocks with higher number of reads and writes per unit time can be migrated from the first storage device to the second storage device with higher performance in order, which is faster The response to the IO request.

In a possible design, the data acquisition and analysis module is further configured to acquire the number of times of reading and writing in a unit time of the first storage device. The strength ratio of the second storage device is the ratio of the number of reads and writes per unit time of the second storage device to the sum of the number of reads and writes per unit time of the first storage device and the number of reads and writes per unit time of the second storage device.

In a possible design, the number of reads and writes per unit time of the first storage device is the product of the number of reads and writes per disk per unit time in the first storage device and the number of disks in the first storage device divided by the number of A conversion factor, the first conversion factor is related to the ratio of read requests and write requests of the first storage device and the RAID level of the first storage device; The product of the number of reads and writes per disk unit time and the number of disks in the second storage device is divided by the second conversion factor, the ratio of the second conversion factor to the read request and write request of the second storage device, and the second storage device related to the RAID level.

By setting and obtaining the specific values of each parameter, the number of data blocks that need to be migrated from the first storage device to the second storage device can be accurately obtained, and as much hot data can be migrated to the high-performance second storage device as much as possible. At the same time, it can ensure the performance of the storage device and improve the efficiency and performance of the storage system.

Description of drawings

In order to illustrate the technical solutions in the embodiments of the present invention more clearly, the following briefly introduces the accompanying drawings that need to be used in the description of the embodiments.

1 is a schematic structural diagram of a storage system in an embodiment of the present invention;

2 is a schematic flowchart of a method for migrating data according to an embodiment of the present invention;

FIG. 3 is a schematic structural diagram of another storage system according to an embodiment of the present invention.

Detailed ways

An embodiment of the present invention provides a method for migrating data in a storage system. The size of the data to be migrated is calculated by collecting and counting load characteristics of IO requests accessing the storage system and estimating the performance of the storage device in the storage system. In order to achieve the purpose of migrating data on demand and give full play to the performance of the storage system

In the embodiment of the present invention, the data whose access frequency reaches a predetermined value is called hot data, and the data whose access frequency is lower than the predetermined value is called cold data.

A storage system applicable to the embodiment of the present invention is shown in FIG. 1 . The storage system 1 includes a storage medium 101 , a processor 103 and a cache 105 . There may be various storage media 101 in the storage system 1, and they are divided into different performance layers according to the performance of the storage media. For example, the storage medium in the storage system consists of SSD disks and common hard disks, SSD disks form high-performance tier storage devices, and common hard disks form common performance tier storage devices. In the embodiments of the present invention, the storage system includes a high-performance layer and a common performance layer as an example for description. For convenience of description, the common performance layer in the storage system is referred to as the first storage device, and the high-performance layer in the storage system is referred to as the first storage device. The tier is called the second storage device. In addition, in the embodiment of the present invention, the hot data is migrated to the high-performance tier as an example for description, and the same method can also be used for the migration of hot data from the high-performance tier to the common performance tier when the hot data becomes cold. When other performance tiers are added to the storage system, the same method can be used for data migration between different performance tiers. The processor 103 is used for executing the following method of migrating data, and the cache 105 is used for storing required information.

The process of the method for migrating data in the storage system provided by the embodiment of the present invention is shown in FIG. 2 , which is completed by the processor in the storage system. The embodiments of this method mainly describe the implementation manner of migrating data from the common performance layer to the high performance layer, and the detailed description is as follows. As mentioned above, the common performance layer in the storage system is called the first storage device, and the high-performance layer in the storage system is called the second storage device. In addition, the IO request in the embodiment of the present invention takes the IO request for accessing the storage medium as the monitoring and analysis object.

Step 201: Acquire the number of times of reading and writing within a unit time of the second storage device.

The processor acquires the number of reads and writes of the second storage device per unit time, and the unit time may be determined by the user according to the service type, which is not limited in this embodiment of the present invention. When the unit time is 1 second, what is obtained is the number of times of reading and writing per second (Input/Output per second, IOPS) of the second storage device.

In this embodiment of the present invention, each disk in the same performance tier storage device is of the same type. An embodiment of the present invention provides a method for acquiring the number of times of reading and writing in a unit time of a second storage device. For example, the number of times of reading and writing of the second storage device per unit time=(the number of times of reading and writing of a single disk per unit time * the number of disks)/RAID conversion factor. The number of disks is the number of disks in the second storage device. For random write IO, the storage device uses RAID to calculate the parity data, so additional IO needs to be generated. The RAID conversion factor is related to the ratio of read and write IO and the write penalty factor. The write penalty factor refers to the write IO amplification factor of the disk. It is related to the RAID level used. For the corresponding relationship between the value of the write penalty coefficient and the RAID level, refer to Table 1. Specifically, RAID conversion factor=read request ratio+write request ratio*write penalty factor.

RAID level write penalty factor 0 1 1 2 5 4 6 6 10 2

Table 1: Write penalty coefficients corresponding to RAID levels

The number of reads and writes of a single disk per unit time is related to the load characteristics and response time of the second storage device, and data can be collected in advance and obtained by statistics, as shown in Table 2. The load characteristic of the second storage device here is the load characteristic of the IO request for accessing the second storage device. In this way, when the number of reads and writes per unit time of the second storage device needs to be obtained, the IOPS of a single disk in the second storage device can be found directly according to the load characteristics of the IO request accessing the second storage device and the value of the response time, and then the IOPS of a single disk in the second storage device can be found through The above method calculates and obtains the number of times of reading and writing within the time of the second storage device.

Table 2: Lookup table of read and write times per disk per unit time in the second storage device

Optionally, the number of reads and writes of a single disk per unit time may also be statistically obtained according to the load characteristics and response duration of the IO request for accessing the second storage device, and it is not necessary to obtain the values in Table 2 in advance.

Step 203: Acquire a strength ratio of the second storage device according to the number of accesses of the second storage device per unit time.

The strength ratio of the second storage device is the ratio of the access times per unit time of the second storage device to the access times per unit time of the storage system.

In the embodiment of the present invention, the storage system includes storage devices with two performance layers: a first storage device and a second storage device, and the strength ratio of the second storage device=the number of accesses per unit time of the second storage device/(the second storage device The number of accesses per unit time of the storage device + the number of accesses per unit time of the first storage device), that is, the strength ratio of the second storage device is the number of reads and writes per unit time of the second storage device in the unit time of the first storage device. The proportion of the number of reads and writes within the second storage device to the sum of the number of reads and writes per unit time of the second storage device. The access times of the second storage device per unit time has been described in step 201, and will not be further described here.

The method for acquiring the number of accesses per unit time of the first storage device is the same as the method for acquiring the number of accesses per unit time of the second storage device, and will not be further described. Similarly, the number of reads and writes of a single disk per unit time in the first storage device may also be pre-collected data and obtained by statistics, as shown in Table 3.

Table 3: Lookup table of read and write times per disk per unit time in the first storage device

It should be noted that the contents in Tables 2 and 3 can be combined into one table for storage, that is, the number of reads and writes per disk per unit time in each performance tier in the storage system can be reflected in one table, as shown in Table 4. . When there are other storage devices with different performance tiers in the storage system, the number of reads and writes per unit time of a single disk in other performance tiers can be reflected in one table, or can be reflected in different tables respectively. In the implementation of the present invention The example is not limited.

Table 4: Look-up table for read and write times per disk per unit time in the storage system

Step 205 : query the correspondence table of the intensity ratio and the capacity ratio to obtain the capacity ratio corresponding to the intensity ratio.

The correspondence table between the intensity ratio and the capacity ratio is obtained by monitoring and analyzing the business data in advance. The service data in the storage system (including the first storage device and the second storage device) is divided into multiple data blocks, and the number of reads and writes of these data blocks per unit time is obtained, that is, the access strength of the data blocks. Business data can be divided into multiple data blocks according to the migration granularity, or divided into multiple data blocks according to the specified size. Count the read and write times of accessing these data blocks within the monitoring time, and then calculate the IOPS of the read and write times of these data blocks per unit time, that is, the access strength. Sort the data blocks in the storage system according to the access intensity from high to low, and then divide the data blocks into several groups according to a certain capacity ratio. For example, as shown in Table 3, the data blocks are grouped with the capacity ratio of 2.5% as the dimension, that is, data1~dataA form a group with a capacity ratio of 2.5%, that is, data1~data2A form a group with a capacity ratio of 5%... and so on, As shown in Table 5.

Table 5: Statistics of hotspot data blocks

Calculate the number of reads and writes of each data block per unit time, and then calculate the sum of the number of reads and writes per unit time of each data block corresponding to the capacity ratio and the number of reads and writes per unit time of all data blocks in the storage system Compare the sum of , and obtain the intensity ratio corresponding to the capacity ratio, that is, the correspondence table between the intensity ratio and the capacity ratio, as shown in Table 6. The data in Table 6 is an exemplary description, and the specific data will have different values according to different services. The value of the capacity ratio can also be set as required.

capacity ratio Intensity ratio 0.0% 0.0% 2.5% 35.0% 5.0% 50.0% 7.5% 60.0% 10.0% 65.0% 12.5% 67.0% 15.0% 69.0% 20.0% 70.0% 25.0% 71.5% 30.0% 72.0% 100.0% 100.0%

Table 6: Corresponding table of intensity ratio and capacity ratio of a typical load

As shown in Table 6, when the capacity ratio of data blocks in the storage system is 10%, the corresponding intensity ratio is 65%, that is, the P intensity ratio [10%]=65%, indicating that the data blocks occupy 10% of the total data volume The number of I/O accesses in the storage system accounts for 65% of the total data block I/O accesses in the storage system. Conversely, when the intensity ratio of the data block is 70%, the corresponding capacity ratio is 20%, and the P hotspot capacity ratio [70%]=20%, which means that the number of I/O accesses accounts for all the data block I/O in the storage system. Data blocks with 70% of the access quantity account for 20% of all data blocks in the storage system. From the data rules in the table, we can see that the intensity ratio corresponding to 10% data blocks has reached 65%; while the intensity ratio corresponding to more data blocks does not change much, indicating that the hot data in this scene is very concentrated and suitable for using classification technology to improve performance.

In order to improve the accuracy of the data, optionally, a data block access situation of a business model can be used as a statistical basis to obtain a corresponding table of the intensity ratio and the capacity ratio.

After the intensity ratio of the second storage device is obtained through step 203, the capacity ratio corresponding to the intensity ratio of the second storage device can be obtained by querying the corresponding table with the capacity ratio. The query rule may adopt a fuzzy matching rule, that is, query the correspondence table of the intensity ratio and the capacity ratio, and obtain the capacity ratio corresponding to the intensity ratio through the fuzzy matching rule. If the intensity ratio to be queried cannot be exactly matched in the intensity ratio and capacity ratio correspondence table, the corresponding capacity ratio is searched according to the obtained intensity ratio of the second storage device one level higher. If the data block that needs to be migrated determined according to the capacity ratio obtained by the query meets the conditions, it is calculated according to the capacity ratio. If the data block to be migrated determined according to the capacity ratio obtained by the query does not satisfy the condition, the corresponding capacity ratio is searched according to the obtained strength ratio of the second storage device one level lower. The specific conditions will be described below, and will not be described in detail here.

Step 207: Obtain the number of data blocks that need to be migrated from the first storage device to the second storage device according to the capacity ratio.

According to the capacity ratio obtained in step 205, the number of data blocks corresponding to the capacity ratio is obtained. Acquire the number of times of reading and writing in the unit time of each data block in the storage system, and arrange the times of reading and writing in the unit time of each data block obtained in order from high to low; according to the obtained and described For the number of data blocks corresponding to the capacity ratio, select the corresponding number of data blocks according to the access intensity from high to low, and determine the number of data blocks that need to be migrated from the first storage device to the second storage device.

If some of the selected corresponding number of data blocks are already in the second storage device, determine the number of data blocks in the first storage device among these data blocks, and determine that this needs to be migrated from the first storage device to the second storage device The number of data blocks for the storage device.

In the following, the data block has not been moved into the second storage device as an example. If there is a data block in the second storage device, the calculated value can be subtracted from the value already in the second storage device. , and no longer explain them one by one. Migration data volume=P capacity ratio [second storage device IOPS/(second storage device IOPS+first storage device IOPS)]*total data volume. If the calculated data volume of the data blocks to be migrated is less than the total capacity of the second storage device, the number of data blocks to be migrated is the calculated number of data blocks. If the calculated data volume of the data blocks to be migrated is greater than the total capacity of the second storage device, the number of data blocks to be migrated is subject to the total capacity of the second storage device.

Step 209: Migrate the corresponding data blocks from the first storage device to the second storage device according to the quantity. According to the number of data blocks that need to be migrated from the first storage device to the second storage device calculated in step 207, the corresponding number of data blocks are migrated from the first storage device to the second storage device according to the access intensity from high to low middle.

The following uses a specific example to further illustrate the method of the embodiment of the present invention.

Through the monitoring and analysis of typical business data, the values of the following parameters are obtained, as shown in Table 7:

Table 7: Service parameter value table

Second storage device IOPS=3500*10/(0.7+0.3*4), first storage device IOPS=180*100/(0.7+0.3*4), then the second storage device strength ratio=second storage device IOPS/ (IOPS of the second storage device + IOPS of the first storage device)=66%, look up the corresponding table of intensity ratio and capacity ratio shown in Table 6 according to the fuzzy matching rule, and take the intensity ratio 67% higher than the 66% intensity ratio as the criterion, It is determined that the corresponding capacity ratio is 12.5%, then the amount of migrated data = the amount of data in the storage system * 12.5% = 3.75TB; because the high-performance tier capacity (6TB)> 3.75TB, the final migration data volume is 3.75TB, according to the access Migrate 3.75TB of data from the first storage device to the second storage device in intensity from high to low. Data accounting for 66% of the access intensity of the storage system is migrated to the second storage device. Since the second storage device is a high-performance storage device, it can respond to user requests in time. In addition, the amount of migrated data does not exceed the capacity of the second storage device, and the performance of the second storage device will not be affected, thereby ensuring the overall performance of the storage system.

An embodiment of the present invention further provides a storage system 3 that can implement the data migration method as described above. The structure of the storage system 3 is shown in FIG. 3 . The storage system 3 includes a first storage device 321 and a second storage device 323, and the performance of the first storage device 321 and the second storage device 323 are different. In this embodiment of the present invention, the first storage device 321 is composed of a storage medium with ordinary performance, such as a traditional disk, and can store user data; the second storage device 323 is composed of a high-performance storage medium, such as an SSD disk, and can be used to store hot data . When it is necessary to point out the same features of the first storage device 321 and the second storage device 323, the storage device is used for explanation. The storage devices in the embodiments of the present invention are only illustrative, and in practical applications, storage media with different performances may also be added. The number of storage media included in the storage device may also be set as required, which is not limited in this embodiment of the present invention. In addition, the storage device may be composed of storage media with the same performance, or may be composed of storage media with similar performance. The embodiment of the present invention is described by taking the data in the first storage device 321 to the second storage device 323 as an example for description. This embodiment only briefly describes the functions of the components in the storage system 3. For details of the involved method steps, please refer to the description of the foregoing method embodiment.

The storage system 3 further includes a processor 31 , and the processor includes a data acquisition and analysis module 311 and a data migration module 313 . The data collection and analysis module 311 is configured to analyze and calculate the IO request of the application accessing the storage device, obtain the number of data blocks that need to be migrated from the first storage device 321 to the second storage device 323, and notify the data migration module 313. The data migration module 313 is configured to migrate the obtained number of data blocks from the first storage device 321 to the second storage device 323 .

Specifically, the data collection and analysis module 311 is used to obtain the read and write times of the second storage device 323 within a unit time; and obtain the intensity ratio of the second storage device 323 according to the read and write times of the second storage device 323 within a unit time. ; Query the strength ratio and capacity ratio correspondence table to obtain the capacity ratio corresponding to the strength ratio; obtain the number of data blocks that need to be migrated from the first storage device 321 to the second storage device 323 according to the capacity ratio ; Send the obtained number of data blocks to be migrated to the data migration module 313 . The detailed processing method has been described in detail in the previous method, and will not be further described here.

The data acquisition and analysis module 311 obtains the intensity ratio of the second storage device 323, and obtains the number of data blocks that need to be migrated to the second storage device 323 according to the correspondence table between the intensity ratio and the capacity ratio. In this way, the amount of data to be migrated can be determined according to the performance of the second storage device 323, which can ensure that the performance of the second storage device 323 is not affected after the data is migrated, and the performance of the entire storage system is guaranteed.

The data migration module 313 is configured to migrate corresponding data blocks from the first storage device 321 to the second storage device 323 according to the received number of data blocks that need to be migrated.

The data collection and analysis module 311 is further configured to collect and analyze the IO requests of the application to the storage system 3 in advance, and obtain a correspondence table between the intensity ratio and the capacity ratio. In this way, after obtaining the intensity ratio of the second storage device 323 , the corresponding table of the intensity ratio and the capacity ratio can be directly queried to quickly obtain the number of data blocks that need to be migrated from the first storage device 321 to the second storage device 323 .

The data collection and analysis module 311 is used to query the intensity ratio and capacity ratio correspondence table to obtain the capacity ratio corresponding to the intensity ratio. Specifically: the data collection and analysis module 311 is used to query the intensity ratio and capacity ratio correspondence table, and obtains the corresponding table through fuzzy matching rules. The intensity ratio corresponds to the capacity ratio.

It is impossible to record all possible data in the correspondence table of intensity ratio and capacity ratio, so when the specific value of the calculated intensity ratio cannot be found in the correspondence table of intensity ratio and capacity ratio, according to the fuzzy matching rule, The corresponding capacity ratio is determined based on the value of the intensity ratio that is one level higher than the calculated intensity ratio in the corresponding table of intensity ratio and capacity ratio, so as to improve the search efficiency.

The data acquisition and analysis module 311 is also used to obtain the read and write times per unit time of each data block in the storage system 3, and press the obtained read and write times per unit time of each data block from high to low. Arrange in order; determine the data blocks corresponding to the capacity ratio according to the capacity ratio and the obtained read and write times per unit time of each data block; The capacity ratio corresponds to the number of data blocks.

In order to determine the migration of hot data with more access times from the first storage device 321 to the second storage device 323, the data acquisition and analysis module 311 obtains the read and write times per unit time of each data block in the storage system 3. Arrange them in order from high to low, so that the data blocks with higher number of reads and writes per unit time can be migrated from the first storage device 321 to the second storage device 321 with higher performance in order according to the determined number of data blocks to be migrated. In the storage device 323, the IO request is responded faster.

The data acquisition and analysis module 311 is further configured to acquire the read and write times of the first storage device 321 within a unit time. The intensity ratio of the second storage device 323 is the ratio of the number of reads and writes per unit time of the second storage device 323 to the number of reads and writes per unit time of the first storage device 321 and the number of reads and writes per unit time of the second storage device 323 and proportion.

The number of times of reading and writing in the first storage device 321 per unit time is the product of the number of times of reading and writing per disk in the first storage device 321 per unit time and the number of disks in the first storage device 321 divided by a first conversion factor, where the first conversion factor is related to the ratio of read requests to write requests of the first storage device 321 and the RAID level of the first storage device 321;

The number of reads and writes in the second storage device 323 per unit time is the product of the number of reads and writes per disk in the second storage device 323 per unit time and the number of disks in the second storage device 323 divided by The second scaling factor is related to the ratio of the read request to the write request of the second storage device 323 and the RAID level of the second storage device 323 .

By setting and obtaining the specific values of each parameter, the number of data blocks that need to be migrated from the first storage device 321 to the second storage device 323 can be accurately obtained, and as much hot data can be migrated to the high-performance second storage device as much as possible. In addition to the storage device 323, the performance of the storage device 323 can be guaranteed, and the efficiency and performance of the storage system can be improved.

The possible values and calculation methods of specific parameters have been described in detail in the foregoing method embodiments, and will not be described in detail here.

Embodiments of the present application further provide a computer storage medium for storing computer software instructions used by the above-mentioned storage system, including a program designed to execute the above-mentioned method embodiments. By executing the stored program, a method for migrating data between storage devices can be implemented.

Embodiments of the present application further provide a computer program, where the computer program includes instructions, when the computer program is executed by a computer, the computer can execute the processes of the above method embodiments.

Although the application is described herein in conjunction with the various embodiments, those skilled in the art will understand and understand from a review of the drawings, the disclosure, and the appended claims in practicing the claimed application. Other variations of the disclosed embodiments are implemented. In the claims, the word "comprising" does not exclude other components or steps, and "a" or "an" does not exclude a plurality. A single processor or other unit may fulfill the functions of several items recited in the claims. The mere fact that certain measures are recited in mutually different dependent claims does not indicate that these measures cannot be combined to advantage.

It should be understood by those skilled in the art that the embodiments of the present application may be provided as a method, an apparatus (apparatus), or a computer program product. Accordingly, the present application may take the form of an entirely hardware embodiment, an entirely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present application may take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) having computer-usable program code embodied therein. The computer program is stored/distributed in a suitable medium, provided with or as part of other hardware, or may take other forms of distribution, such as over the Internet or other wired or wireless telecommunication systems.

The present application is described with reference to the flowcharts and/or block diagrams of the methods, apparatuses (devices) and computer program products of the embodiments of the present application. It will be understood that each flow and/or block in the flowchart illustrations and/or block diagrams, and combinations of flows and/or blocks in the flowchart illustrations and/or block diagrams, can be implemented by computer program instructions. These computer program instructions may be provided to the processor of a general purpose computer, special purpose computer, embedded processor or other programmable data processing device to produce a machine such that the instructions executed by the processor of the computer or other programmable data processing device produce Means for implementing the functions specified in a flow or flow of a flowchart and/or a block or blocks of a block diagram.

These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable memory result in an article of manufacture comprising instruction means, the instructions The apparatus implements the functions specified in the flow or flow of the flowcharts and/or the block or blocks of the block diagrams.

These computer program instructions can also be loaded on a computer or other programmable data processing device to cause a series of operational steps to be performed on the computer or other programmable device to produce a computer-implemented process such that The instructions provide steps for implementing the functions specified in the flow or blocks of the flowcharts and/or the block or blocks of the block diagrams.

The above embodiments are only used to illustrate the technical solutions of the present invention, but not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that the foregoing embodiments can still be used for The technical solutions described in the examples are modified, or some or all of the technical features thereof are equivalently replaced; and these modifications or replacements do not make the corresponding technical solutions depart from the scope of the claims.

Claims (12)

1. A method for migrating data in a storage system, the storage system comprising a first storage device and a second storage device, characterized in that, comprising: Obtain the number of reads and writes per unit time of the second storage device; Obtain the intensity ratio of the second storage device according to the number of reads and writes per unit time of the second storage device; query the correspondence table of intensity ratio and capacity ratio, and obtain the capacity ratio corresponding to the intensity ratio; obtaining the number of data blocks that need to be migrated from the first storage device to the second storage device according to the capacity ratio; The corresponding data blocks are migrated from the first storage device to the second storage device according to the number. 2 . The method according to claim 1 , wherein the correspondence table between the intensity ratio and the capacity ratio is obtained by pre-monitoring and analysis. 3 . 3. The method according to claim 1 or 2, wherein the step of querying a correspondence table of intensity ratio and capacity ratio, and obtaining the capacity ratio corresponding to the intensity ratio is specifically: The corresponding table of intensity ratio and capacity ratio is queried, and the capacity ratio corresponding to the intensity ratio is obtained through the fuzzy matching rule. 4. The method according to claim 1 or 2, wherein the method further comprises: Acquiring the number of times of reading and writing in the unit time of each data block in the storage system, and arranging the number of times of reading and writing in the unit time of each data block obtained in order from high to low; Obtaining the number of data blocks to be migrated from the first storage device to the second storage device according to the capacity ratio specifically includes: Determine the data block corresponding to the capacity ratio according to the capacity ratio and the obtained read and write times per unit time of each data block; The number of data blocks corresponding to the capacity ratio in the first storage device is confirmed. 5. The method according to claim 1 or 2, wherein the method further comprises: Obtain the number of times of reading and writing in the first storage device per unit time; The strength ratio of the second storage device is the sum of the number of reads and writes per unit time of the second storage device to the sum of the number of reads and writes per unit time of the first storage device and the number of reads and writes per unit time of the second storage device. proportion. 6. The method according to claim 5, wherein the number of times of reading and writing per unit time of the first storage device is the number of times of reading and writing per disk in the first storage device and the number of times of reading and writing per unit time of the first storage device. dividing the product of the number of disks in the storage device by a first conversion factor, the first conversion factor being related to the ratio of read requests to write requests of the first storage device and the RAID level of the first storage device; The number of reads and writes per unit time of the second storage device is the product of the number of reads and writes per disk per unit time in the second storage device and the number of disks in the second storage device, and then divided by the second conversion coefficient, the second conversion coefficient is related to the ratio of the read request to the write request of the second storage device and the RAID level of the second storage device. 7 . The method according to claim 6 , wherein the method further comprises: the number of reads and writes per disk per unit time in the first storage device and the load characteristics and response duration of the first storage device. 8 . related; The number of reads and writes of a single disk in the second storage device per unit time is related to the load characteristics and response duration of the second storage device. 8. A storage system for implementing data migration, wherein the storage system comprises a first storage device, a second storage device, and a processor, and the processor is configured to execute the method of claims 1-7. 9. A storage system for implementing data migration, wherein the storage system comprises a first storage device, a second storage device and a processor, and the processor includes a data acquisition and analysis module and a data migration module; The data acquisition and analysis module is used to obtain the read and write times of the second storage device per unit time; obtain the intensity ratio of the second storage device according to the read and write times of the second storage device per unit time; capacity ratio correspondence table, obtain the capacity ratio corresponding to the strength ratio; obtain the number of data blocks that need to be migrated from the first storage device to the second storage device according to the capacity ratio; The number of data blocks is sent to the data migration module; The data migration module is used for migrating the obtained number of data blocks from the first storage device to the second storage device. 10. The storage system according to claim 9, wherein the data acquisition and analysis module is also used for: Collect and analyze the IO requests of the application to the storage system in advance, and obtain the corresponding table of intensity ratio and capacity ratio. 11. The storage system according to claim 9 or 10, wherein the data acquisition and analysis module is used to query the intensity ratio and capacity ratio correspondence table to obtain the capacity ratio corresponding to the intensity ratio, specifically: the data acquisition and analysis module uses For querying the strength ratio and capacity ratio correspondence table, obtain the capacity ratio corresponding to the strength ratio through a fuzzy matching rule. 12. The storage system according to claim 9 or 10, wherein the data acquisition and analysis module is also used to obtain the number of times of reading and writing in the unit time of each data block in the storage system, and will obtain each data block. The number of reads and writes within a unit time of a data block is arranged in order from high to low; The data acquisition and analysis module obtains, according to the capacity ratio, the number of data blocks that need to be migrated from the first storage device to the second storage device. The number of data blocks migrated from the first storage device to the second storage device.

Images