CN101349983B - Method and device for measuring disk effective cache capacity - Google Patents

Abstract

The invention discloses a method and device for measuring the effective cache (Cache) capacity of a disk, wherein the method includes: A. According to a preset number of times H, cyclically read M data blocks of a fixed size H times, and calculate the current cycle B. Increase M according to a certain step size, and judge whether M is smaller than the preset maximum value N. If yes, repeat the above step A; if not, execute step C; C. Whether the average throughput decreases and remains stable, and if so, obtain the M value when the average throughput decreases, and calculate the effective cache capacity as the size of M data blocks. The method and device proposed by the invention can measure the effective cache capacity of the disk cache.

Description

Method and device for measuring disk effective cache capacity

technical field

The invention relates to measurement technology, in particular to a method for measuring the effective cache (Cache) capacity of a disk.

Background technique

In the prior art, in order to increase the reading and writing speed of a central processing unit (CPU) to a physical disk, a disk cache is usually set between the CPU and the physical disk to reduce the number of direct accesses to the physical disk by the CPU. In this way, if the CPU needs to cache the data read from the physical disk in the disk Cache, the CPU can directly read from the disk Cache; if the CPU needs to write data to the physical disk, there are two methods, one of which is also In the commonly used method, the CPU only needs to write the data to the Cache to return the write success message, so that the CPU does not need to wait for the physical disk's mechanical arm to move the addressing process; another method is to wait for the data to be completely written to the disk before returning Write a success message, this method is relatively inefficient.

Among them, according to the replacement algorithm, when the disk Cache can no longer accommodate more data, if the data in the physical disk needs to be read into the disk Cache, or the CPU needs to rewrite data to the disk Cache, it is necessary to use the pre-set The preset scheduling algorithm replaces some existing data in the disk cache.

In this way, since the CPU does not need to wait for the addressing process of the physical disk to move through its mechanical arm, the read and write speed of the CPU to the physical disk is greatly accelerated.

In practical applications, the reading speed of the CPU to the physical disk depends on the access speed of the disk cache, and the access speed of the disk cache depends on the capacity of the storage space that can actually be used in the disk cache, that is, the effective cache capacity.

However, due to the limitations of various existing scheduling algorithms, the actual capacity of the disk Cache generally cannot be utilized 100%, that is, the effective Cache capacity generally cannot reach the maximum capacity of the disk Cache, thus making it possible for a disk Cache with a larger actual capacity to Due to its low effective Cache capacity, it cannot effectively improve the reading speed of the CPU to the physical disk.

It can be seen that, when selecting a disk cache, not only its nominal capacity must be considered, but more importantly, the effective cache capacity of the disk cache must also be considered.

However, in the prior art, there is no solution capable of measuring the size of the effective cache capacity, so that the disk cannot be better selected according to the size of the effective cache capacity.

Contents of the invention

In view of this, the present invention provides a method for measuring the effective cache capacity of the disk, which can measure the effective cache capacity of the disk cache.

The invention also provides a measuring device for the effective Cache capacity of the disk, capable of measuring the effective Cache capacity of the disk Cache.

Technical scheme of the present invention is realized like this:

A kind of measuring method of disk effective Cache capacity provided by the invention comprises:

A. Read M data blocks of fixed size H times in a cycle according to the preset number H, and calculate the average throughput of this cycle; said H is greater than 0;

B. Increase M according to a certain step size, judge whether M is less than the preset maximum value N, if yes, then repeat the above step A; if not, then perform step C; said N is greater than the manufacturer of the magnetic disk The nominal cache capacity is divided by the data block size and rounded up; the initial value of M is 1, and the step size of M is 1;

C. Judging whether the average throughput drops and remains stable, if so, obtain the M value when the average throughput drops, and calculate the effective cache capacity as the M value and the data block size when the average throughput drops product.

The present invention also provides a kind of measuring device of disk effective Cache capacity, comprising:

The reading module is used to cyclically read M data blocks of a fixed size H times, and the H is a preset number of times;

The average throughput calculation module is used to calculate the average throughput of the reading module's cyclic reading;

A judging module, configured to judge whether the number M of data blocks read by the reading module is less than a preset maximum value N, if so, increase the value of M according to a certain step size, and instruct the reading module to recycle Read the data block, the number of the data block is the increased M value; if not, then instruct the effective cache capacity calculation module to calculate;

The effective buffer capacity calculation module is used to obtain all the average throughput calculated by the average throughput calculation module after receiving the instruction of the judgment module, and judge whether the average throughput has declined and remains stable, and if so, obtain the average throughput. The M value when the throughput drops, and the effective cache capacity is calculated as the product of the M value and the data block size when the average throughput drops;

The H is greater than 0; the N is greater than the nominal cache capacity of the disk manufacturer divided by the data block size and rounded up; the initial value of M is 1, and the step size of M increases is 1.

It can be seen from the above technical solution that the method and device proposed by the present invention gradually increase the number of data blocks by cyclically reading a plurality of continuous data blocks, so that the throughput remains stable after a significant decline, and when the throughput begins to decline, the read The number of data blocks is taken, and the effective cache capacity of the disk can be determined according to the number of blocks at the drop point and the size of the data block.

Description of drawings

FIG. 1 is a schematic diagram of a variation curve of throughput in an embodiment of the present invention;

Fig. 2 is a schematic diagram of the mean value and standard deviation curve of the average throughput of the embodiment of the present invention;

Fig. 3 is the flow chart of the method for measuring disk effective Cache capacity according to an embodiment of the present invention;

Fig. 4 is the first result schematic diagram of applying the method of the present invention to test the effective Cache capacity of disk A;

Fig. 5 is the second result schematic diagram of applying the method of the present invention to test the effective Cache capacity of disk A;

Fig. 6 is the first result schematic diagram of applying the method of the present invention to test the effective Cache capacity of disk B;

FIG. 7 is a schematic diagram of the second result of testing the effective Cache capacity of disk B by applying the method of the present invention.

Detailed ways

In order to make the object, technical solution and advantages of the present invention clearer, the present invention will be further described in detail below with reference to the accompanying drawings and examples.

The embodiment of the present invention proposes a method for measuring the effective cache capacity of a disk, including:

A. According to the preset number of times H, read M data blocks of fixed size H times in a loop, and calculate the average throughput of this loop;

B. Increase M according to a certain step size, and judge whether M is smaller than the preset maximum value N, if yes, repeat the above step A; if not, then execute step C;

C. Judging whether the average throughput drops and remains stable, if so, obtain the M value when the average throughput drops, and calculate the effective cache capacity as the product of the M value when the average throughput drops and the data block size .

As mentioned earlier, when reading data, the system first checks whether the requested data is in the Cache, if it is in the Cache, it will be read directly from the Cache; if it is not in the Cache, it will be read from the disk, and the read The data is kept in the Cache, and the next time the data is read, it can be read directly from the Cache. The present invention utilizes this feature just to read M data blocks of a large file in a loop until M is large enough to make the Cache invalid, and the read throughput will significantly decrease with the increase of the M value and finally It tends to be stable and the M value when the throughput drops is the calculation point of the effective cache size. As shown in FIG. 1 , FIG. 1 is a schematic diagram of a variation curve of throughput according to an embodiment of the present invention. In Figure 1, the throughput decreases and remains stable with the increase of M, and the drop point B is the calculation point of the effective cache size, that is:

Effective cache size = data block size × number of data blocks at point B

The following is a specific introduction in combination with the above methods:

In step A, when reading for the first time, the initial value of M should be made small enough to ensure that the total amount of M data blocks is less than the capacity of the effective Cache. When M data blocks are read for the first time, a mandatory invalidation generally occurs, because the data blocks read for the first time are generally not in the Cache. At this time, the data block needs to be read from the disk and stored in the Cache. Afterwards, when looping to read M data blocks, if the total size of the read data blocks is smaller than the effective cache, since the M data blocks read for the first time have been saved in the Cache, subsequent data blocks will hit , can be read directly from the Cache, and the calculated average throughput at this time is equivalent to the throughput of reading directly from the Cache, and the throughput is relatively high.

The average throughput is calculated after looping H times in step A, the purpose of which is to eliminate the influence of a small number of error data in the test. The value of H should be large enough.

In step B, the main purpose of increasing the value of M is to make the read data blocks from small to large eventually exceed the size of the effective cache. M is gradually increased according to a certain step size, the purpose of which is to make the measurement accuracy higher, and different measurement accuracy can be determined by modifying the step size of M increase. In the embodiment of the present invention, the step size is taken as 1.

If the total amount of data blocks read in each cycle exceeds the size of the effective cache by modifying the value of M, data block replacement will occur during the cycle read process. For example: in order to read data block B into the cache, according to the replacement algorithm, the previously read data block A will be replaced. When the data block A is read again and the data block is missing, the data block will be generated again. Replacement may replace data block C, which is constantly replaced and missing, resulting in almost all data blocks needing to be read directly from the disk. The work of the cache is to replace continuously, but rarely hits. The average throughput calculated in this case is equivalent to the throughput of reading directly from the disk, and the throughput is very low.

There are two ways to judge whether the average throughput drops significantly and whether the results of multiple drops remain stable: ① draw a curve according to the output average throughput results, and directly observe whether it drops significantly and remains stable. This method is simple and suitable for testing The resulting data is relatively standardized. ②According to the test results, for the average throughput of each output, calculate the mean and standard deviation of the previous X consecutive average throughput (including the current average throughput), and draw two corresponding curves. As shown in FIG. 2 , FIG. 2 is a schematic diagram of the mean value and standard deviation curve of the average throughput of the embodiment of the present invention. In the case that the Cache does not fail, the calculated average is relatively high, and the value difference is not large, that is, the standard deviation value is relatively small; if at a certain point B, the Cache fails, then there are X consecutive values before this point (including this point) The mean value of will drop, and the standard deviation value will increase significantly, that is, the throughput variance will increase. According to this method, after point C, the mean value of the continuous output X results maintains a relatively low value after a decline and eventually flattens out, while the standard deviation value gradually decreases after a significant increase, and finally flattens out , and finally maintained in a relatively low value range, which can be considered as a stable state, then point B is the corresponding point of Cache failure, and it is considered that the curve remains stable after falling. The product of the number of data blocks corresponding to point B and the size of the data block is equal to the capacity of the effective Cache.

In addition, the maximum value N can be calculated according to the manufacturer's nominal cache size, specifically:

其中，符号

表示向上取整，其含义是取不小于某数的最小整数；n为自然数，加n的目的是使得N的值大于磁盘厂家标称的Cache容量除以数据块大小后向上取整的值，则N个数据块的容量必然大于有效Cache容量，当M增大到N时，一定会引起Cache失效。

Among them, the symbol

Indicates rounding up, and its meaning is to take the smallest integer not less than a certain number; n is a natural number, and the purpose of adding n is to make the value of N larger than the disk manufacturer’s nominal cache capacity divided by the data block size and then rounding up. Then the capacity of N data blocks must be greater than the effective Cache capacity, and when M increases to N, it will definitely cause Cache failure.

When the manufacturer's nominal cache size cannot be known, the value of N can be calculated based on the capacity of mainstream caches in the market. If M increases to the preset N, the average throughput does not decline and remains stable, indicating that the preset N value is not large enough, then increase the N value at a preset speed, such as increasing the N value To double the original, continue to perform the steps of cyclic reading and calculating the average throughput until the average throughput drops and remains stable, obtain the M value when the average throughput drops, and calculate the effective Cache capacity as the average throughput The product of the M value and the data block size when the volume drops.

Specific embodiments are given below for introduction, referring to FIG. 3 , which is a flowchart of a method for measuring effective Cache capacity according to an embodiment of the present invention. include:

Step 301: Set the maximum value N of read data blocks, set the number of read cycles H, and set the initial value of the number of read data blocks to 1, that is, the initial value of M is 1. Wherein, the setting method of N is the same as the above method, which will not be repeated here. The value of H needs to be large enough to ensure that the calculated average throughput is as accurate as possible.

Step 302: cyclically read M data blocks H times, and calculate the average throughput of this cycle. To increase the value of M, in this embodiment, the step size of M can be set to be 1, that is, M=M+1.

Step 303: Judging whether M is smaller than the preset N, if yes, execute step 302; otherwise, execute step 304.

Step 304: Judging whether the average throughput calculated in each cycle decreases and remains stable. In this embodiment, the judging method may be: judging according to the mean value and standard deviation. If there is a drop and remains stable, go to step 305 . If the preset N is calculated based on the manufacturer's nominal cache capacity, the average throughput will definitely decrease and remain stable. However, if the preset N is calculated based on the mainstream Cache capacity in the market, when M increases to N, the total length of M data blocks may not exceed the effective Cache capacity, and there will be no average A drop in throughput. When it is determined that the average throughput does not decrease and remains stable, step 306 is executed.

Step 305: Calculate the effective Cache capacity=the value of M when the average throughput decreases×the data block size, and end this process.

Step 306: Accelerate the increase of the value of N. In this embodiment, N is doubled, ie, N=N×2, and M is increased by 1, and step 302 is continued.

After that, you can increase the value of N multiple times until the average throughput does not decrease and remains stable, then calculate the effective cache capacity.

In addition, in the method proposed by the present invention, the clock cycle is used as the timing unit, so the timing is accurate and the error is small. The timing starts after each M value is set, and the timing ends after all reading cycles of the current M value are completed, and the average throughput of this cycle is calculated.

The following application method proposed by the present invention tests the effective Cache of the disk. The test environment is: CPU3.0GHz, Gigabit network card, two SCSI disks that are 147G, wherein disk A is produced by manufacturer 1, and the nominal Cache is 8M. B is produced by manufacturer 2, the nominal Cache is 8M, and it was tested on WINDOWSSERVER 2003. Among them, the read data block size is 64K.

First, test the effective cache capacity of disk A. Referring to FIG. 4 , FIG. 4 is a schematic diagram of the first result of testing the effective Cache capacity of disk A by applying the method of the present invention. In Figure 4, the X-axis is the number of data blocks read, and the Y-axis is the average throughput in MB/S. It can be seen from Figure 4 that when the number of data blocks is 82, the average throughput begins to decline, and then gradually remains stable.

Then calculate the mean value and standard deviation of the average throughput according to the results in FIG. 4 , as shown in FIG. 5 , which is a schematic diagram of the second result of applying the method of the present invention to test the effective Cache capacity of disk A. In 5, the X-axis is the serial number of the mean value, the upper curve represents the mean value calculated by taking 10 consecutive average throughputs, and the lower curve represents the standard deviation of the average throughput. It can be seen from Figure 5 that when the 73rd mean value is calculated, the mean value of the average throughput begins to decrease, and then gradually remains stable; and when the 73rd mean value is calculated, the standard deviation of the average throughput begins to increase and reaches the highest value decrease and maintain a small value. Since the first average value can only be calculated when the 10th average throughput occurs; therefore, the moment of the 73rd average value corresponds to the 82nd average throughput, and the number of data blocks read at this time is also 82. It can be seen that the verification result is the same as that in Figure 4.

From this, it can be calculated that the effective cache capacity of disk A=64K×82=5.125M

Then test the effective cache capacity of disk B. FIG. 6 is a schematic diagram of the first result of testing the effective Cache capacity of disk B by applying the method of the present invention. FIG. 7 is a schematic diagram of the second result of testing the effective Cache capacity of disk B by applying the method of the present invention. Both Figure 6 and Figure 7 show that when the number of data blocks is 19, the average throughput begins to decline, and then gradually remains stable. From this, it can be calculated that the effective cache capacity of disk B=64K×19=1.826M

The embodiment of the present invention also proposes a device for measuring the effective cache capacity of a disk, including:

The reading module is used to cyclically read M data blocks of a fixed size H times, and the H is a preset number of times;

The average throughput calculation module is used to calculate the average throughput of the reading module's cyclic reading;

A judging module, configured to judge whether the number M of data blocks read by the reading module is less than a preset maximum value N, if so, increase the value of M according to a certain step size, and instruct the reading module to recycle Read the data block, the number of the data block is the increased M value; if not, then instruct the effective cache capacity calculation module to calculate;

The effective buffer capacity calculation module is used to obtain all the average throughput calculated by the average throughput calculation module after receiving the instruction of the judgment module, and judge whether the average throughput has declined and remains stable, and if so, obtain the average throughput. For the M value when the throughput drops, the effective cache capacity is calculated as the product of the M value when the average throughput drops and the data block size.

The above-mentioned effective cache capacity calculation module is also used to calculate the average value and standard deviation of a plurality of consecutive average throughputs, and judge whether the average throughput decreases and remains stable according to the average value and standard deviation.

The above judging module is also used for, when the effective cache capacity computing module judges that the average throughput has not declined and remains stable, increasing the N value at a preset speed to judge the data block read by the reading module Whether the number M is smaller than the increased N value, if yes, instruct the reading module to re-execute cyclic reading, if not, instruct the effective cache capacity calculation module to perform calculation.

In the above device, the N is greater than the value rounded up after dividing the cache capacity advertised by the cache manufacturer by the size of the data block.

In summary, the method and device for measuring the effective cache capacity of a disk proposed by the embodiments of the present invention take blocks as units, read multiple data blocks from a large file in a loop, and gradually increase the number of continuously read data blocks , until the throughput drops significantly and remains stable after several consecutive drops, determine the number of data blocks read when the throughput starts to drop, and determine the effective cache of the disk according to the number of blocks at the drop point and the size of the data block capacity. The calculation results can be used as a reference for evaluating disk cache algorithms.

The above descriptions are only preferred embodiments of the present invention, and are not intended to limit the protection scope of the present invention. Any modifications, equivalent replacements and improvements made within the spirit and principles of the present invention shall be included within the protection scope of the present invention.

Claims (6)

1. the measuring method of a disk valid cache capacity is characterized in that, described method comprises:

A, according to predefined number of times H, data block H time of M fixed size is read in circulation, calculates this round-robin average throughput; Described H is greater than 0;

B, M is increased according to a fixed step size, whether judge M less than predefined maximal value N, if then repeat above-mentioned steps A; If not, execution in step C then; The value that described N rounds up after divided by the data block size greater than the buffer memory capacity of institute of the producer nominal of described disk; The initial value of described M is 1, and the step-length that described M increases is 1;

C, judge described average throughput whether occur descending and keep stable, if, the M value when then obtaining average throughput and occurring descending, the amassing of M value when calculating the valid cache capacity and be described average throughput and occurring descending and data block size.

2. method according to claim 1 is characterized in that, describedly judges whether average throughput occurs descending and keep stable manner and is: calculate the average and the standard variance of continuous a plurality of average throughputs, judge according to described average and standard variance.

3. method according to claim 1 and 2, it is characterized in that, described method further comprises: if descending and keeping stable does not appear in described average throughput, then increase the N value according to predefined speed, continue to carry out described steps A and step B, occur to descend and keep when stablize until average throughput, the M value when obtaining average throughput and occurring descending, calculating valid cache capacity.

4. the measurement mechanism of a disk valid cache capacity is characterized in that, described device comprises:

Read module, data block H time that is used to circulate and reads M fixed size, described H is predefined number of times;

The average throughput computing module is used to calculate the average throughput that the read module circulation is read;

Judge module, whether the number M of the data block that is used to judge that described read module reads is less than predefined maximal value N, if then increase the M value according to a fixed step size, the indication read module read block that circulates again, the number of described data block is the M value after increasing; If not, then indicate valid cache calculation of capacity module to calculate;

Valid cache calculation of capacity module, be used for after the indication that receives judge module, obtain all average throughputs that the average throughput computing module is calculated, judge whether described average throughput occurs descending and keeping stable, if, M value when then obtaining average throughput and occur descending, the amassing of M value when calculating the valid cache capacity and be described average throughput and occurring descending and data block size;

Described H is greater than 0; The value that described N rounds up after divided by the data block size greater than the buffer memory capacity of institute of the producer nominal of described disk; The initial value of described M is 1, and the step-length that described M increases is 1.

5. device according to claim 4, it is characterized in that, described valid cache calculation of capacity module also is used for, and calculates the average and the standard variance of continuous a plurality of average throughputs, judges according to described average and standard variance whether average throughput occurs descending and keeping stable.

6. according to claim 4 or 5 described devices, it is characterized in that, described judge module also is used for, judge average throughput when effective buffer memory capacity computing module and do not occur descending and keep when stablize, increase the N value according to predefined speed, whether the number M of judging the data block that described read module reads is less than the N value after the increase, if, then indicate read module to re-execute circulation and read, if not, then indicate valid cache calculation of capacity module to calculate.

Images