CN114879909A - Data storage method and device, electronic equipment and storage medium - Google Patents

Abstract

The invention discloses a data storage method and device, electronic equipment and a storage medium. Wherein, the method comprises the following steps: receiving a data processing request initiated by a client, wherein the data processing request at least carries: service data to be processed; storing the service data into a plurality of cache blocks, and identifying the cache blocks to obtain a plurality of identified cache blocks; combining the plurality of identification cache blocks to obtain a write-back request under the condition that the number of the plurality of identification cache blocks is greater than a preset number threshold; write back requests are written to respective segmented data spaces in the data plane and a dynamic mapping of data pointed to the segmented data spaces by the identified cache blocks is determined. The invention solves the technical problem that the random write-back request can not be effectively processed when the data is written back to the data layer from the cache layer.

Description

Data storage method and device, electronic device, storage medium

technical field

The present invention relates to the technical field of data storage, and in particular, to a data storage method and device, an electronic device, and a storage medium.

Background technique

In the related art, the current data storage system generally includes a variety of storage media (for example, hard disk drive HDD and solid state drive SATA SSD) or mixed media as a cost-effective solution, usually divided into a cache layer (or cache disk) and a data layer. (or data layer), the cache layer generally includes high-speed disks (such as NVMe SSD or ordinary SSD), and the data layer includes relatively slow disks (such as ordinary SSD, HDD or QLC SSD), among which, the capacity of the cache disk less than the data disk capacity. In this way, the request from the client is first written to the cache disk, and then it can be returned immediately, and then the data is slowly written to the data disk through the writeback mechanism (writeback). It will interfere with front-end IO.

In the process of data storage and write-back through the data storage system, the core is the Cache algorithm, and the core of the write-back mechanism in the Cache algorithm is the mapping relationship between the block of the cache disk and the block of the data disk. The current mechanism is usually to divide the cache disk into several blocks (block size can be 4KB, 8KB, etc.), and then we also divide the data disk into several corresponding blocks; the current request needs to be written to the data disk Block A At this time, the Cache algorithm will select a certain Block A' of an SSD, first write the data to Block A', and then mark this Block A' as dirty data (Dirty), and then create a cache disk Block A' The mapping relationship with the data disk Block A.

In the above-mentioned Cache algorithm, the blocks of the cache disk and the blocks of the low-speed data disk are in one-to-one correspondence. However, this process does not re-regulate the requests from the front-end, that is, if the front-end IO is random, the write-back process will also be random, and the random write-back will put a lot of pressure on the data disk. If it is composed of HDD or QLC SSD, the pressure on the data disk will be great.

For the above problems, no effective solution has been proposed yet.

SUMMARY OF THE INVENTION

Embodiments of the present invention provide a data storage method and device, an electronic device, and a storage medium to at least solve the technical problem that a random write-back request cannot be effectively processed when data is written back from the cache layer to the data layer.

According to an aspect of the embodiments of the present invention, a data storage method is provided, which is applied to a cache layer in a data storage system. The cache space corresponding to the cache layer includes a plurality of pre-divided cache blocks, including: receiving a client The data processing request initiated by the terminal, wherein the data processing request at least carries: the business data to be processed; the business data is stored in multiple cache blocks, and the multiple cache blocks are identified to obtain multiple cache blocks. each identifier cache block; when the number of the multiple identifier cache blocks is greater than the preset number threshold, combine the multiple identifier cache blocks to obtain a write-back request; write the write-back request into the data layer in each segmented data space, and determine the data dynamic mapping relationship pointed to the segmented data space by the identified cache block.

Optionally, the step of combining the multiple identification cache blocks to obtain a write-back request includes: acquiring a cache interval to which each identified cache block belongs, and determining an interval start identifier of the cache interval; based on the The start identifier of the interval, and the cache interval is merged to obtain the write-back request.

Optionally, after writing the write-back request into each segmented data space in the data layer, the method further includes: acquiring interval information of the cache interval and the pre-stored service data; The data is preprocessed to obtain business processing data and preprocessing information; the business processing data is determined as segmented data, and the preprocessing information is determined as segmented header information; the segmented header information and the The segment data is combined into segment record information; the segment record information is additionally written into the segment data space.

Optionally, the step of determining the dynamic mapping relationship of data pointed to the segmented data space by the identifier cache block includes: based on the interval start identifier of the cache interval to which the identifier cache block belongs and the interval of the cache interval. size, determine the mapping start bit; obtain the segment start bit in the overall data space of the segmented data space in the data layer; based on the segment start bit, determine the cache interval to which the identified cache block belongs The segment space start bit and segment space size in the segment data space; establish a data dynamic mapping relationship in which the mapping start bit points to the segment space start bit.

Optionally, after determining the data dynamic mapping relationship from the identification cache block to the segmented data space, the method further includes: when a new write-back request is generated, assigning a valid segment to the new write-back request. segment data space; write the new write-back request to the allocated valid segment data space.

Optionally, it also includes: in the case of generating a new write-back request, judging whether the cache interval used by the new write-back request is reused; if the cache interval used by the new write-back request is reused In this case, the reclaiming process is performed on the segmented data space pointed to by the repeatedly used cache section in the historical process, and the reclaimed segmented data space is marked invalid.

Optionally, the step of reclaiming the segmented data space pointed to in the historical process by the repeatedly used cache interval includes: when there are multiple segmented data spaces to be reclaimed, Analyzing the space usage rate and the amount of internal storage data of the segmented data space; according to the sorting order of the space usage rate and the amount of internal storage data, reclaim the plurality of segmented data spaces to be reclaimed respectively. .

According to another aspect of the embodiments of the present invention, a data storage device is also provided, which is applied to a cache layer in a data storage system. The cache space corresponding to the cache layer includes a plurality of pre-divided cache blocks, including: a receiving unit, configured to receive a data processing request initiated by a client, wherein the data processing request at least carries: business data to be processed; a storage unit, configured to store the business data in multiple cache blocks, and Identifying the multiple cache blocks to obtain multiple identified cache blocks; a combining unit for combining the multiple identified cache blocks when the number of the multiple identified cache blocks is greater than a preset number threshold, Obtaining a write-back request; a writing unit, configured to write the write-back request into each segmented data space in the data layer, and determine the data dynamic mapping pointed to the segmented data space by the identification cache block relation.

Optionally, the combining unit includes: a first acquiring module, configured to acquire a cache interval to which each of the identified cache blocks belongs, and determine an interval start identifier of the cache interval; a first combining module, configured to be based on The interval start identifier is combined with the cache interval to obtain the write-back request.

Optionally, the data storage device further includes: a second obtaining module, configured to obtain interval information of the cache interval and pre-store the interval information of the cache interval after writing the write-back request into each segmented data space in the data layer. the business data; a preprocessing module for preprocessing the business data to obtain business processing data and preprocessing information; a first determining module for determining the business processing data as segmented data, The preprocessing information is determined as segment header information; the second merging module is used to combine the segment header information and the segment data into segment record information; the first write module is used to The segment record information is additionally written into the segment data space.

Optionally, the writing unit includes: a first determining module configured to determine a mapping start bit based on an interval start identifier of the buffer interval to which the identified cache block belongs and an interval size of the buffer interval; a third an acquisition module, configured to acquire the segment start bit of the segment data space in the overall data space in the data layer; The start bit of the segment space and the size of the segment space in the segment data space; the first establishing module is used to establish a data dynamic mapping relationship in which the start bit of the mapping points to the start bit of the segment space.

Optionally, the data storage device further includes: a first allocation module, configured to, after determining the data dynamic mapping relationship from the identification cache block to the segmented data space, in the case of generating a new write-back request, for The new write-back request allocates a valid segment data space; the second writing module is configured to write the new write-back request into the allocated valid segment data space.

Optionally, the data storage device further includes: a first judging module for judging whether the cache interval used by the new write-back request is reused when a new write-back request is generated; a recycling module for When the cache interval used by the new write-back request is reused, the segment data space pointed to by the reused cache interval in the historical process is reclaimed, and the reclaimed segment data space is reclaimed. Segment data space is invalid marked.

Optionally, the recycling module includes: an analysis submodule, configured to analyze the space usage rate and the amount of internal storage data of the segmented data space when there are multiple segmented data spaces to be recycled; The recycling sub-module is configured to perform recycling processing on a plurality of the segmented data spaces to be recycled according to the sorting order of the space usage rate and the amount of the internal storage data.

According to another aspect of the embodiments of the present invention, there is also provided an electronic device, comprising: a processor; and a memory for storing executable instructions of the processor; wherein the processor is configured to execute the The instructions are executable to perform the data storage method described in any of the above.

According to another aspect of the embodiments of the present invention, there is also provided a computer-readable storage medium, the computer-readable storage medium comprising a stored computer program, wherein the computer-readable storage medium is controlled when the computer program is executed The device where the medium is located executes the data storage method described in any one of the above.

In this embodiment of the present invention, a data processing request initiated by a client is received, wherein the data processing request at least carries: business data to be processed, the business data is stored in multiple cache blocks, and the multiple cache blocks are identified , obtain multiple identification cache blocks, and when the number of multiple identification cache blocks is greater than the preset number threshold, combine multiple identification cache blocks to obtain a write-back request, and write the write-back request to each partition in the data layer In the segment data space, and determine the data dynamic mapping relationship from the identification cache block to the segment data space. In this embodiment, the random small-block write data from the front end can be converted into a sequential large-block write-back request through the data re-warping technology, which can cope with random write-back in different situations and improve the access speed of the mixed storage medium. Realize quick data sorting, improve data processing efficiency, and solve the technical problem that random write-back requests cannot be effectively processed when data is written back from the cache layer to the data layer.

Description of drawings

The accompanying drawings described herein are used to provide a further understanding of the present invention and constitute a part of the present application. The exemplary embodiments of the present invention and their descriptions are used to explain the present invention and do not constitute an improper limitation of the present invention. In the attached image:

1 is a flowchart of an optional data storage method according to an embodiment of the present invention;

2 is a schematic diagram of an optional data storage device according to an embodiment of the present invention;

FIG. 3 is a block diagram of a hardware structure of an electronic device (or mobile device) of a data storage method according to an embodiment of the present invention.

Detailed ways

In order to make those skilled in the art better understand the solutions of the present invention, the technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only Embodiments are part of the present invention, but not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by persons of ordinary skill in the art without creative efforts shall fall within the protection scope of the present invention.

It should be noted that the terms "first", "second" and the like in the description and claims of the present invention and the above drawings are used to distinguish similar objects, and are not necessarily used to describe a specific sequence or sequence. It is to be understood that the data so used may be interchanged under appropriate circumstances such that the embodiments of the invention described herein can be practiced in sequences other than those illustrated or described herein. Furthermore, the terms "comprising" and "having" and any variations thereof, are intended to cover non-exclusive inclusion, for example, a process, method, system, product or device comprising a series of steps or units is not necessarily limited to those expressly listed Rather, those steps or units may include other steps or units not expressly listed or inherent to these processes, methods, products or devices.

For the convenience of those skilled in the art to understand the present invention, some terms or nouns involved in each embodiment of the present invention are explained below:

Mechanical disk, Hard Disk Drive, referred to as HDD;

Solid state disk, Solid State Disk, referred to as SSD, flash memory particles of storage media;

NVMe SSD, SSD with NVMe interface, a high-speed SSD;

QLC (quad-level cell) SSD, a high-density SSD, its price is lower than the previous SSD, but its write hit is relatively short.

The present invention can be applied to various data storage systems. The data storage system adopts a variety of persistent devices. The persistent devices may include but are not limited to: a cache layer and a data layer, wherein the cache layer may include high-speed disks (such as NVMeSSD or Ordinary SSD), the data tier will include relatively slow disks (eg, ordinary SSDs, HDDs, or QLC SSDs), where the cache tier capacity is smaller than the data tier capacity.

The present invention adopts a mixed media management mechanism in the data storage system, which can manage one disk or a logical space composed of a group of disks, that is, several cache disks can be composed into a logical cache space, and relatively low-speed cache disks can be combined into a logical cache space. The data disk forms a logical low-speed data space, and the write-back mechanism from the cache space to the low-speed data space can be implemented according to the same strategy.

In the present invention, the logical data space scheme composed of multiple groups of disks, under the premise of random small IO, when the cache space is insufficient and the data needs to be written back to the data layer, the data disk is inefficient for random small IO data processing. , this application introduces the data re-arrangement technology, which converts random small IOs into sequential large IOs, fully volatilizing the large IO processing capability of the data disk, that is, the re-arranging technology can be used to conveniently fill data, without needing to store data in the data layer. Divide the data blocks (block blocks) corresponding to the cache layer one-to-one.

In this application, the write-back data can be set to meet the storage conditions of the disk storage medium, so as to improve the processing capability of the disk (for example, if the resource pool adopts EC as the copy strategy, the data can be filled to align the EC stripe; for example, , the SSD write behavior can be optimized by aligning the erase block size of the data pool SSD).

The present invention will be described below with reference to various embodiments.

Example 1

According to an embodiment of the present invention, an embodiment of a data storage method is provided. It should be noted that the steps shown in the flowchart of the accompanying drawings may be executed in a computer system such as a set of computer-executable instructions, and, although in A logical order is shown in the flowcharts, but in some cases steps shown or described may be performed in an order different from that herein.

This embodiment provides a data storage method, which is applied to a cache layer in a data storage system, where a cache space corresponding to the cache layer includes a plurality of pre-divided cache blocks.

FIG. 1 is a flowchart of an optional data storage method according to an embodiment of the present invention. As shown in FIG. 1 , the method includes the following steps:

Step S102, receiving a data processing request initiated by the client, wherein the data processing request at least carries: business data to be processed;

Step S104, storing the business data in multiple cache blocks, and identifying multiple cache blocks to obtain multiple identified cache blocks;

Step S106, in the case that the number of multiple identification cache blocks is greater than the preset number threshold, combine multiple identification cache blocks to obtain a write-back request;

Step S108, write the write-back request into each segmented data space in the data layer, and determine the data dynamic mapping relationship pointed to by the identification cache block to the segmented data space.

Through the above steps, a data processing request initiated by the client can be received, wherein the data processing request at least carries: the business data to be processed, the business data is stored in multiple cache blocks, and the multiple cache blocks are identified to obtain Multiple identifier cache blocks, when the number of multiple identifier cache blocks is greater than the preset number threshold, combine multiple identifier cache blocks to obtain a write-back request, and write the write-back request to each segmented data in the data layer space, and determine the data dynamic mapping relationship from the identification cache block to the segmented data space. In this embodiment, the random small-block write data from the front end can be converted into sequential large-block write-back requests through the data re-warping technology, so that different write-back requests can be handled, data can be quickly sorted, and data processing efficiency can be improved , so as to solve the technical problem that the random write-back request cannot be effectively processed when data is written back from the cache layer to the data layer.

In this embodiment, the data re-regularization technology is introduced. For the Cache algorithm used by the data storage system, when establishing the mapping relationship between the Block of the cache layer and the Block of the data layer, it is not necessary to separate the Block of the cache layer and the Block of the low-speed data layer one by one. Correspondingly, by splitting it into Log Append writing technology, the random small block write data from the client (or the front end) is converted into a sequential large block write request, and the sequential write effectively exerts the sequential large block write capability of the data layer.

The embodiments of the present invention will be described in detail below with reference to the above implementation steps.

Step S102: Receive a data processing request initiated by the client, wherein the data processing request at least carries: service data to be processed.

In this embodiment, the service write IO from the client/front end will be sent to the cache layer/cache disk/cache space through a data processing request. The data write data of the cache layer is faster, and the data cache can be regularized in time. All business write IO from the client/front end can be written to the cache space of the cache layer first.

In step S104, the service data is stored in multiple cache blocks, and the multiple cache blocks are identified to obtain multiple identified cache blocks.

In this embodiment, the cache space in the cache layer may include at least one high-speed storage disk, and the cache space is pre-divided into multiple cache blocks Block, which are managed with the granularity of the cache block Block, and the entire cache space may be divided into N Blocks . Optionally, the size of the divided cache blocks in this embodiment is divided according to the size of data written in each request, and the sizes of the cache blocks are inconsistent, for example, 4K, 8K, 16K, 128K, and so on.

A corresponding number of Block spaces are allocated for each newly written business data. After the data is written to these blocks, the cache blocks that have already written data can be marked to obtain the identified cache blocks. For example, these blocks are marked as dirty. Block; After completing the block marking, the writing success information can be returned to the client/front end.

As the client continues to write data, the blocks in the cache layer will gradually be consumed. When the consumption reaches a certain level, the data in the cache disk needs to be written back to the data layer to free up enough space to process new requests. . Step S106, in the case that the number of the multiple identification cache blocks is greater than the preset number threshold, combine the multiple identification cache blocks to obtain a write-back request.

That is, as cache blocks are continuously allocated in the cache space and data is written, so that the number of identified cache blocks increases, the consumption of cache space also increases, and then adjustment can be triggered according to the preset write-back policy (for example, When the number of cache blocks reaches a certain threshold, the cache duration reaches a preset duration threshold, and the usage of the cache space reaches a certain usage threshold), these identified cache blocks are pieced together to form a large write-back ( writeback) request, ready to write to the data space in the data layer. That is, with the consumption of blocks, there are more and more dirty blocks. At this time, we can piece together these blocks to form a large writeback request and prepare to write to the data space.

In this embodiment, for the data layer in the data storage system, the data space is divided into logical segmented data spaces-Segment in advance. The size of the segment is flexibly configured, generally greater than 1M, and each segment corresponds to a segment in the data space. physical space.

Optionally, the step of combining multiple identified cache blocks to obtain a write-back request includes: obtaining a cache interval to which each identified cache block belongs, and determining an interval start identifier of the cache interval; and combining the cache intervals based on the interval start identifier , get a writeback request.

In the cache space, since the size of the data written each time is inconsistent and the number of data blocks divided each time is inconsistent, it is necessary to set multiple cache intervals that identify the cache block for one or more data processing requests, which is formed as The size of the write-back request is also inconsistent, and the request from the write-back is not necessarily continuous. It contains several cache intervals, marked as A(off,len), B(off,len), C(off,len), etc. , merge these intervals - Merge. For example, for the cache interval A in the SSD, its size is 4K, starting from 0K-4k, starting from the 1M in the cache space, then the interval to which the cache interval A belongs is the 1M+0K to 1M+4K, and the cache interval A belongs The starting identifier of the interval is 1M; for the cache interval B in the SSD, its size is 4K, starting from 8k-12K, starting from the 2M in the cache space, then the interval to which the cache interval B belongs is 2M+8k to 2M+ 12K, the interval start identifier of the cache interval B is 2M+8k; for the cache interval C in the SSD, its size is 16K, from 16k-32K, starting from the 3M in the cache space, the interval to which the cache interval C belongs From 3M+16k to 3M+32K, the interval start identifier of the buffer interval C is 3M+16k.

In this embodiment, the selection of the cache interval during the write-back process is flexibly configured, and the selection can also be made according to a certain characteristic, so as to reduce the cost of dynamic mapping and persistence of data in the subsequent mixed media. For example, the present application can select the corresponding cache space according to the characteristics of persistence, so that only a small amount of data needs to be updated for subsequent persistence.

Step S108, write the write-back request into each segmented data space in the data layer, and determine the data dynamic mapping relationship pointed to by the identification cache block to the segmented data space.

Alternatively, after writing the write-back request into each segmented data space in the data layer, the method further includes: acquiring interval information of the cache interval and pre-stored business data; preprocessing the business data , obtain business processing data and preprocessing information; determine business processing data as segmented data, and determine preprocessing information as segmented header information; combine segmented header information and segmented data into segmented record information; Record information is appended to the segmented data space.

The data that can be written back at a time can be very large, and each cache interval is shaped as a segment record information-SegmentEntry, the information and data of these cache intervals are recorded in the Entry, which is called segment header information-EntryHeader and segment data, record data info - EntryData. The data can be processed (checksum or compression, etc.) as needed, and these additional information can be easily added to the EntryHeader, and then written to the above-mentioned segmented data space-Segment in the form of additional writing.

Optionally, the step of determining the data dynamic mapping relationship that identifies the cache block to the segmented data space includes: determining the mapping start position based on the interval start identifier of the cache interval to which the identifier cache block belongs and the interval size of the cache interval; Obtain the segment start bit of the segment data space in the overall data space in the data layer; based on the segment start bit, determine the segment space start bit in the segment data space of the cache interval that identifies the cache block and the The size of the segment space; establish a dynamic mapping relationship of data that the mapping start bit points to the start bit of the segment space.

The dynamic data mapping of mixed media is the core idea of the whole algorithm. By continuously updating the mapping, the writing of large blocks of sequential data space can be realized. The data dynamic mapping relationship can be called metadata mapping, which needs to be persisted. Since each cache space needs to form a mapping, the mapping persistence process may bring additional overhead.

Although the data dynamic mapping created each time is small, at least one logical block of the disk needs to be aligned for persistence, and only updating one mapping at a time is very wasteful of the disk. In this embodiment, the overhead of the mapping information part can be reduced by batch updating.

During the write-back process, it is necessary to record the mapping relationship between the cache space and the segmented data space, for example, record the mapping from the cache space A(off,len) to the segment space SA(off,len), which is called the data dynamic mapping of mixed media .

In this embodiment, for a resource pool composed of several disks, the write-back process can fill data so that the written-back data satisfies a certain condition, and under this condition, certain physical characteristics of the data disk (for example, data If the space adopts EC as the copy strategy, the data can be filled to align the EC stripe; for example, the SSD write behavior can be optimized by aligning the erase block size of the data pool SSD).

As an optional embodiment of the present invention, after determining the data dynamic mapping relationship from the identification cache block to the segmented data space, the method further includes: in the case of generating a new write-back request, assigning a valid segment to the new write-back request Data space; writes new writeback requests to the allocated, valid segmented data space.

Optionally, the present application further includes: in the case of generating a new write-back request, judging whether the cache interval used by the new write-back request is reused; in the case that the cache interval used by the new write-back request is reused , to reclaim the segmented data space pointed to in the historical process by the reused cache interval, and to invalidate the reclaimed segmented data space.

When an overwrite is generated and a new write-back request is formed, the same cache space A(off,len) will correspond to a different segment space SB(off,len), and the old space SA(off,len) will be If the mark is invalid and needs to be garbage collected, in this application, the bitmap can be used to mark the valid/invalid space of the segmented data space, and the bitmap can reduce invalid data movement in the subsequent garbage collection stage.

Optionally, the step of reclaiming the segmented data space pointed to by the reused cache interval in the historical process includes: analyzing the segmented data when there are multiple segmented data spaces to be reclaimed. The space usage rate and the internal storage data volume of the space; according to the sorting order of the space usage rate and the internal storage data volume, the multiple segmented data spaces to be reclaimed are reclaimed respectively.

In this embodiment, the garbage collection algorithm is started according to the level of the used capacity of the data disk, and data space with a high amount of garbage can be preferentially recycled when the capacity usage rate is low.

Alternatively, in the low-peak stage of the business, more stringent measures can be taken to perform garbage collection in advance or intensify efforts to release space in advance.

The space reclamation process is the process of reclaiming invalid segmented data space-Segment space. During the reclamation process, the valid data space is read according to the segment bitmap content, and then a new segment is allocated and then written to the new data space by additional writing. , and then modify the data dynamic mapping of the mixed media so that the cache space points to the new data space. After the modification is completed, the old data space is reclaimed for subsequent data write-back.

In this embodiment, the cache disk caches hot data, and writes the colder data back to the data disk, which takes advantage of the high performance and long life of the cache disk; It has the advantages of poor writing ability but good large-block sequential writing ability; it is very convenient to fill some data in the process of additional writing, including reprocessing the data (such as checksum and compression, etc.). The reprocessing process is meaningful for some characteristics of the data disk. major.

The present application will be described below with reference to another optional embodiment.

Embodiment 2

This embodiment provides a data storage device, which is applied to a cache layer in a data storage system. The cache space corresponding to the cache layer includes a plurality of pre-divided cache blocks. The data storage device provides a plurality of implementation units. Each implementation unit corresponds to each implementation step in the foregoing first embodiment.

FIG. 2 is a schematic diagram of an optional data storage device according to an embodiment of the present invention. As shown in FIG. 2, the device may include: a receiving unit 21, a storage unit 23, a combining unit 25, and a writing unit 27, wherein,

The receiving unit 21 is configured to receive a data processing request initiated by a client, wherein the data processing request at least carries: business data to be processed;

The storage unit 23 is used to store the business data in multiple cache blocks, and identify multiple cache blocks to obtain multiple identified cache blocks;

The combining unit 25 is configured to combine multiple identification cache blocks to obtain a write-back request when the number of multiple identification cache blocks is greater than a preset number threshold;

The writing unit 27 is configured to write the write-back request into each segmented data space in the data layer, and determine the data dynamic mapping relationship pointed to the segmented data space by the identification cache block.

The above data storage device can receive a data processing request initiated by a client through the receiving unit 21, wherein the data processing request at least carries: the business data to be processed, and the business data is stored in a plurality of cache blocks by the storage unit 23, and Identify multiple cache blocks to obtain multiple identified cache blocks, and combine multiple identified cache blocks through the combining unit 25 when the number of multiple identified cache blocks is greater than the preset number threshold to obtain a write-back request. The writing unit 27 writes the write-back request into each segmented data space in the data layer, and determines the data dynamic mapping relationship pointed to by the identification cache block to the segmented data space. In this embodiment, the random small-block write data from the front end can be converted into sequential large-block write-back requests through the data re-warping technology, so that different write-back requests can be handled, data can be quickly sorted, and data processing efficiency can be improved , so as to solve the technical problem that the random write-back request cannot be effectively processed when data is written back from the cache layer to the data layer.

Optionally, the combining unit includes: a first acquiring module, used for acquiring the cache interval to which each identified cache block belongs, and determining the interval start identifier of the cache interval; a first merging module, used for combining based on the interval start identifier. Cache interval, get write-back request.

Optionally, the data storage device further includes: a second obtaining module, configured to obtain interval information of the cache interval and pre-stored service data after the write-back request is written into each segmented data space in the data layer; The preprocessing module is used for preprocessing the business data to obtain the business processing data and preprocessing information; the first determining module is used for determining the business processing data as segmented data and the preprocessing information as segmented header information The second merging module is used to combine the segment header information and the segment data into segment record information; the first write module is used to additionally write the segment record information into the segment data space.

Optionally, the writing unit includes: a first determining module for determining a mapping start bit based on an interval start identifier identifying the buffer interval to which the cache block belongs and the interval size of the buffer interval; a third acquiring module for acquiring The segment start bit of the segment data space in the overall data space in the data layer; based on the segment start bit, determine the segment space start bit and the segment value in the segment data space that identify the cache interval to which the cache block belongs. The size of the segment space; the first establishment module is used to establish a data dynamic mapping relationship in which the mapping start bit points to the start bit of the segment space.

Optionally, the data storage device further includes: a first allocation module, configured to generate a new write-back request in the case of a new write-back request after determining the data dynamic mapping relationship that identifies the cache block to the segmented data space. Allocate a valid segmented data space; the second writing module is used to write a new write-back request into the allocated valid segmented data space.

Optionally, the data storage device further includes: a first judging module for judging whether the cache interval used by the new write-back request is reused when a new write-back request is generated; In the case that the cache area used by the write request is reused, the segment data space pointed to by the reused cache area in the historical process is reclaimed, and the reclaimed segment data space is marked invalid.

Optionally, the recycling module includes: an analysis submodule for analyzing the space usage rate and internal storage data volume of the segmented data space when there are multiple segmented data spaces to be recycled; a recycling submodule for According to the sorting order of the space usage rate and the amount of internal storage data, the multiple segmented data spaces to be reclaimed are reclaimed respectively.

The above-mentioned data storage device can also include a processor and a memory, and the above-mentioned receiving unit 21, storage unit 23, combining unit 25, writing unit 27, etc. are all stored in the memory as program units, and the processor executes the above-mentioned stored in the memory. program unit to achieve the corresponding function.

The above-mentioned processor includes a kernel, and the corresponding program unit is called from the memory by the kernel. The kernel can set one or more, and write the write-back request into each segmented data space in the data layer by adjusting the kernel parameters, and determine the data dynamic mapping relationship from the identification cache block to the segmented data space.

The above-mentioned memory may include non-persistent memory in computer readable medium, random access memory (RAM) and/or non-volatile memory, such as read only memory (ROM) or flash memory (flash RAM), the memory includes at least a memory chip.

According to another aspect of the embodiments of the present invention, there is also provided an electronic device, comprising: a processor; and a memory for storing executable instructions of the processor; wherein the processor is configured to execute the above-mentioned execution by executing the executable instructions The data storage method for any item.

According to another aspect of the embodiments of the present invention, a computer-readable storage medium is also provided, where the computer-readable storage medium includes a stored computer program, wherein when the computer program runs, the device where the computer-readable storage medium is located is controlled to execute any of the above An item's data storage method.

The present application also provides a computer program product, which, when executed on a data processing device, is suitable for executing a program initialized with the following method steps: receiving a data processing request initiated by a client, wherein the data processing request carries at least: Business data to be processed; store business data in multiple cache blocks, and identify multiple cache blocks to obtain multiple identified cache blocks; when the number of multiple identified cache blocks is greater than the preset number threshold, combine A plurality of identification cache blocks are obtained to obtain a write-back request; the write-back request is written into each segmented data space in the data layer, and the data dynamic mapping relationship between the identification cache block and the segmented data space is determined.

FIG. 3 is a block diagram of a hardware structure of an electronic device (or mobile device) of a data storage method according to an embodiment of the present invention. As shown in FIG. 3, the electronic device may include one or more processors 102 (illustrated by 102a, 102b, . A processing device such as a device FPGA), a memory 104 for storing data. In addition, may also include: display, input/output interface (I/O interface), universal serial bus (USB) port (may be included as one of the ports of the I/O interface), network interface, keyboard , power supply and/or camera. Those skilled in the art can understand that the structure shown in FIG. 3 is only a schematic diagram, which does not limit the structure of the above electronic device. For example, the electronic device may also include more or fewer components than shown in FIG. 3 , or have a different configuration than that shown in FIG. 3 .

The above-mentioned serial numbers of the embodiments of the present invention are only for description, and do not represent the advantages or disadvantages of the embodiments.

In the above-mentioned embodiments of the present invention, the description of each embodiment has its own emphasis. For parts that are not described in detail in a certain embodiment, reference may be made to related descriptions of other embodiments.

In the several embodiments provided in this application, it should be understood that the disclosed technical content can be implemented in other ways. The device embodiments described above are only illustrative, for example, the division of the units may be a logical function division, and there may be other division methods in actual implementation, for example, multiple units or components may be combined or Integration into another system, or some features can be ignored, or not implemented. On the other hand, the shown or discussed mutual coupling or direct coupling or communication connection may be through some interfaces, indirect coupling or communication connection of units or modules, and may be in electrical or other forms.

The units described as separate components may or may not be physically separated, and components shown as units may or may not be physical units, that is, may be located in one place, or may be distributed to multiple units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution in this embodiment.

In addition, each functional unit in each embodiment of the present invention may be integrated into one processing unit, or each unit may exist physically alone, or two or more units may be integrated into one unit. The above-mentioned integrated units may be implemented in the form of hardware, or may be implemented in the form of software functional units.

The integrated unit, if implemented in the form of a software functional unit and sold or used as an independent product, may be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention is essentially or the part that contributes to the prior art, or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium , including several instructions for causing a computer device (which may be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention. The aforementioned storage medium includes: U disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), mobile hard disk, magnetic disk or optical disk and other media that can store program codes .

The above are only the preferred embodiments of the present invention. It should be pointed out that for those skilled in the art, without departing from the principles of the present invention, several improvements and modifications can be made. It should be regarded as the protection scope of the present invention.

Claims (10)

1. a data storage method, is characterized in that, is applied to the cache layer in the data storage system, and the cache space corresponding to the cache layer includes a plurality of pre-divided cache blocks, including: Receive a data processing request initiated by a client, wherein the data processing request at least carries: business data to be processed; storing the business data in multiple cache blocks, and identifying the multiple cache blocks to obtain multiple identified cache blocks; In the case that the number of the multiple identification cache blocks is greater than the preset number threshold, combining the multiple identification cache blocks to obtain a write-back request; The write-back request is written into each segmented data space in the data layer, and a data dynamic mapping relationship pointed to the segmented data space by the identified cache block is determined. 2. The method according to claim 1, wherein the step of combining the multiple identification cache blocks to obtain a write-back request comprises: Obtain the cache section to which each of the identified cache blocks belongs, and determine the section start identifier of the cache section; Based on the interval start identifier, the cache interval is merged to obtain the write-back request. 3. The method according to claim 2, wherein after writing the write-back request into each segmented data space in the data layer, the method further comprises: Obtain the interval information of the cache interval and the pre-stored service data; Preprocessing the business data to obtain business processing data and preprocessing information; Determining the business processing data as segmented data, and determining the preprocessing information as segmented header information; combining the segment header information and the segment data into segment record information; The segment record information is additionally written into the segment data space. 4. The method according to claim 2, wherein the step of determining the data dynamic mapping relationship pointed to the segmented data space by the identification cache block comprises: Determine the mapping start bit based on the interval start identifier of the buffer interval to which the identifier cache block belongs and the interval size of the buffer interval; Obtain the segment start bit of the segment data space in the overall data space in the data layer; Based on the segment start bit, determine the segment space start bit and the segment space size in the segment data space of the cache interval to which the identifier cache block belongs; A data dynamic mapping relationship in which the mapping start bit points to the segment space start bit is established. 5. The method according to claim 2, characterized in that, after determining the data dynamic mapping relationship pointed to by the identification cache block to the segmented data space, the method further comprises: In the case of generating a new write-back request, allocate a valid segmented data space for the new write-back request; The new write-back request is written to the allocated valid segment data space. 6. The method of claim 5, further comprising: In the case of generating a new write-back request, determine whether the cache interval used by the new write-back request is reused; In the case that the cache interval used by the new write-back request is reused, reclaim the segmented data space pointed to by the reused cache interval in the historical process, and reclaim the reclaimed Fragmented data space for invalid marking. 7. The method according to claim 6, wherein the step of reclaiming the segmented data space pointed to in the historical process by the repeatedly used cache interval comprises: In the case that there are multiple segmented data spaces to be reclaimed, analyzing the space usage rate and internal storage data volume of the segmented data spaces; According to the sorting order of the space usage rate and the amount of the internal storage data, reclamation processing is performed on the plurality of segmented data spaces to be reclaimed respectively. 8. A data storage device, characterized in that it is applied to a cache layer in a data storage system, and the cache space corresponding to the cache layer includes a plurality of pre-divided cache blocks, including: a receiving unit, configured to receive a data processing request initiated by a client, wherein the data processing request at least carries: service data to be processed; a storage unit, configured to store the service data in multiple cache blocks, and identify the multiple cache blocks to obtain multiple identified cache blocks; a combining unit, configured to combine the multiple identification cache blocks to obtain a write-back request when the number of the multiple identification cache blocks is greater than a preset number threshold; A writing unit, configured to write the write-back request into each segmented data space in the data layer, and determine a data dynamic mapping relationship from the identification cache block to the segmented data space. 9. An electronic device, characterized in that, comprising: processor; and a memory for storing executable instructions for the processor; Wherein, the processor is configured to perform the data storage method of any one of claims 1 to 7 by executing the executable instructions. 10. A computer-readable storage medium, characterized in that the computer-readable storage medium comprises a stored computer program, wherein, when the computer program runs, the device where the computer-readable storage medium is located is controlled to execute claim 1 The data storage method described in any one of to 7.

Images