TW201935243A - SSD, distributed data storage system and method for leveraging key-value storage - Google Patents

Abstract

A solid state drive (SSD) includes: a plurality of data blocks; a plurality of flash memory channels and a plurality of channels for accessing the plurality of data blocks; and a solid state drive controller for the plurality of data The block's block size is configured. The data file is stored in the solid state drive together with one or more key-value pairs, and each of the one or more key-value pairs has a block identifier as a key and has a block Data as values. The size of the data file is equal to the block size or a multiple of the block size.

Description

System and method for using key-value storage in a distributed file system to efficiently store data and metadata

This disclosure generally relates to a key-value storage device, and more particularly, to a system and method for efficiently storing data and metadata using key-value storage in a distributed file system.

In traditional data storage nodes, an existing file system located on the data storage node is usually used to store the key-value mapping (for example, a block identifier (ID) to data content mapping). This is because the underlying storage device does not natively support the key-value interface required by the data storage node. Therefore, an additional software layer (usually a file system) is required to provide a key-value interface. Adding a file system introduces memory overhead and processor overhead.

The file system residing between the data storage node and the actual data storage device forces the data storage device to cause additional inefficiencies (for example, overprovisioning and higher write amplification) and is required in a device environment with limited resources More central processing unit (CPU) loops to perform tasks such as garbage collection.

According to an embodiment, a solid-state drive (SSD) includes: a plurality of data blocks; a plurality of flash memory channels and a plurality of paths for accessing the plurality of data blocks; and a solid state drive The driver controller configures a block size of the plurality of data blocks. A data file is stored in the solid state drive together with one or more key-value pairs, and each key-value pair has a block identifier as a key and has block data as a value. The size of the data file is equal to the block size or a multiple of the block size.

According to another embodiment, a distributed data storage system includes: a client; a name node including a first key-value (KV) solid state drive (SSD); and a data node including a second key-value solid state A drive, wherein the second key-value solid-state drive includes a plurality of data blocks, a plurality of flash memory channels and a plurality of channels for accessing the plurality of data blocks, and a method for configuring the plurality of data blocks Block size solid state drive controller. The client sends an archive creation request including an archive identifier for storing a data archive to the name node, and sends an allocation command to the name node to allocate the plurality of data blocks related to the data archive Associated one or more data blocks. The name node returns a block identifier of the one or more data blocks and a data node identifier of the data node assigned to store the one or more data blocks to the client. The client sends a block storage command to the data node to store the one or more data blocks. The second key-value solid-state drive stores the one or more data blocks as key-value pairs, and at least one key-value pair in the key-value pairs has the block identifier as a key and Has block data as the value. The size of the data file is equal to the block size or a multiple of the block size.

According to yet another embodiment, a method includes sending a create archive request from a client to a name node, wherein the create archive request includes an archive identifier for storing a data archive; and storing the archive identifier as a key-value pair In the first key-value (KV) solid state drive (SSD) of the name node, wherein the file identifier is stored as a key in the key-value, and the value associated with the key is empty Sending an allocation command from the client to the name node to allocate one or more data blocks associated with the data file; assigning a block identifier to the one at the name node At least one of the plurality of data blocks and assigns a data node to store the one or more data blocks; returning the block identifier and the data node's A data node identifier; sending a write block request from the client to the data node, wherein the write block request includes the block identifier and content; and the one or more data areas As the key - the value of the second key stored in the node data - values of the solid state drive. The second key-value solid-state drive of the data node includes one or more data blocks having a block size. At least one of the key-value pairs has a block identifier as a key and has block data as a value. The size of the data file is equal to the block size or a multiple of the block size.

The above and other preferred features including various novel details and event combinations of the embodiments will now be explained in more detail with reference to the accompanying drawings, and the above and other preferred features are pointed out in the claims. It should be understood that the specific systems and methods described herein are shown by way of illustration only and are not limiting. As those skilled in the art would understand, the principles and features described herein may be used in various embodiments without departing from the scope of the present disclosure.

Each of the features and teaching contents disclosed herein can be used alone or in combination with other features and teaching contents to provide a system and system for efficiently storing data and metadata using key-value storage in a distributed file system and method. Representative examples are explained in more detail with reference to the drawings, which use individual features and many of these additional features and teaching contents individually and in combination. This detailed description is only intended to teach a person skilled in the art further details for practicing various aspects of the teachings and is not intended to limit the scope of the claims. Therefore, the combination of features disclosed above in this detailed description may not necessarily be necessary to practice the teaching content in the broadest sense, but rather, it is merely taught to specifically illustrate a representative example of the teaching content.

In the following description, specific terms are set forth for explanatory purposes only to provide a thorough understanding of the present disclosure. However, it will be apparent to those skilled in the art that these specific details are not necessary to practice the teachings of the present disclosure.

Some parts of the detailed description in this article are presented by algorithms and symbolic notations of operations on data bits in computer memory. These algorithmic descriptions and notations are used by those skilled in the data processing arts to effectively communicate the substance of their work to other skilled persons in the art. An algorithm is here and generally considered a self-consistent sequence of steps that can achieve the desired result. The steps require physical manipulation of physical quantities. Usually (though not necessarily), these quantities are in the form of electrical or magnetic signals that can be stored, transmitted, combined, compared, and otherwise manipulated. It has proven convenient, sometimes for general reasons, to refer to these signals as bits, values, components, symbols, characters, terms, digits, etc.

It should be borne in mind, however, that all of these terms and all similar terms are associated with the appropriate physical quantities and are merely convenient labels applied to these quantities. Unless it is clearly stated otherwise by reading the following discussion, it should be understood that throughout this description, for example, "processing", "computing", "calculating", "determining" ) "," Displaying "and other terms refer to the actions and processes of computer systems or similar electronic computing devices, which are manipulated in the registers and memory of computer systems and are represented as physical (Electronic) quantities of data and transforming said data into other data similarly expressed as physical quantities in computer system memory or registers or other such information storage devices, information transmission devices or information display devices.

In addition, the features of the representative examples and the appended claims may be combined in ways that are not specifically and explicitly enumerated to provide additional useful embodiments of the present teachings. It should also be explicitly noted that for the purpose of the original disclosure and for the purpose of limiting the claimed subject matter, all value ranges or indications of groups of entities are used to disclose every possible intermediate value or intermediate entity. It should also be explicitly noted that the dimensions and shapes of the components shown in the figures are designed to help understand how to practice the teachings, and are not intended to limit the dimensions and shapes shown in the examples.

This disclosure describes a system and method for solving inefficiencies caused by distributed file systems (eg, Hadoop Distributed File System (HDFS)). The system and method eliminate the need for a key-value file system that uses the file name as a block identifier and uses the data content of the file (or part of the data content of the file) as the value by storing the data directly in the data storage device. . This type of data storage device that can store data directly in key-value pairs is referred to herein as a key-value (KV) solid state drive (SSD), referred to as a KV SSD. KV SSD supports key-value storage where block identifiers are used as keys and data is used as values. The system and method can provide an efficient and simplified key-value data storage system including a KV SSD that can store data as key / value pairs directly in one or more KV SSDs. Therefore, this key-value data storage system can consume less energy and resources while providing a faster, simpler, and scalable data storage solution.

According to one embodiment, a KV SSD can be implemented in combination with a data storage node to implement a file system that stores data in key-value pairs. By using one or more KV SSDs that can directly store key-value data, the key-value data storage system can eliminate the need for a file system in the data storage node. The data storage node can pass down the information about the behavior of the data storage node to the KV SSD to optimize the internal data structure and resources of the KV SSD, so as to adapt to the workload specified by the data storage node. In addition, the in-memory mapping table can be offloaded to the KV SSD to provide permanent data using the key-value interface between the data storage node and the KV SSD.

According to one embodiment, the key-value data storage system can support an existing file system (eg, HDFS). Specifically, a file system optimized for large data blocks can benefit from this key-value data storage system. For example, the metadata (or hash table) of a KV SSD is allocated in large block sizes (for example, 10 million bytes (MB) to 100 MB).

Decentralized file systems (eg, HDFS) have immutable data blocks that do not need to be moved back and forth because the values of the keys do not change, thereby enabling data stored in KV SSDs And the internal write amplification factor (WAF) of the metadata is minimized. In addition, the key-value data storage system can reduce the CPU overhead associated with updating hash table values.

This key-value data storage system has a simplified flash translation layer (FTL) while improving performance and resource utilization. KV SSDs reduce metadata overhead when used with immutable decentralized storage systems (eg, HDFS). This is because in this distributed file system, the contents of the keys cannot be changed, so KV SSDs that store key / value pairs no longer need to mark the values as old and point the keys to the flash memory media in the KV SSD New value content. In other words, KV SSDs do not need to support overwrite. In addition, for decentralized file systems (for example, HDFS), blocks have a fixed size, so KV SSDs do not need to handle dynamic size values, which makes the management of value locations easier. For example, when all blocks have a fixed size, a data structure based on direct indexing can be used. With these simplifications in a decentralized file system, the FTL management of key / value tuples can be simplified.

A distributed file system may keep metadata in the memory of a single data storage node, thereby limiting the scalability of the metadata. This key-value data storage system eliminates memory limitations in managing metadata that may be required by other distributed file systems.

This key-value data storage system can achieve high throughput that is not latency-oriented. Because HDFS has such a large block size and a data set that may exceed the memory capacity, page cache memory may not be able to significantly improve data storage and management performance. Therefore, even if KV-SSD does not support page cache, KV SSD will not degrade the performance of KV-capable data nodes.

HDFS's centralized cache memory management feature provides a mechanism to explicitly inform data nodes to off-heap cache blocks. This feature can be implemented in KV-enabled data nodes by allowing KV-capable data nodes to still benefit from memory-based cache memory without having to make strategic decisions to determine which blocks to cache. .

This key-value data storage system can achieve high parallelism in read operations and write operations. Since the latency of each data block is less important, and HDFS shows a high degree of parallelism by reading / writing a large number of data blocks, there is no need to separate commands (for example, read commands, write commands). And send the command to many channels on the KV SSD. Each data block can be directly written to and read from a channel or a die / die of the KV SSD to provide throughput by taking advantage of the inherent parallelism. This also simplifies the FTL of the KV SSD and the complexity of the lookup process. The parallelism can also be applied to multiple channels or wafers / die according to the context of the erase block of the KV SSD. This, in turn, can minimize or eliminate SSD over-provisioning by matching the SSD block / page size to the block size of a decentralized file system (eg, HDFS). Therefore, this key-value data storage system can use a device sent to the KV SSD and aligned to a fixed block size to erase the block to increase the throughput. This is because the aligned erase block and the data size are fast. Less synchronization will be required between flash memory channels. By offloading the metadata map to the KV SSD, the memory in the metadata node is no longer a bottleneck in a distributed storage system.

FIG. 1A illustrates a block diagram of a prior art decentralized data storage system. The client application 101 stores the file 105 in the data node 121 in the distributed data storage system 100A. The file 105 includes two data blocks (ie, Ω and Σ). After writing the archive 105 to the data node 121, the client 101 stores the metadata associated with the archive 105 in the block map 115 of the name node (or metadata node) 111. In the context of HDFS, the name node 111 is referred to as the master and the data node 121 is referred to as the slave. The host device can store metadata of the entire HDFS file in the HDFS directory structure. Although HDFS is mentioned in some of the examples described below, it should be understood that other file systems optimized for large amounts of data can be used without departing from the scope of this disclosure.

The name node 111 maintains a block map 115 that includes mapping information between the file 105 including the block identifier and the data node 121 storing the blocks included in the file 105. In this example, the blocks Ω and Σ have a block identifier “11” and a block identifier “99”, respectively. When the client 101 needs to access the file 105 (or the data blocks Ω and Σ), the client 101 communicates with the name node 111 to identify the association with the file 105 based on the associated information stored in the block map 115 And the data node 121 (DN 1) of the file 105 (or data block) to be accessed.

The data node 121 includes a local file system (for example, the ext4 file system of Linux) with a directory structure to store each block as a file in the directory. The file name may be a unique block identifier ("11" or "99") corresponding to the corresponding block of the file 105, and the content of the file is block data. Since the blocks need to be stored as files, the data node 121 needs additional software layers (for example, the local ext4 file system), additional memory (for example, the Linux directory entry cache), and key-values. CPU processing (for example, Portable Operating System Interface (POSIX) and file system-specific command processing) converted to the file system by block, and file system overhead includes metadata Management. The controller logic of the SSD 140 needs to perform additional processing to maintain the consistency of the block map 115. HDFS block size may not be aligned with the internal SSD page / block map. This can increase the internal WAF of the SSD and the space for over-provisioning , Which results in more frequent garbage collection and an increase in total cost of ownership (TCO).

FIG. 1B illustrates a block diagram of an exemplary decentralized data storage system including a key-value storage device according to one embodiment. The client application 101 stores the file 105 in a data node 221 in the distributed data storage system 100B. For example, the distributed data storage system 100B is HDFS. The data node 221 including the key-value SSD 150 can directly store the data blocks Ω and Σ. In contrast to the data node 121 including the traditional SSD 140, the data node 221 does not require a local file system (eg, ext4) because the data blocks of the file 105 are stored directly as key-value pairs in the KV SSD 150.

The KV SSD 150 provides an interface for the data node 221 to communicate with the client application 101. The interface enables direct storage of data blocks as key-value pairs. Therefore, the data node 221 does not need a local file system layer, and thus may not cause the memory overhead and CPU overhead of the traditional data node 121 shown in FIG. 1A.

According to one embodiment, the distributed data storage system 100B enables the client application 101 and the data node 221 to exchange information. This process is called the registration process or the configuration process. During the registration process, the data node 221 can inform the client application 101 that the data node 221 has one or more KV SSDs that can store data blocks as key-value pairs. After completing the registration process, the client application 101 knows that it can issue a KV SSD-specific input / output (I / O) device command to the KV SSD 150 included in the data node 221 (eg, / dev / kvssd1, Where kvssd1 is the id of the data node 221). This will simplify the input / output path between the client application 101 and the data node 221. The data node 221 may issue a "put" command to store each data block as a key-value pair, instead of relying on a local file system to create a data block and write the data block to a file. The process of reading the stored key-value pairs in the KV SSD 150 is similar; the data node 221 can issue a "get" command directly to the KV SSD 150 to retrieve the data block associated with the key rather than through a file System interface to retrieve data blocks. The deletion process can be followed by a similar process.

During the registration process, KV SSD 150 may be provided with information about the behavior of the distributed data storage system 100B. Based on the behavior of the distributed data storage system 100B, the flash memory conversion layer (FTL) of the KV SSD 150 can be specifically optimized for the distributed data storage system 100B.

The KV SSD 150's SSD controller can write and read data in the form of strips between different memory chips (eg, non-chip) to speed up write operations and read operations. The decentralized data storage system 100B (for example, HDFS) can send many input / output requests in parallel, and can tolerate long delays as long as the throughput is high. This parallelism can reduce latency by adding complexity to the SSD controller. According to one embodiment, the FTL of the KV SSD 150 can be optimized to read and write large blocks to a single channel based on the information of the distributed data storage system 100B. In this case, the FTL of the KV SSD 150 does not cause the data to be striped between multiple chips through multiple channels, but can perform concurrent read and write operations in parallel to achieve high performance. Of delivery.

Distributed file systems (eg, HDFS) can be optimized for data-centric and data-intensive applications that frequently read stored data. In this case, the data read operation is much more frequent than the data write operation. Some of these distributed file systems provide write-once semantics and use large block sizes. In contrast, KV SSD 150 supports dynamic block sizes and frequent updates to data blocks.

According to another embodiment, the KV SSD 150 may be optimized so as not to cause garbage collection that might otherwise be required for the case of using an internal file system (eg, SSD 140 shown in FIG. 1A). Garbage collection is the process of moving valid pages from blocks that include outdated pages, so the blocks can be erased and rewritten. Garbage collection is a costly process that can cause write amplification, input / output uncertainty (I / O indeterminism), and wear leveling of drives. Once optimized, the FTL of the KV SSD 150 can use the same granularity for write and erase operations. When a block is deleted, the block can be immediately marked for erasure, thereby eliminating the need for garbage collection. KV SSD 150's optimized FTL improves performance and durability while eliminating garbage collection and simplifying FTL.

According to one embodiment, the KV SSD 150 supports dynamic page and block sizes. For example, the KV SSD 150 can adjust the block size of the blocks to be stored therein based on the HDFS configuration. For example, during the configuration, the distributed data storage system 100B may inform the KV SSD 150 that only aligned fixed-size write operations will be issued to the KV SSD 150, and the KV SSD 150 will correspondingly block its size Configure it. As another option, the KV SSD 150 may disclose its erase block size (or possible erase block size) and requires a distributed data storage system 100B to configure the block size of the KV SSD 150 accordingly. In either case, the block sizes in the KV SSD 150 and the distributed data storage system 100B can be configured relative to each other.

According to one embodiment, the distributed data storage system 100B may configure the KV SSD 150 to allow or disallow block updates. For example, the distributed data storage system 100B may pass additional parameters (referred to herein as update flags) to the KV SSD 150. By using the update flag, the SSD controller of the KV SSD 150 can configure itself to provide additional flash memory blocks and threads to handle garbage collection associated with the block update request received from the client application 101. By not allowing block updates (for example, update to flag = false), the distributed data storage system 100B can achieve a large increase in throughput due to the parallelism between different flash memory channels or die. . When using each new write with a new key, the KV SSD 150 does not need to perform synchronization between channels or die to verify that the written block is a rewrite. In this case, the data node 221 may set the block update flag to false.

FIG. 2A illustrates a system configuration of an exemplary SSD. Referring to FIG. 1A, the distributed data storage system 100A configures the SSD 140 by installing its local file system (for example, ext4 of Linux) as “/ mnt / fs”. Files stored in the SSD 140 can be accessed by an installed file system of the SSD 140.

FIG. 2B illustrates an exemplary system configuration of a key-value SSD according to one embodiment. Referring to FIG. 1B, the distributed data storage system 100B configures the KV SSD 150 based on the information of the KV SSD 150 received during the registration process. For example, the distributed data storage system 100B may configure the storage type of the KV SSD 150 as a key-value SSD (KV SSD) and set the input / output path of the KV SSD 150 to “dev / kvssd”. KV SSD 150 may also be configured to set the block update flag to false, the block size to 64 MB, and the alignment flag to true.

When the KV SSD 150 is configured to disable cross-channel input / output operations by setting the erase block size of the KV SSD 150 and the data size of the distributed data storage system 100B to be equal and aligned, the distributed data storage system 100B performs lock-less I / O operations between all channels or die in the KV SSD 150. For example, KV SSD 150 uses a simple hash function (for example, address mod 10) to determine the channel to which the input / output should be routed among all possible channels. In this case, all input / output operations for a given address will be consistently routed to the same flash memory channel. In the case where a channel is performed by a continuous processing unit, all input / output operations routed to this channel are ordered without any cross-channel locking. Therefore, the distributed data storage system 100B can achieve full parallelism between the input / output threads without synchronization.

This KV SSD can achieve parallelism based on how the KV SSD treats erased blocks as garbage collection units. FIG. 3 illustrates an exemplary SSD channel and die architecture to achieve channel-level parallelism according to one embodiment. KV SSDs can utilize channel-level parallelism to improve the input / output performance of KV SSDs. In this example, the KV SSD has N channels and M channels, where N and M are integers equal to or greater than 1. The data size of a distributed file system (for example, HDFS) is set to be equal to the block size of the KV SSD or a multiple of the block size. The block size of a KV SSD is determined by the product of the size of the erase unit in the chip (or the bare die) and the number of internal channels of the KV SSD, that is, the block size = the size of the erase unit * the number of channels. The data can be striped between individual wafers in the same channel. For example, the KV SSD has an erase unit size of 6 MB and 8 channels, and the data size of the distributed file system is set to 48 MB to fit in the data group of the KV SSD. Garbage collection does not occur in channel-level parallelism, because deleting 48 MB of data will cause 8 full erase blocks to be reset in the same channel. However, channel-level parallelism does not take full advantage of the potential parallelism that the multiple channels can provide.

FIG. 4 illustrates an exemplary SSD channel and die architecture to achieve way-level parallelism according to one embodiment. KV SSDs can utilize channel-level parallelism to improve the input / output performance of KV SSDs. In this case, the data size of the distributed file system (for example, HDFS) is set to be equal to the block size of the KV SSD or a multiple of the block size. The block size of the KV SSD is determined by the product of the size of the erase unit and the number of channels, that is, the block size = the size of the erase unit * the number of channels. The garbage collection unit and data size are multiplications of the block size. The data can be striped between individual wafers in the same path. Data striping can be performed between all channels, allowing full use of channel parallelism.

FIG. 5 illustrates an exemplary SSD channel and die architecture to achieve die / wafer level parallelism according to one embodiment. In this example, the number of SSD channels is N, and the number of die is M, which is greater than N. Die / wafer level parallelism provides the highest parallelism between channels and chips in a KV SSD. In this case, the block size is equal to the erase unit, and garbage collection is performed in the erase unit. The data size is aligned with multiples of the erase unit. Die / wafer level parallelism is similar to consistent hashing with virtual nodes. In this case, each virtual node corresponds to an erase block unit, and the physical node corresponds to a channel.

Decentralized file systems (for example, HDFS) maintain metadata to manage the location of the data. For example, each file maintains a list of all the blocks that make up the file. Duplicate decentralized storage systems maintain separate maps that list the locations of all nodes that store a given block (or file). In some distributed data storage systems, these mapping tables are kept in the memory of a single node, which limits the scalability of the distributed data storage system. For example, when the metadata nodes storing these mapping tables do not have sufficient memory to store additional mapping data, no blocks or files may be added. A file system can be used on top of the data storage device to store these mappings, but the file system introduces additional overhead.

The KV SSD can store data directly in key-value pairs by persistently maintaining file-to-block list mapping and block-to-node list mapping without the need for a local file system. Therefore, the node responsible for storing metadata is not limited by its memory capacity and does not cause the overhead caused by having an additional file system. Since these mapping information are stored directly on the KV SSD, these mapping information can be stored in a single mapping table by means of file indexing. This enables a single lookup in the KV SSD to retrieve all data blocks. A single mapping table makes the metadata more scalable (only one mapping table) and more efficient (one lookup).

The process of reading files stored in a KV SSD is similar to the process using a regular hash map or similar data structure. The data structure can be a library linked directly to the KV SSD. For example, a client application issues an archive retrieval operation to read an archive using an archive ID. The metadata node returns a block list of the file in the form of a binary large object (blob), and the metadata node can be mapped to the format in which the block list was originally written. The block list also contains a node list in which each block in the block list is stored. The metadata node may then pass the list of blocks and associated nodes back to the client application to issue a read of the block. In this scheme, the metadata node still needs to store the mapping table in its memory for each lookup to pass the list back to the client application; however, the metadata node does not need to keep all mapping information In its memory. For example, the cache memory of recently read files can provide a compromise between scalability and efficiency.

FIG. 6A illustrates an exemplary mapping scheme of an exemplary SSD (eg, the SSD 140 illustrated in FIGS. 1A and 2A). The edit log refers to a log of all metadata operations performed on the file and mapping table. These edit logs are persisted to disk. This is necessary because the file and block mapping table is only in memory; if the name node crashes, the name node will rebuild the file and block mapping table in memory by reading the edit log from the memory. FIG. 6B illustrates an exemplary mapping scheme of a KV SSD (eg, KV SSD 150 shown in FIGS. 1B and 2B) according to one embodiment. The SSD 140 stores a file mapping table and a block mapping table in its memory. Using this mapping information, the client application can retrieve blocks of data associated with the file. Meanwhile, the KV SSD 150 stores a file mapping table including a single mapping of a block list and a node list.

7, 8, and 9 are diagrams illustrating exemplary input / output processes of creating and writing archives, reading archives, and deleting archives in a decentralized archive system (eg, HDFS). In the description of each figure, differences and advantages compared to prior art processes will be discussed.

FIG. 7 illustrates an exemplary process of creating and storing archives in a KV SSD of a distributed archive system according to one embodiment. The distributed file system includes a client 710, a name node (or metadata node) 720 including a KV SSD (kv1) 730, and a data node 740 including a KV SSD (kv2) 750. To create a new file, the client 710 sends a request 761 (createFile (fileID)) with a file ID (fileID) to the name node 720. Name node 720 registers the file ID internally by sending a key-value store command 762 (kv.store (fileID, “”)) and stores the file ID as a value-less key of the key-value pair in KV SSD 730. The KV SSD 730 responds by sending a completion message 763 back to the name node 720, and then, the name node 720 responds to the client 710 with a completion message 764. After the name node 720 responds to the client 710, the client 710 allocates a block allocation command 765 (allocateBlock (fileID)) for the file. In response, the name node 720 assigns a block ID (blockID) and a data node (eg, the data node 740) to store the block and sends a response 766 back to the client 710. The name node 720 may assign a monotonically increasing ID as the block ID. The client 710 sends a block write request 767 (writeBlock (blockID, content)) with the data content of the block to the data node 740 using the block ID. In response to the block write request 767, the data node 740 sends a key-value store command 768 (kv.store (blockID, content)) with the incoming parameters (block ID and content) to the KV SSD 750. In a traditional data node with a conventional SSD, a write operation to the data node will request a write to the file system and then a write to the basic data storage medium of the SSD. After storing the block, the KV SSD 750 responds to the data node 740 with a completion message 768, and the data node 740 responds to the client 710 with a completion message 770. The client 710 then sends a commit write command 771 (commit (Write (fileID, blockID)) to the name node 720 to submit the block ID and data node tuple to the associated archive. The name node 720 sends an additional to the KV SSD 730 Command 772 (kv.append (fileID, blockID + dataNode)). During the append process, the append command 772 is a single direct operation on the KV SSD 730, rather than two as in a traditional decentralized storage system Storage operations for separate mappings (ie, file-block mapping and block-data node mapping).

FIG. 8 illustrates an exemplary process of reading an archive stored in a KV SSD of a distributed file system according to one embodiment. To read the file stored in the data node 740, the client 710 sends a read file request 861 (openFile (fileID)) with a file ID (fileID) to the name node 720. By using the archive ID, the name node 720 sends a retrieval command (kv.retrieve (fileID)) to the KV SSD 730, and the KV SSD 730 returns the mapping information 863, which maps the block to the data node associated with the archive ID. The name node 720 forwards the mapping information 864 to the client 710. By using the block ID included in the block-data node mapping information, the client 710 sends a block read command 865 (readBlock (blockID)) to the data node 740. The data node 740 sends a block retrieval command 866 (kv.retrieve (blockID)) to the KV SSD 750 to retrieve the block content. The KV SSD 750 sends the content 867 of the requested block back to the data node 740, and the data node 740 forwards the retrieved block content 868 back to the client 710. The basic difference between this and the traditional read operation is that the name node 720 issues a single direct KV SSD read operation to retrieve the block-data node mapping, instead of searching the archive for the block list and area in the memory hash table. List of block-to-data nodes. In addition, the data node 740 directly sends a request to retrieve data to the KV SSD, thereby bypassing any storage software intermediary software (such as a file system).

FIG. 9 illustrates an exemplary process of deleting archives in a KV SSD of a distributed archive system according to one embodiment. The client 710 sends a file delete command 961 (deleteFile (fileID)) with a file ID to the name node 720. The name node 720 sends a key-value retrieval command 962 (kv.retrieve (fileID)) to the KV SSD 730 to retrieve the mapping information of the file, and the KV SSD 730 returns the mapping information 963. The mapping information 963 maps the block to the file ID. Linked data nodes. The name node 720 may temporarily cache the block ID mapping for subsequent asynchronous deletion processes of the associated block. The name node 720 sends a key-value delete command 964 (kv.delete (fileID)) to the KV SSD 730, and the KV SSD 730 sends a completion message 965 to the name node 720 after deleting the archive ID and the associated mapping. When retrieving the mapping information associated with the archive ID, the name node 720 retrieves the block-data node tuple to find the archive to be deleted from the KV SSD 730. This process is different from traditional decentralized storage systems that will involve looking up hash tables in multiple memories. Instead, the name node 720 deletes this file-based key from the KV SSD 730 containing the block-data node mapping. The name node 720 returns control to the client 710, and the name node 720 asynchronously sends a block delete command to the data node 740 to delete the corresponding block from the KV SSD 750. It should be noted that the file deletion process shown in FIG. 9 is based on the assumption that the block size of the distributed file system is equal to or can be divided into the erase block size of the KV SSD 750. This minimizes the overhead of moving from memory operations to KV SSD-based operations.

According to an embodiment, a solid state drive (SSD) includes: a plurality of data blocks; a plurality of flash memory channels and a plurality of channels for accessing the plurality of data blocks; and an SSD controller, The block size of the multiple data blocks is configured. The data file is stored in the SSD together with one or more key-value pairs, and each key-value pair has a block identifier as a key and has block data as a value. The size of the data file is equal to the block size or a multiple of the block size.

The SSD may be used in a distributed file system including a Hydrup Distributed File System (HDFS).

The SSD controller may also be configured to enable or disable block updates based on a block update flag.

The SSD controller may be further configured to align the data file with the plurality of data blocks based on an alignment flag.

The block size may be determined based on the erasure unit of the SSD multiplied by the number of flash memory channels.

The block size may be determined based on the erase unit of the SSD multiplied by the number of channels.

The block size may be equal to an erasing unit of the SSD.

The SSD may store a file mapping table, and the file mapping table includes a first mapping of the file to one or more data blocks associated with the file among the plurality of data blocks and the one or A second mapping of at least one of the plurality of data blocks to a data node including the SSD.

According to another embodiment, a distributed data storage system includes: a client; a name node including a first key-value (KV) solid state drive (SSD); and a data node including a second KV SSD, wherein the second The KV SSD includes multiple data blocks, multiple flash memory channels and multiple channels for accessing the multiple data blocks, and an SSD for configuring the block size of the multiple data blocks. Controller. The client sends an archive creation request including an archive identifier for storing a data archive to the name node, and sends an allocation command to the name node to allocate the plurality of data blocks related to the data archive Associated one or more data blocks. The name node returns a block identifier of the one or more data blocks and a data node identifier of the data node assigned to store the one or more data blocks to the client. The client sends a block storage command to the data node to store the one or more data blocks. The second KV SSD stores the one or more data blocks as key-value pairs, and at least one key-value pair has the block identifier as a key and has block data as a value. The size of the data file is equal to the block size or a multiple of the block size.

The decentralized data storage system may adopt a Haidupu Decentralized File System (HDFS).

The second KV SSD may store a file mapping table, the file mapping table including a first mapping of the data file to one or more data blocks associated with the file and the one or more data areas A second mapping of at least one of the blocks to a data node.

According to yet another embodiment, a method includes: sending a create archive request from a client to a name node, wherein the create archive request includes a file identifier for storing a data file; and storing the file identifier as a key-value pair In the first key-value (KV) solid state drive (SSD) of the name node, wherein the file identifier is stored as a key in the key-value, and the value associated with the key is empty Sending an allocation command from the client to the name node to allocate one or more data blocks associated with the data file; assigning a block identifier to the one at the name node At least one of the plurality of data blocks and assigns a data node to store the one or more data blocks; returning the block identifier and the data node's A data node identifier; sending a write block request from the client to the data node, wherein the write block request includes the block identifier and content; and the one or more data areas As the key - the value of the second KV SSD data stored in the node. The second KV SSD of the data node includes one or more data blocks having a block size. At least one key-value pair has a block identifier as a key and has block data as a value. The size of the data file is equal to the block size or a multiple of the block size.

The client, the name node, and the data node may be nodes in a Haidupu distributed file system (HDFS).

The method may further include setting a block update flag to enable or disable the block update.

The method may further include setting an alignment flag to align the data file with the plurality of data blocks of the second KV SSD of the data node.

The method may further include: sending a write commit command from the client to the name node, the write commit command including the file identifier and the block identifier; and attaching a single direct operation to The file identifier, the block identifier, and the data node are attached to the name node.

The method may further include: sending a read archive request from the client to the name node to read the profile; returning to the client the one or more chunks of data that are associated with the profile The block identifier and the data node identifier of at least one data block associated with a data file; sending a block read command from the client to the data node to retrieve the data node stored in the data node The one or more data blocks in the second KV SSD; and returning the block data identified by the block identifier from the data node to the client.

The method may further include: sending an archive delete command from the client to the name node, the archive delete command including the archive identifier; returning the one or more data blocks to the client Send the key identifier of the block identifier and the data node identifier of at least one data block associated with the data file from the name node to the first KV SSD of the name node A command, the key-value delete command including the file identifier of the data file, and sending a block delete command from the name node to the data node, the block delete command including the one or more A list of data blocks; and deleting the one or more data blocks stored in the second KV SSD of the data node.

The second KV SSD can store a file mapping table, and the file mapping table includes a first mapping of the file to one or more data blocks associated with the file and the one or more data blocks At least one of the second mapping to the data node.

The foregoing exemplary embodiments have been set forth above to illustrate various embodiments of systems and methods that implement systems and methods for providing efficient storage of data and metadata using key-value storage in a distributed file system. Those of ordinary skill in the art will recognize various modifications to the disclosed exemplary embodiments and differences from the disclosed exemplary embodiments. The subject matter which is intended to fall within the scope of this disclosure is set forth in the following claims.

100A, 100B‧‧‧ Distributed Data Storage System

101‧‧‧Client Application / Client

105‧‧‧File

111, 720‧‧‧ name nodes

115‧‧‧block mapping

121, 221, 740‧‧‧ data nodes

140‧‧‧SSD

150‧‧‧key-value SSD / KV SSD

710‧‧‧Client

730‧‧‧KV SSD (kv1) / KV SSD

750‧‧‧KV SSD (kv2) / KV SSD

761‧‧‧request

762, 768‧‧‧ key-value storage commands

763, 764, 769, 770, 773, 774, 965‧‧‧ Completion

765‧‧‧ Allocation order

766‧‧‧ Response

767‧‧‧block write request

771‧‧‧ Submit write command

772‧‧‧ additional order

861‧‧‧Read File Request

862‧‧‧Search Order

863, 864, 963‧‧‧ mapping information

865‧‧‧block read command

866‧‧‧block search command

867‧‧‧Contents

868‧‧‧block content

961‧‧‧File delete command

962‧‧‧Key-Value Search Command

964‧‧‧key-value delete command

966, 967‧‧‧ block delete order

968, 969‧‧‧ return

970‧‧‧Asynchronous message

The drawings included as part of this specification show the presently preferred embodiments, and are used to explain and teach the text along with the general description given above and the detailed description of the preferred embodiments given below. The principle.

FIG. 1A illustrates a block diagram of a prior art decentralized data storage system.

FIG. 1B illustrates a block diagram of an exemplary decentralized data storage system including a key-value storage device according to one embodiment.

FIG. 2A illustrates a system configuration of an exemplary SSD.

FIG. 2B illustrates an exemplary system configuration of a key-value SSD according to one embodiment.

FIG. 3 illustrates an exemplary SSD channel and die architecture to achieve channel-level parallelism according to one embodiment.

FIG. 4 illustrates an exemplary SSD channel and die architecture to achieve way-level parallelism according to one embodiment.

FIG. 5 illustrates an exemplary SSD channel and die architecture for implementing die / chip-level parallelism according to one embodiment.

FIG. 6A illustrates an exemplary mapping scheme for an exemplary SSD.

FIG. 6B illustrates an exemplary mapping scheme for a KV SSD according to one embodiment.

FIG. 7 illustrates an exemplary process of creating and storing archives in a KV SSD of a distributed archive system according to one embodiment.

FIG. 8 illustrates an exemplary process of reading an archive stored in a KV SSD of a distributed file system according to one embodiment.

FIG. 9 illustrates an exemplary process of deleting archives in a KV SSD of a distributed archive system according to one embodiment.

The figures are not necessarily drawn to scale, and for illustrative purposes, elements with similar structures or functions are generally represented by the same reference numbers in all figures. The figures are only intended to facilitate the description of the various embodiments described herein. The figures do not illustrate every aspect of the teachings disclosed herein and do not limit the scope of the claims.

Claims (19)

A solid-state drive includes: Multiple data blocks; Multiple flash memory channels and multiple channels for accessing the multiple data blocks; and A solid state drive controller configured to configure a block size of the plurality of data blocks; Wherein the data file is stored in the solid state drive together with one or more key-value pairs, and at least one key-value pair has a block identifier as a key and has block data as a value, and The size of the data file is equal to the block size or a multiple of the block size. The solid-state drive according to item 1 of the scope of patent application, wherein the solid-state drive is used in a distributed file system including a Haidupu distributed file system. The solid-state drive according to item 1 of the patent application scope, wherein the solid-state drive controller is further configured to enable or disable the block update based on a block update flag. The solid-state drive according to item 1 of the patent application scope, wherein the solid-state drive controller is further configured to align the data file with the plurality of data blocks based on an alignment flag. The solid-state drive according to item 1 of the scope of patent application, wherein the block size is determined based on the erasing unit of the solid-state drive multiplied by the number of flash memory channels. The solid state drive according to item 1 of the scope of patent application, wherein the block size is determined based on the erasing unit of the solid state drive multiplied by the number of channels. The solid-state drive according to item 1 of the patent application scope, wherein the block size is equal to an erasing unit of the solid-state drive. The solid-state drive according to item 1 of the scope of patent application, wherein the solid-state drive stores a file mapping table, and the file mapping table includes one or more of the files associated with the files among the plurality of data blocks. A first mapping of a plurality of data blocks and a second mapping of at least one of the one or more data blocks to a data node including the solid state drive. A distributed data storage system includes: Client computer; Name nodes, including first key-value solid-state drives; and A data node includes a second key-value solid-state drive, wherein the second key-value solid-state drive includes multiple data blocks, multiple flash memory channels for accessing the multiple data blocks, and multiple data blocks. Channels, and a solid-state drive controller for configuring a block size of the plurality of data blocks, The client sends an archive creation request including an archive identifier for storing a data archive to the name node, and sends an allocation command to the name node to allocate the plurality of data blocks to the data archive. One or more data blocks associated with it, Wherein the name node returns a block identifier of the one or more data blocks and a data node identifier of the data node assigned to store the one or more data blocks to the client, The client sends a block storage command to the data node to store the one or more data blocks. Wherein the second key-value solid-state drive stores the one or more data blocks as key-value pairs, and at least one key-value pair has the block identifier as a key and has block data as a value, And The size of the data file is equal to the block size or a multiple of the block size. The decentralized data storage system according to item 9 of the scope of patent application, wherein the decentralized data storage system adopts a Haidupu decentralized file system. The distributed data storage system according to item 9 of the scope of patent application, wherein the second key-value solid-state drive stores a file mapping table, and the file mapping table includes the data file to a file associated with the file. A first mapping of one or more data blocks and a second mapping of at least one of the one or more data blocks to a data node. A method including: Sending a create archive request from the client to the name node, wherein the create archive request includes an archive identifier for storing a data archive; Storing the file identifier as a key-value pair in a first key-value solid state drive of the name node, wherein the file identifier is stored as a key in the key-value and is the same as the key The associated value is empty; Sending an allocation command from the client to the name node to allocate one or more data blocks associated with the data file; Assigning a block identifier to at least one of the one or more data blocks at the name node and assigning a data node to store the one or more data blocks; Returning the block identifier and the data node identifier of the data node from the name node to the client; Sending a write block request from the client to the data node, wherein the write block request includes the block identifier and content; and Storing the one or more data blocks as key-value pairs in a second key-value solid-state drive of the data node, Wherein the second key-value solid-state drive of the data node includes one or more data blocks having a block size, At least one of the key-value pairs has a block identifier as a key and has block data as a value, and The size of the data file is equal to the block size or a multiple of the block size. The method according to item 12 of the scope of patent application, wherein the client, the name node, and the data node are nodes in a Haidupu distributed file system. The method according to item 12 of the patent application scope further includes: setting a block update flag to enable or disable block update. The method according to item 12 of the patent application scope further comprises: setting an alignment flag to align the data file with a plurality of data blocks of the second key-value solid state drive of the data node. The method described in item 12 of the patent application scope further includes: Sending a write commit command from the client to the name node, the write commit command including the file identifier and the block identifier; and Attach a single direct operation to attach the file identifier, the block identifier, and the data node to the name node. The method described in claim 16 of the scope of patent application further includes: Sending a read archive request from the client to the name node to read the profile; Returning to the client the block identifier and the data node identifier of at least one data block associated with the data file in the one or more data blocks; Sending a block read command from the client to the data node to retrieve the one or more data blocks stored in the second key-value solid-state drive of the data node; and Returning the block data identified by the block identifier to the client from the data node. The method described in claim 17 of the scope of patent application further includes: Sending an archive delete command from the client to the name node, the archive delete command including the archive identifier; Returning to the client the block identifier and the data node identifier of at least one data block associated with the data file in the one or more data blocks; Sending a key-value delete command from the name node to the first key-value solid-state drive of the name node, the key-value delete command including the file identifier of the data file; Sending a block delete command from the name node to the data node, the block delete command including a list of the one or more data blocks; and Deleting the one or more data blocks stored in the second key-value solid-state drive of the data node. The method of claim 12, wherein the second key-value solid-state drive stores a file mapping table, and the file mapping table includes the file to one or more data areas associated with the file. A first mapping of blocks and a second mapping of at least one of the one or more data blocks to the data node.