[go: up one dir, main page]

CN120832077A - Method of implementing RAID using file system SSD - Google Patents

Method of implementing RAID using file system SSD

Info

Publication number
CN120832077A
CN120832077A CN202410494437.9A CN202410494437A CN120832077A CN 120832077 A CN120832077 A CN 120832077A CN 202410494437 A CN202410494437 A CN 202410494437A CN 120832077 A CN120832077 A CN 120832077A
Authority
CN
China
Prior art keywords
file system
storage device
custom command
file
command
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN202410494437.9A
Other languages
Chinese (zh)
Inventor
王祎磊
程雪
周维君
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Beijing Starblaze Technology Co ltd
Original Assignee
Beijing Starblaze Technology Co ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Beijing Starblaze Technology Co ltd filed Critical Beijing Starblaze Technology Co ltd
Priority to CN202410494437.9A priority Critical patent/CN120832077A/en
Publication of CN120832077A publication Critical patent/CN120832077A/en
Pending legal-status Critical Current

Links

Classifications

    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
    • G06F3/0601—Interfaces specially adapted for storage systems
    • G06F3/0668—Interfaces specially adapted for storage systems adopting a particular infrastructure
    • G06F3/0671—In-line storage system
    • G06F3/0683—Plurality of storage devices
    • G06F3/0689—Disk arrays, e.g. RAID, JBOD
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
    • G06F3/0601—Interfaces specially adapted for storage systems
    • G06F3/0628—Interfaces specially adapted for storage systems making use of a particular technique
    • G06F3/0629—Configuration or reconfiguration of storage systems
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
    • G06F3/0601—Interfaces specially adapted for storage systems
    • G06F3/0628—Interfaces specially adapted for storage systems making use of a particular technique
    • G06F3/0638—Organizing or formatting or addressing of data
    • G06F3/0643—Management of files
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
    • G06F3/0601—Interfaces specially adapted for storage systems
    • G06F3/0628—Interfaces specially adapted for storage systems making use of a particular technique
    • G06F3/0655—Vertical data movement, i.e. input-output transfer; data movement between one or more hosts and one or more storage devices
    • G06F3/0656—Data buffering arrangements

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Human Computer Interaction (AREA)
  • Physics & Mathematics (AREA)
  • General Engineering & Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Information Retrieval, Db Structures And Fs Structures Therefor (AREA)

Abstract

The application provides a method for realizing RAID by using a file system SSD, wherein the method applied to electronic equipment without file system functions comprises the steps of responding to a first file system basic operation indicating file writing operation, generating a first custom command and a second custom command which bear the file system basic operation and follow an NVMe protocol, sending the first custom command to a first storage device and sending the second custom command to a second storage device to access a first logical address space managed by the first storage device and a second logical address space managed by the second storage device, and bearing the first file system basic operation by the first custom command and the second custom command. The application can realize RAID1 function aiming at the equipment without file system function accessing storage equipment, reduce the hardware burden when realizing RAID1 function and improve the equipment performance.

Description

Method for realizing RAID (redundant array of independent disks) by using SSD (solid State disk) of file system
Technical Field
The application relates to the technical field of storage equipment, in particular to a method for realizing RAID by using a file system SSD.
Background
FIG. 1A illustrates a block diagram of a storage device. The storage device 102 is coupled to a host for providing storage capability for the host. The host and the storage device 102 may be coupled by a variety of means including, but not limited to, interfacing the host with the solid state storage device 102 via a variety of storage protocols such as SATA (SERIAL ADVANCED Technology Attachment ), SCSI (Small Computer system interface), SAS (SERIAL ATTACHED SCSI ), IDE (INTEGRATED DRIVE Electronics), USB (Universal Serial Bus ), PCIE (PERIPHERAL COMPONENT INTERCONNECT EXPRESS, PCIE, peripheral component interconnect), NVMe (NVM Express, high speed nonvolatile storage), ethernet, fibre channel, wireless communication network, etc. The host may be an information processing device capable of communicating with the storage device in the manner described above, such as a personal computer, tablet, server, portable computer, network switch, router, cellular telephone, personal digital assistant, or the like. The memory device 102 includes an interface 103, a control unit 104, one or more NVM chips 105, and a DRAM (Dynamic Random Access Memory ) 110.
NAND flash memory, phase change memory, feRAM (Ferroelectric RAM, ferroelectric memory), MRAM (Magnetic Random Access Memory, magnetoresistive memory), RRAM (RESISTIVE RANDOM ACCESS MEMORY, resistive memory), XPoint memory, and the like are common NVM.
The interface 103 may be adapted to exchange data with a host by means of, for example SATA, IDE, USB, PCIE, NVMe, SAS, ethernet, fibre channel, etc.
The control unit 104 is used to control data transfer among the interface 103, NVM chip 105, and DRAM 110, and also for memory management, host logical address to flash physical address mapping, erase balancing, bad block management, etc. The control component 104 can be implemented in a variety of ways, such as software, hardware, firmware, or a combination thereof, for example, the control component 104 can be in the form of an FPGA (Field-programmable gate array), an ASIC (Application SPECIFIC INTEGRATED Circuit), or a combination thereof. The control component 104 may also include a processor or controller in which software is executed to manipulate the hardware of the control component 104 to process IO (Input/Output) commands. Control unit 104 may also be coupled to DRAM 110 and may access data of DRAM 110. FTL tables and/or cached data of IO commands may be stored in the DRAM.
The control section 104 includes a flash interface controller (or referred to as a media interface controller, a flash channel controller) that is coupled to the NVM chip 105 and issues commands to the NVM chip 105 in a manner conforming to an interface protocol of the NVM chip 105 to operate the NVM chip 105 and receive a command execution result output from the NVM chip 105. Known NVM chip interface protocols include "Toggle", "ONFI", and the like.
Fig. 1B shows a detailed block diagram of the control components of the storage device.
The host accesses the storage device with read/write commands following the NVMe protocol. The control component generates one or more media interface commands based on NVMe commands from the host and provides the media interface commands to the media interface controller. The media interface controller generates storage media access commands (e.g., program commands, read commands, erase commands) that follow the interface protocol of the NVM chip according to the media interface commands. The control unit also keeps track of all media interface commands generated from one NVMe command being executed and indicates the processing result of the NVMe command to the host.
Referring to fig. 1B, the control part includes, for example, a host interface, a host command processing unit, a storage command processing unit, a media interface controller, and a storage media management unit. The host interface acquires the NVMe command provided by the host and generates a storage command to be provided to the storage command processing unit. The storage commands, for example, access the same size of storage space, e.g., 4KB. The data unit of the data accessed by the corresponding one of the storage commands recorded in the NVM chip is referred to as a data frame. The physical page records one or more frames of data. For example, if the physical page size is 17664 bytes and the data frame size is 4KB, then one physical page can store 4 data frames.
The storage medium management unit maintains a logical address to physical address translation for each storage command. For example, the storage medium management unit includes FTL tables. For a read command, the storage medium management unit outputs a physical address corresponding to a logical address accessed by the storage command, for a write command, the storage medium management unit allocates an available physical address to the storage medium management unit, and records a mapping relationship between the logical address accessed by the storage medium management unit and the allocated physical address.
The storage command processing unit operates the medium interface controller to issue a storage medium access command to the NVM chip according to the physical address provided by the storage medium management unit. For the sake of clarity, commands sent by the host to the storage device are referred to as NVMe commands, commands sent by the host command processing unit to the storage command processing unit are referred to as storage commands, commands sent by the storage command processing unit to the media interface controller are referred to as media interface commands, and commands sent by the media interface controller to the NVM chip are referred to as storage media access commands. The storage medium access command follows the interface protocol of the NVM chip.
RAID (Redundant Arrays of INDEPENDENT DISKS, redundant array of independent disks) is a type of multiple disk management technology, consisting of multiple independent disks, providing higher storage performance and data redundancy than a single disk. RAID mainly utilizes data stripe, mirror image and data checking technology to obtain high performance, reliability, fault tolerance and expansibility, and according to the strategy and architecture applying or combining the three technologies, RAID can be classified into different grades, such as RAID 0-RAID 5. Each level of RAID represents an implementation method and technique, and there is no high or low score between levels. When writing data to the disk, RAID1 writes the same data to two disks (a working disk and a mirror disk) indiscriminately, which is equivalent to making a redundant backup for the data. For RAID0, which combines multiple disks together to form one mass storage, each disk may store different data.
At present, RAID implementation modes comprise soft RAID and hard RAID, wherein no special control chip or I/O chip is arranged for the soft RAID, RAID functions are realized based on an operating system and a CPU (Central Processing Unit ), RAID control processing and I/O processing chips are arranged for the hard RAID, for example, RAID cards, RAID chips integrated on a mainboard and the like, and RAID functions are realized by the RAID cards or the RAID chips.
FIG. 2A is a schematic diagram illustrating a prior art implementation of RAID1 functions using soft RAID.
As shown in fig. 2A, the host/FPGA device is coupled to two storage devices (storage device 1 and storage device 2), where storage device 2 is a mirrored storage device of storage device 1. The host/FPGA device includes applications, file systems, soft RAIDs (such as operating system and CPU) and NVMe driver circuitry. The process of the host/FPGA equipment accessing the storage equipment to realize the RAID1 function by adopting a soft RAID mode comprises the following steps that an application program generates a file system basic operation according to user requirements, the file system determines a corresponding LBA according to the file system basic operation, the operation system and the CPU corresponding to the soft RAID copy the LBA to obtain two identical LBAs, an NVMe driving circuit generates two identical NVMe commands based on the two identical LBAs, the two identical NVMe commands are respectively sent to the storage equipment 1 and the storage equipment 2, and the storage equipment 2 is a mirror image storage equipment of the storage equipment 1.
FIG. 2B illustrates a prior art schematic diagram for implementing RAID1 functionality using hard RAID.
As shown in fig. 2B, the host/FPGA device includes an application, a file system, and an NVMe driving circuit. A hard RAID component (such as a special RAID control processing chip and an I/O processing chip corresponding to the hard RAID) is arranged on the periphery of the host/FPGA device, and the host/FPGA device and the storage device (the storage device 1 and the storage device 2) communicate through the hard RAID component. An application program in the host/FPGA equipment generates basic operation of a file system according to user requirements, and the file system determines a corresponding LBA according to the basic operation of the file system; the NVMe drive circuit generates an NVMe command based on the LBA and sends the NVMe command to the hard RAID component. The hard RAID component receives the NVMe command, analyzes the NVMe command to obtain an LBA, copies the LBA to obtain two identical LBAs, encapsulates the two LBAs to generate two new NVMe commands, and sends the two new NVMe commands to the storage device 1 and the storage device 2 respectively, wherein the storage device 2 is a mirror image storage device of the storage device 1.
Disclosure of Invention
Whereas prior art devices coupled to storage devices (such as host or FPGA devices) have file systems, they can be provided in block device fashion to access the storage devices based on file systems and by soft or hard RAID. In accessing storage devices in a block device manner to implement RAID1 functionality, two identical NVMe commands need to be provided, through which identical data is written to two storage devices (one being a primary storage device and the other being a backup storage device of the primary storage device). However, to generate two identical NVMe commands, whether a soft RAID or a hard RAID is used to implement the RAID1 function requires copying LBAs determined by the file system according to the file system operation. However, the soft RAID copy LBAs need to occupy CPU resources, and the hard RAID copy LBAs need to occupy hardware resources supporting hard RAID, which are limited, so that the copy LBAs increase CPU or hardware burden, and further affect the performance of the device. In addition, in some special scenarios, the host or FPGA device may not have a file system or file system function, and cannot access the storage device to implement the RAID1 function using the prior art. The application hopes to realize RAID1 function aiming at the host or FPGA equipment without file system and file system function accessing storage equipment, and reduce CPU or hardware burden when realizing RAID1 function and improve equipment performance in the process of realizing RAID1 function. In addition, the RAID technology comprises a plurality of implementation methods and technologies (such as RAID 0-RAID 5, etc.), and the scheme provided by the application is expected to be compatible with the plurality of implementation methods and technologies included in the RAID technology as far as possible.
In a first aspect, an embodiment of the present application provides a method for accessing a storage device, applied to an electronic device, where the electronic device couples a first storage device and a second storage device, and does not have a function of a file system, the method includes:
Generating a first custom command and a second custom command carrying the file system basic operation and conforming to an NVMe protocol in response to a first file system basic operation indicating a file writing operation;
Transmitting the first custom command to the first storage device, and transmitting the second custom command to the second storage device to access a first logical address space managed by the first storage device and a second logical address space managed by the second storage device;
Wherein the first custom command and the second custom command both bear the first file system basic operation.
Optionally, the method further comprises:
generating a third custom command which carries the first file system basic operation and follows an NVMe protocol in response to the first file system basic operation indicating the file writing operation;
generating a fourth custom command which carries the second file system basic operation and follows an NVMe protocol in response to the second file system basic operation indicating the file writing operation;
And sending the third custom command to the first storage device and the fourth custom command to the second storage device to access a first logical address space managed by the first storage device and a second logical address space managed by the second storage device.
Optionally, generating the first custom command based on the first file system basic operation and copying the first custom command to obtain the second custom command identical to the first custom command, or
And copying the first file system basic operation to obtain a third file system basic operation which is the same as the first file system basic operation, and generating the same first custom command and second custom command according to the first file system basic operation and the third file system basic operation.
Optionally, in response to the first and second file system base operations being the same file system base operation, the third and fourth custom commands carry the same file system base operation, or
And responding to the first file system basic operation and the second file system basic operation as different file system basic operations, wherein the third custom command and the fourth custom command bear different file system basic operations.
Optionally, in response to the first custom command and the second custom command carrying the same file system basic operation, writing the same file data into a first logical address space of the first storage device and a second logical address space of the second storage device based on the first custom command and the second custom command, respectively.
Optionally, in response to the third custom command and the fourth custom command carrying different file system basic operations indicating a file writing operation, different file data is written to the first logical address space of the first storage device and the second logical address space of the second storage device based on the third custom command and the fourth custom command, respectively.
The first storage device manages file data written into the first logical address space through a first file system, and the second storage device manages file data written into the second logical address space through a second file system.
Optionally, the first user-defined command and the second user-defined command carry the same file system basic operation indicating a file writing operation, and file data managed by a first file system of the first storage device is the same as file data managed by a second file system of the second storage device;
And responding to the third custom command and the fourth custom command to bear different file system basic operations indicating file writing operations, wherein file data managed by a first file system of the first storage device is different from file data managed by a second file system of the second storage device.
Optionally, in response to sending the first custom command to the first storage device, sending the second custom command to the second storage device, the second storage device is a mirrored storage device of the first storage device.
Optionally, in response to generating a fourth file system basic operation of the file reading operation, generating a fifth custom command which carries the fourth file system basic operation and follows an NVMe protocol, and sending the fifth custom command to the first storage device;
and responding to the failure of the first storage device, generating a sixth custom command which carries the basic operation of the fourth file system and conforms to an NVMe protocol based on the failure of the fifth custom command to read data, and sending the sixth custom command to the second storage device so as to read the file data corresponding to the basic operation of the fourth file system from the second storage device based on the sixth custom command.
Optionally, in response to the first storage device failing, the electronic device is further coupled to a third storage device;
copying the file data in the second logical address space of the second storage device and moving the copied file data to the third storage device.
Optionally, in response to a fifth file system basic operation indicating a file writing operation, generating the same seventh and eighth custom commands based on the fifth file system basic operation;
sending the seventh custom command to the second storage device and the eighth custom command to the third storage device to write file data indicated by the fifth file system basic operation to the second storage device and to the third storage device.
In a second aspect, an embodiment of the present application provides an electronic device, where the electronic device includes an application program, an NVMe driving circuit, and a RAID module;
the application program generates a first file system basic operation indicating a file writing operation;
The NVMe driving circuit is matched with the RAID module to generate a first custom command and a second custom command which bear the basic operation of the first file system and follow an NVMe protocol;
The NVMe drive circuit or the RAID module sends the first custom command to a first storage device and the second custom command to a second storage device to access a first logical address space of the first storage device and a second logical address space of the second storage device.
Optionally, the application program generates a first file system basic operation indicating a file writing operation, and generates a second file system basic operation indicating a file writing operation;
The NVMe driving circuit is matched with the RAID module, generates a third custom command which carries the basic operation of the first file system and follows an NVMe protocol, and generates a fourth custom command which carries the basic operation of the second file system and follows the NVMe protocol;
The NVMe drive circuit or the RAID module sends the third custom command to the first storage device and the fourth custom command to the second storage device to access a first logical address space of the first storage device and a second logical address space of the second storage device.
In a third aspect, an embodiment of the present application provides a data processing system, including an electronic device, a first storage device, and a second storage device, where the electronic device does not have a file system function;
The electronic device is coupled with the first storage device and the second storage device, generates a first custom command and a second custom command which bear file system basic operation and follow NVMe protocol based on first file system basic operation indicating file writing operation, sends the first custom command to the first storage device and sends the second custom command to the second storage device, and the first custom command and the second custom command bear the first file system basic operation;
The first storage device processes basic file system operation carried by the first custom command by using a first file system and obtains a processing result, and response information of the first custom command is generated and sent to the electronic device;
And the second storage device processes the basic file system operation carried by the second custom command by using a second file system, obtains a processing result, generates response information of the second custom command and sends the response information to the electronic device.
Optionally, the electronic device generates a third custom command carrying the first file system basic operation and following an NVMe protocol based on the first file system basic operation indicating the file writing operation, and generates a fourth custom command carrying the second file system basic operation and following the NVMe protocol based on the second file system basic operation indicating the file writing operation;
The electronic device sends the third custom command to the first storage device and the fourth custom command to the second storage device to access a first logical address space managed by the first storage device and a second logical address space managed by the second storage device;
the first storage device processes basic file system operation carried by the third custom command by using a first file system and obtains a processing result, and response information of the third custom command is generated and sent to the electronic device;
and the second storage device processes the file system basic operation carried by the fourth custom command by using a second file system and obtains a processing result, generates response information of the fourth custom command and sends the response information to the electronic device.
In a fourth aspect, an embodiment of the present application provides a method for data processing, including:
The electronic device generates a first custom command and a second custom command which bear the basic operation of the file system and follow the NVMe protocol based on the basic operation of the first file system, which indicates the file writing operation;
the electronic equipment sends the first custom command to first storage equipment and sends the second custom command to second storage equipment, wherein the first custom command and the second custom command both bear the basic operation of the first file system;
And the electronic equipment receives response information fed back by the first storage equipment for processing the basic operation of the file system carried by the first custom command, and receives response information fed back by the second storage equipment for processing the basic operation of the file system carried by the second custom command.
Optionally, the electronic device generates a third custom command carrying the first file system basic operation and following NVMe protocol based on the first file system basic operation indicating the file writing operation;
the electronic equipment generates a fourth custom command which carries the second file system basic operation and follows an NVMe protocol based on the second file system basic operation indicating the file writing operation;
The electronic device sends the third custom command to the first storage device and the fourth custom command to the second storage device to access a first logical address space managed by the first storage device and a second logical address space managed by the second storage device;
And the electronic equipment receives response information fed back by the first storage equipment for processing the basic operation of the file system carried by the third custom command, and receives response information fed back by the second storage equipment for processing the basic operation of the file system carried by the fourth custom command.
According to the embodiment of the application, the electronic device without the file system function generates two identical custom commands carrying basic operations of the same file system, and sends the two identical custom commands to the first storage device and the second storage device serving as a mirror image storage device of the first storage device respectively for backing up data in the storage devices, so that the two storage devices are accessed in a file form with high performance based on the two identical custom commands, and RAID1 function is realized.
Drawings
FIG. 1A shows a block diagram of a storage device;
FIG. 1B illustrates a detailed block diagram of the control components of the storage device;
FIG. 2A illustrates a soft RAID implementing RAID1 using replicated LBAs;
FIG. 2B illustrates a hard RAID implementing RAID1 with replicated LBAs;
FIG. 3A illustrates a format of a custom command provided by an embodiment of the present application;
FIG. 3B is a diagram illustrating a format of a custom command according to another embodiment of the present application;
FIG. 3C is a diagram illustrating a format of a custom command according to another embodiment of the present application;
FIG. 4A is a diagram illustrating a format of a custom command according to another embodiment of the present application;
FIG. 4B is a diagram illustrating a format of a custom command according to another embodiment of the present application;
FIG. 4C is a diagram illustrating a format of a custom command according to another embodiment of the present application;
FIG. 5 illustrates a block diagram of a storage device supporting basic operations of a file system provided by an embodiment of the present application;
FIG. 6 is a schematic diagram showing an FPGA device accessing a storage device through a custom command according to an embodiment of the present application;
FIG. 7 is a schematic diagram illustrating a host accessing a storage device through IO commands according to an embodiment of the present application;
FIG. 8 is a block diagram of a memory device according to yet another embodiment of the present application;
FIG. 9 is a block diagram of a memory device according to another embodiment of the present application;
FIG. 10 is a schematic diagram showing an electronic device accessing two storage devices to implement RAID1 functionality through a custom command according to an embodiment of the present application;
FIG. 11 is a schematic diagram showing an electronic device accessing two storage devices to implement RAID1 functionality through a custom command according to another embodiment of the present application;
FIG. 12 is a schematic diagram showing an electronic device accessing two storage devices to implement RAID0 functionality through two custom commands according to an embodiment of the present application;
FIG. 13 is a schematic diagram showing an electronic device accessing two storage devices through two custom commands to implement RAID0 functionality according to another embodiment of the present application.
Related art terminology
The basic operation of a file system, namely realizing basic operations of reading, writing, deleting, opening or creating files and the like on the files in the form of the file system;
custom command, which is a command which accords with the definition of a storage protocol in the form or format of the command and carries the basic operation of a file system;
A file system, namely an executable program for providing the function of the file system or a hardware unit for realizing the function of the file system;
File system data, which includes file system metadata and file data, wherein the file system metadata is used for describing hierarchical directory structure of the file system, file/directory attribute, storage position of the file data and the like;
File system path describing path of directory/file in directory structure layered in file system;
OP, indicating the operation type corresponding to the basic operation of the file system;
FILEOBJ indicating file parameters corresponding to basic operation of the file system;
PRP List, which is used for indexing a plurality of PRP items in a read operation or a write operation, wherein each PRP item is used for indexing a host memory or an FPGA device memory with a fixed size (such as 4K);
file system metadata describing logical addresses of file data managed by the file system in the LBA space;
inodes a data structure constituting file system metadata, the file system metadata including a plurality of inodes, each of which may have a specific size and be recorded in a logical address space of a storage device.
Detailed Description
The following description of the embodiments of the present application will be made clearly and fully with reference to the accompanying drawings, in which it is evident that the embodiments described are some, but not all embodiments of the application. All other embodiments, which can be made by those skilled in the art based on the embodiments of the application without making any inventive effort, are intended to be within the scope of the application.
In the embodiment of the application, the electronic device does not mount a file system, does not run a file system in a software form, does not include a hardware unit for providing a file system function, and includes a file system in a software or hardware form in the storage device, for example, the electronic device is a host or FPGA device. The electronic equipment without the file system generates two identical custom commands carrying basic operation of the same file system through the soft RAID or the hard RAID, and sends the two identical custom commands to two storage equipment respectively, wherein one of the two storage equipment is used as a working storage equipment, and the other is used as a mirror image storage equipment of the working storage equipment for backing up data in the working storage equipment, so that the two storage equipment can be accessed in a file form with high performance based on the two identical custom commands, and the function of RAID1 is realized.
The electronic device according to the embodiment of the application accesses the storage device by sending the custom command provided by the implementation of the application to the storage device. The custom command provided by the implementation of the application accords with the definition of a storage protocol (such as NVMe protocol) in the form or format of the command, carries basic operation of a file system, and is differentiated based on different command types (operation types of basic operation of the file system) and specific information indicated by command contents. The electronic device and the storage device interact through adopting the custom command (carrying the basic operation of the file system), so that the storage device processes the basic operation of the file system through processing the custom command, and the electronic device can access the storage device in a file system mode.
An electronic device according to an embodiment of the present application uses a logical address space provided by a storage device by accessing the storage device in the form of a file. After the electronic device writes file data to the storage device in the form of a file, file system metadata and file data, collectively referred to as file system data, are recorded in the LBA space of the storage device.
Among them, the file system basic operations include, for example, the following operations:
fopen (file_path_name) indicates opening or creating a file, where file_path_name is, for example, a string representing the file system path and file name of the operated file;
fwrite (file_path_name, data) indicates write file data, where file_path_name represents a file system path and a file name of an operated file, and data represents data to be written;
fread (file_path_name, buf) indicating read file data, wherein file_path_name represents a file system path and a file name of an operated file, and buf represents a memory buffer accommodating read data;
fdelect (file_path_name) indicates deletion of file data, where file_path_name represents a file system path and a file name of the operated file;
fset (file_path_name, attribute) indicates setting and modifying file attributes, wherein file_path_name represents a file system path and a file name of an operated file, attribute represents an attribute name to be modified, and value represents an attribute new value;
flist (file_path) indicates that all files and directories under a specified file system path are listed, file_path representing the specified file system path;
fsquery indicates a query of file system status, such as how many clusters there are in total for the current partition of the file system, cluster size, whether each cluster is used, etc.;
format indicates a formatted file system;
fmnt indicates loading and unloading the file system;
fdisk indicates partition creation and management operations such as creating a partition, deleting a partition, viewing a partition, and the like. The custom command provided according to the embodiment of the present application includes at least a field accommodating an OP and a field accommodating PRPList, and optionally a field accommodating FILEOBJ.
The custom command has a size specified by, for example, NVMe protocol, e.g., the custom command includes 16 DWORDs (each of which is 64 bits in size), different DWORDs indicate different contents, e.g., 10 th DWORD is OP, indicating an operation type of a basic operation of the file system, may indicate fopen, fwrite, fread, flist, etc., 11 th DWORD is FILEOBJ, indicating file parameters corresponding to the basic operation of the file system, including file_path_name, attribute, etc. (e.g., may include all parameters except data, buf), 12 th and 13 th DWORDs are PRPList, indicating a device memory space corresponding to data, buf.
Since the length of the string indicated by FILEOBJ is not constant and may exceed the size of the custom command, the size of the data/buf may be changed along with the data size of the read-write file, and often exceeds the size of the custom command, so that an improved technical scheme is required to enable the custom command with a fixed size and format to carry various file parameters, data/buf, of which the length/form/number of the file system basic operation and the file system basic operation are changed.
FIG. 3A illustrates the format of a custom command provided by one embodiment of the present application.
By way of example, referring to FIG. 3A, a custom command carrying the basic operation of a file system includes a field to hold an OP, a field to hold FILEOBJ, and a field to hold PRPList. The field holding the OP is, for example, 10 bits in size, the field holding FILEOBJ is, for example, 1 DWORD in size, and the field holding PRPList is, for example, 1 or several DWORD in size. As an example, PRPList is carried in the custom command in the embodiment of the present application using the same format as that of the NVMe IO command carrying PRPList in the NVMe protocol.
Since 1 DWORD only includes 64 bits, the field holding FILEOBJ is, for example, 1 DWORD size, and when holding FILEOBJ, FILEOBJ cannot carry more than 64 bits of information. For example, when custom command indication fileopen (file_path_name), FILEOBJ includes file_path_name, and the size of file_path_name cannot exceed 64 bits. For a file such as fileopen ("/a.o"), its file path length is small, so that it can be carried in the custom command format shown in FIG. 3A. For a file such as fileopen ("/root/bin/foo.txt"), its file path length is well over 64 bits and thus cannot be carried with the custom command format shown in fig. 3A.
In the embodiment of fig. 3A, PRPList includes, for example, 1 or 2 PRP entries, which are fully accommodated in the custom command. When PRPList includes, for example, more PRP entries, these PRP entries are too numerous to accommodate by the custom command and thus cannot be in the custom command format illustrated in fig. 3A.
FIG. 3B illustrates a format of a custom command provided by yet another embodiment of the present application.
By way of example, referring to FIG. 3B, a custom command carrying file system basic operations includes a field to hold an OP, a field to hold FILEOBJ ptr, and a field to hold PRPList. The field holding the OP is, for example, 10 bits in size, the field holding FILEOBJ ptr is, for example, 1 DWORD in size, and the field holding PRPList is, for example, 1 or several DWORD in size. As an example, PRPList is carried in the custom command in the embodiment of the present application using the same format as that of the NVMe IO command carrying PRPList in the NVMe protocol.
The custom command shown in FIG. 3B does not directly accommodate the FILEOBJ fields, but rather the FILEOBJ ptr fields. FILEOBJ ptr indicates a FILEOBJ index (e.g., FILEOBJ pointer), which FILEOBJ index is used to index FILEOBJ of records in a specified memory space. The FILEOBJ index, as an index of index FILEOBJ, is of a size that is independent of the FILEOBJ size and, as an index, is of a size that does not exceed 64 bits, e.g., FILEOBJ index 15 bits, and is fully capable of accommodating FILEOBJ indexes, e.g., with 1 DWORD size (64 bits).
For example, at custom command indication fdelect (file_path_name), where file_path_name is "/root/bin/foo.txt", its size exceeds 64 bits. Since FILEOBJ exceeds 64 bits, it cannot be accommodated directly in custom commands by 1 DWORD. The embodiment of the present application adopts the custom command format illustrated in fig. 3B, and the custom command does not directly accommodate FILEOBJ, but records FILEOBJ index by setting FILEOBJ ptr field in the custom command, and then indexes to the designated memory space by FILEOBJ index, and FILEOBJ is recorded in the designated memory space. For example, a memory space size of 4KB is specified, sufficient to accommodate "/root/bin/foo.txt".
FIG. 3C illustrates a format of a custom command provided by yet another embodiment of the present application.
For example, referring to FIG. 3C, custom commands carrying file system basic operations do not directly host the FILEOBJ fields, only include the OP hosting field and the PRPList hosting field. The field holding the OP is, for example, 10 bits in size, and the field holding PRPList is, for example, 1 or several DWORDs in size. By way of example, the index of the first PRP entry of PRPList (e.g., the pointer to the first PRP entry) is recorded in a field of the custom command of embodiments of the present application that accommodates PRPList. The field holding PRPList records an index of the first PRP entry of PRPList, which is used to index the first PRP entry of PRPList (e.g., PRP 0), through the first PRP entry of PRPList to its corresponding specified size memory space (e.g., 4 KB), in which FILEOBJ is recorded, and FILEOBJ is found.
By way of example, fileopen ("/root/bin/foo.txt") has a file path length well in excess of 64 bits, FILEOBJ in excess of 64 bits. Since FILEOBJ exceeds 64 bits, it cannot be accommodated directly in custom commands by 1 DWORD. The embodiment of the application adopts the custom command format shown in fig. 3C, and the custom command does not directly hold FILEOBJ, but records FILEOBJ into the memory space corresponding to the first PRP entry. The FILEOBJ is found by accommodating the index of the first PRP entry of the PRPList field record PRPList in the custom command, indexing the first PRP entry by the index of the first PRP entry, and further indexing its corresponding memory space by the first PRP entry. FILEOBJ are stored in a memory space of, for example, 4KB, the 4KB memory space being sufficient to accommodate "/root/bin/foo.txt".
Also by way of example, PRPList includes the 11 PRP entries PRP0, PRP1, PRP2, PRP3, PRP4, PRP5, PRP6, PRP7, PRP8, PRP9, and PRP 10. Since PRPList includes more PRP entries, these PRP entries are too numerous to be accommodated by custom commands. In the embodiment of the present application, the custom command format illustrated in fig. 3C is adopted, and the field in the custom command for accommodating PRPList does not directly accommodate the PRP entry corresponding to PRPList, but rather records the index of the first PRP entry of PRPList, determines the index of other PRP entries in PRPList according to the index of the first PRP entry, and indexes the corresponding PRP entries according to the indexes of the PRP entries. For example, the second PRP entry index of PRPList = first PRP entry index +1 of PRPList, and so on to obtain the index of the other PRP entries in PRPList. Data, buf are recorded in the memory space indicated in the other PRP entries (except the first PRP entry) in PRPList. The custom command format as illustrated in fig. 3C can support PRPList cases where more PRP entries are included. The above-mentioned several custom command formats are acceptable to both electronic devices and storage devices, and the storage devices analyze the custom commands in different formats in different manners.
In some implementations, the electronic device agrees with the storage device in the custom command format employed. In other embodiments, in order to enable the storage device to identify which format the custom command takes, a flag field is also carried in the custom command, where the flag field is used to indicate which format the custom command takes. The storage device identifies the corresponding format based on the content of the flag field record, and thus identifies the fields of the custom command. By way of example, FIG. 4A shows a format of a custom command provided in another embodiment of the present application, see FIG. 4A, with a value of a flag field in the custom command being flag0, indicating that the custom command format is the format shown in FIG. 3A, FIG. 4B shows a format of a custom command provided in another embodiment of the present application, see FIG. 4B, with a value of a flag field in the custom command being flag1, indicating that the custom command format is the format shown in FIG. 3B, and FIG. 4C shows a format of a custom command provided in another embodiment of the present application, see FIG. 4C, with a value of a flag field in the custom command being flag2, indicating that the custom command format is the format shown in FIG. 3C.
FIG. 5 illustrates a block diagram of a storage device supporting basic operation of a file system provided by an embodiment of the present application. As an example, the storage device includes a control section, as shown in fig. 5, including a host interface, a host command processing unit, a file system (implemented in software or hardware), a storage command processing unit, a media interface controller, and a storage media management unit. The storage device is coupled to an electronic device (e.g., a host or FPGA device).
The storage device is coupled with the electronic device, and the host command processing unit receives a user-defined command carrying basic operation of the file system sent by the electronic device through the host interface. After receiving the custom command carrying the basic operation of the file system, the host command processing unit forwards the basic operation of the file system carried by the custom command to the file system.
The file system generates one or more storage commands according to the type of basic operation of the file system and file parameters, and provides the generated storage commands to the storage command processing unit. The control unit provides the storage command to the storage command processing unit by the file system instead of the host command processing unit as shown in fig. 1B when processing a custom command provided by the electronic device carrying the basic operation of the file system. The storage command provided by the file system to the storage command processing unit is the same as the format or carried information of the storage command sent by the host command processing unit to the storage command processing unit when the control unit processes the NVMe command sent by the host.
The storage command processing unit processes the storage command. The storage medium management unit maintains a logical address to physical address translation for each storage command. The storage command processing unit generates a media interface command according to the physical address provided by the storage media management unit, and operates the media interface controller to send a storage media access command to the NVM chip through the media interface command. It should be understood that when the control unit processes the custom command carrying the basic operation of the file system sent by the electronic device, the processing procedure of the storage command processing unit, the storage medium management unit and the medium interface controller in the control unit is similar to that of the NVMe command sent by the processing host.
For a storage device, its LBA space is divided into two parts for recording file system metadata and file data, respectively, for example, a specified area of the LBA space records file system metadata, and the file system metadata is used to describe a recording position (logical address) of file data managed by the file system in the LBA space. Different file systems, whose file system metadata have different formats, meanings and recording locations. For example, for a FAT file system, the file system metadata is a FAT table located at the start and end of the LBA space, and for an XFS file system, the file system metadata includes superblocks, inodes, free space information, etc., located at the start of the LBA space. And the organization manner/data structure of the file system data of different file systems are the same.
Taking FPGA devices as an example, a procedure of accessing the storage device by the electronic device based on the custom command will be described below.
Fig. 6 illustrates a schematic diagram of an FPGA device accessing a storage device through a custom command according to an embodiment of the present application.
In fig. 6 file system metadata is located at the beginning of the LBA space. Referring to fig. 6, after generating a custom command carrying the basic operation of the file system, the fpga device sends the custom command to the storage device. A host command processing unit in a control component of the storage device receives the custom command through the host interface, and forwards the file system basic operation carried by the custom command to the file system in response to the custom command carrying the file system basic operation (e.g., OP-based content). The file system generates one or more storage commands to process the file system basic operation according to the type of the file system basic operation and the file parameters in response to receiving the file system basic operation, and optionally returns the processing result to the host command processing unit. The custom command at least carries OP and file parameters, when the custom command indicates file reading operation and file writing operation, PRPList representing buf or dataf is further included, and when the custom command indicates file listing operation, the custom command includes PRPList indicating buf.
As an example, the custom command sent by the FPGA device carries a fread (file_path_name, buf), which indicates a file reading operation. Since path lengths corresponding to file data to be accessed by the custom command sent by the FPGA device are possible, and depths of file system paths are possible, after the file system receives a free (file_path_name, buf), the file system sends one or more storage commands according to the file_path_name to find and read out inodes (marked as inodes_d) of all levels of directories under the file system path indicated by the file_path_name in the metadata of the file system.
After obtaining the inode_d of the directory where the file to be read indicated by the file_path_name is located, one or more inodes (denoted as inode_f) in which the file data of the file to be read indicated by the file_path_name is recorded are obtained according to the inode_d. In the case of a large file, it corresponds to a plurality of inodes_f. The file system obtains the LBA address in the LBA space where the file data is stored based on inode_F and generates one or more storage commands to read the file data.
For example, for fread ("/root/abc. Txt", buf), an inode representing a root directory "/" (inode_d) is first read by a store command, located at a known location in the LBA space, and the LBA address of the inode representing the directory "root" is obtained from the inode. The inode (inode_d) representing the directory "root" is then read by another store command, and the LBA address of the inode (inode_f) representing the file "abc. Txt" is obtained from the inode. Next, an inode (inode_f) representing the file "abc. Txt" is read by another storage command, and the LBA address representing the storage location of the file data itself of the file "abc. Txt" is acquired from the inode, and then the file data of the file "abc. Txt" is read by a further storage command. Alternatively, when the file "abc. Txt" is large, there are a plurality of inodes_f record LBA addresses representing storage locations of the file data itself of the file "abc. Txt".
Still by way of example, the file_path_name (data) carried by the custom command sent by the FPGA device indicates a write file operation. For fwrite ("/root/a. So", data), an inode representing a root directory "/" (inode_d) is first read by a store command, located at a known location in the LBA space, and the LBA address of the inode representing the directory "root" is obtained from the inode. The inode (inode_d) representing the directory "root" is then read by another store command, and the LBA address of the inode (inode_f) representing the file "a.so" is obtained from the inode. Next, an inode (inode_f) representing the file "a.so" is read by another storage command, and the LBA address of the storage location of the file data itself representing the file "a.so" is acquired from the inode, and then the file data of "a.so" is written by a further storage command. Optionally, a new LBA space is allocated for the file "a.so" to accommodate more file data, the LBA address of the newly allocated LBA space is recorded in inode_f, and data is also written to the newly allocated LBA address by a storage command. Still alternatively, if the original inode_f cannot accommodate more LBA addresses, a new inode_f is allocated, and in addition to writing file data to the newly allocated LBA addresses by a store command, the newly allocated inode_f is written to the LBA space by a store command, which becomes part of the file system metadata.
The storage command generated by the file system is processed by the storage command processing unit. For example, for fwrite (file_path_name), the storage command processing unit determines, in response to the received storage command, a physical address (PPA) corresponding to the LBA carried by the storage command, generates a media interface command carrying the PPA, and provides the media interface command to the media interface controller. Because the media interface command carries a physical address, the accessed logic unit is determined based on the physical address carried by the media interface command, and the storage media access command is generated and sent to the corresponding logic unit, so that the writing of file data to the corresponding physical page is realized.
The control section transmits a processing completion generation completion message to the FPGA device in response to a custom command indicating an fwrite (file_path_name) operation.
For another example, for the file_path_name (buf), the storage command processing unit determines, in response to the received storage command, a physical address (PPA) corresponding to the LBA carried by the storage command, generates a media interface command carrying the PPA, and provides the media interface command to the media interface controller. Because the media interface command carries a physical address, the accessed logic unit is determined based on the physical address carried by the media interface command, and the storage media access command is generated and sent to the corresponding logic unit to read the data in the physical page indicated by the PPA. The file system manages file system metadata and file data in the LBA space without awareness of the physical address.
After reading the file data based on the file_path_name, the control section moves the read data into the memory space buf described by PRPList, and transmits a generation completion message to the FPGA device in response to the processing completion of the custom command indicating the file_path_name operation.
In another example, a custom command sent by the FPGA device carries flist (file_path) that indicates the listing of all files and directories under the specified file system path. For example, for flist ("/root/Directory 1"), an inode representing a root Directory "/" (inode_d) is first read by a store command, located at a known location in the LBA space, and the LBA address of the inode representing the Directory "root" is obtained from the inode. The inode (inode_d) representing the Directory "root" is then read by another store command, and the LBA address of the inode (inode_d) representing the Directory "Directory1" is obtained from the inode. Next, the inode (inode_d) representing the Directory "Directory1" is read by another storage command, and the LBA addresses of the inodes of all files and directories under the file system path of "/root/Directory1" are recorded in the inode, so that all files and directories under the file system path of "/root/Directory1" are enumerated.
The foregoing describes the case that the custom command sent by the FPGA device carries a free (file_path_name, buf), an fwrite (file_path_name, data), and flist (file_path), and the custom command sent by the FPGA device to the storage device may also carry other information and indicate other types of operations, which are not listed herein.
According to the embodiment of the application, the FPGA equipment sends a custom command carrying basic operation of a file system to the storage equipment, and a directory structure and written files are formed in the LBA space of the storage equipment. For example, the FPGA device sequentially sends the custom commands to the storage device, and after the storage device processes the custom commands, the custom commands carry fwrite("/root/bin/foo.s","aabbccdd"),fwrite("/root/dev/abc.txt","hello"),fwrite("/root/bar.dat","……"), respectively, and file system data with the following structure is formed in a logical address space of the storage device:
/root
/bin/foo.s
/dev/abc.txt
bar.dat
Wherein,/root,/bin,/dev represents a directory, and/bin and/dev are subdirectories of/root, and foo.s, abc.txt and bar.dat represent files, the foo.s file is located in the directory/bin, the abc.txt file is located in the directory/dev, and the bar.dat file is located in the directory/root.
Thus, the FPGA device can write the files in the path of the specified file system to the storage device in a file mode without including the file system. Moreover, the file data written by the FPGA device to the storage device according to the embodiment of the application can be identified by the host including the file system in the prior art without modifying the host.
It should be understood that when the electronic device provided in the embodiment of the present application is a host, the process of accessing the storage device through the custom command is similar to that of the FPGA device, and will not be described herein.
The electronic device can access the storage device to read the written file data after writing the file data into the storage device based on the custom command, and in addition, the storage device can be coupled with other devices (target devices such as a host or an FPGA device) of the electronic device which are different from the written file data, and the written file data of the electronic device can be read through the other devices. For example, other devices than electronic devices that write file data have file systems that can access the storage device in a file system manner. After the electronic device writes the file data to the storage device based on the custom command, other devices than the electronic device writing the file data may access the file data written by the electronic device through the file system. In the following, the electronic device is taken as an FPGA device, the other devices are taken as hosts, and a process of accessing file data written in the electronic device by the other devices through a file system is described, with particular reference to fig. 7.
Fig. 7 is a schematic diagram of a host accessing a storage device through an IO command according to an embodiment of the present application.
Referring to fig. 7, the host includes a file system, an application program, and an NVMe driver (not labeled in fig. 7), etc. It will be appreciated that the file system in the host, which includes file system metadata, differs from the file system in the control unit shown in fig. 6 described above.
The application running in the host is to read file data, and issue a file system basic operation to the file system running in the host, and the file system of the host issues one or more NVMe read commands to the storage device through, for example, an NVMe driver according to the received file system basic operation, the storage device processes the NVMe read commands in a prior art manner (see fig. 1B), and provides the read file data to the host.
For a host, it generates a file system basic operation by running an application program, and the file system of the host determines an LBA in which operated file data is stored according to a file system path (file_path_name) of a file operated in the file system basic operation and file system metadata. The file system sends the LBAs to, for example, an NVMe driver, which generates NVMe commands based on the LBAs by running the NVMe driver. The LBA space used by the file system of the storage device is consistent with the LBA space provided by the storage device to the host, so that the host can access the LBA space of the storage device through NVMe commands, so that the host can write data to the LBA space of the storage device through NVMe commands, and can also access file data stored in the LBA space of the storage device.
By way of example, the file system basic operation is a read file operation indicated by a read ("/root/bar. Dat", buf), the file system of the host finds and reads out an inode (inode_d) representing a root directory "/", reads an inode representing a directory "root" from the inode, acquires an inode (inode_f) representing a file "bar. Dat" from the inode, acquires an LBA address representing a storage location of file data itself of the file "bar. Dat" from the inode, and sends the LBA to the NVMe driver.
For example, the specific application scenario of the embodiment of the present application includes a first stage in which the FPGA device writes the file data and a second stage in which the host reads the file data written by the FPGA device. The first stage occurs at the data acquisition site, writing the acquired data to the storage device via the FPGA device without the involvement of the host. The FPGA equipment and the storage equipment can form a miniaturized or special system processing system, the system processing system is deployed or applied to occasions such as the field, ships, spacecrafts, factories and the like, and the collected field data is written into the storage equipment in a file form in a high-speed and low-power consumption mode. The second phase occurs, for example, in a data center or laboratory that analyzes data. The laboratory has a fully functional, fully functional information processing device such as a host computer to read out file data from the storage device and analyze it.
In the first stage (data acquisition and recording stage), the FPGA device is connected to the storage device, and the collected field data is written into the storage device in a file operation manner, and the process of writing the file data into the storage device by the FPGA device is described in the above related description, which is not repeated here. Because the FPGA equipment does not need to realize a file system, the performance of the FPGA equipment can be fully used for collecting data and generating a custom command, and higher data throughput bandwidth can be realized.
In the second stage, the storage device is disconnected with the FPGA device and connected with the host, the host reads file data written in the storage device by the FPGA device through the file system by using an NVMe read command, and the storage device processes the NVMe read command sent by the host according to the existing mode.
After the electronic device provided by the embodiment of the application writes the file data into the storage device in the form of a file, the file system metadata and the file data are recorded in the LBA space of the storage device. The organization/data structure of the file system metadata and the file data (collectively referred to as file system data) is the same as that of file system data generated by a software-form file system in another device (having a file system) other than the electronic device in which the file data is written, so that the software-form file system in the other device can directly recognize the file system data of the storage device and mount it in a directory form to a specified file system path of the other device (e.g., host). The file data that the other device accesses the electronic device to write to the storage device is made in a file system manner, so that the other device generates NVMe IO commands to access the files written by the electronic device in the storage device in the prior art scheme (e.g., by a software form file system running through a host and an NVMe driver). When a user operates other devices to access files written by the electronic devices in the storage device, the user only needs to access the files by means of a file system in the prior art, and no modification is needed to the other devices.
It should be noted that, the embodiment of the present application may be understood that the electronic device sends a custom command carrying a basic operation of a file system to the storage device, and the storage device processes the basic operation of the file system by processing the custom command, so as to enable the electronic device to access the storage device in a file system manner. After the electronic equipment writes the file data into the storage equipment in a file system mode, other equipment accesses the file data written into the storage equipment by the electronic equipment based on IO commands between the file system and the storage equipment, and the other equipment can recognize and access the file data written into the storage equipment by the electronic equipment under the condition that the other equipment is not changed.
Fig. 8 is a block diagram of a storage device according to another embodiment of the present application.
As an example, referring to fig. 8, the control part of the memory device includes a control circuit 801 and a control circuit 802. The control circuit 802 loads a file system as shown in fig. 5 and 6. The storage device is coupled to an electronic device (host or FPGA device). The electronic device can send NVMe commands or NVMe VU commands to the storage device to access the storage device. If the electronic device (e.g., host) sends an NVMe command to the storage device to access the NVM chip, the control circuit 801 processes the NVMe command, and if the electronic device sends a custom command to the storage device to access the storage device in a file system manner, the control circuit 801 and the control circuit 802 cooperate to process the custom command. It should be appreciated that the custom command sent by the electronic device to the storage device indicates the file system basic operation by which the electronic device controls the storage device to access the LBA space in a file system manner.
In the case that the electronic device sends a custom command carrying a basic operation of a file system to the storage device, the host command processing unit of the control circuit 801 receives the custom command, sends the basic operation of the file system indicated by the custom command to the control circuit 802, the file system loaded in the control circuit 802 generates a storage command carrying an LBA according to the received basic operation of the file system, and sends the storage command to the storage command processing unit in the control circuit 801, and the storage command processing unit determines a PPA corresponding to the LBA carried by the storage command, and provides the PPA to the media interface controller in the form of a media interface command. The medium interface controller determines the accessed logic unit based on the physical address carried by the medium interface command, determines the LUN controller corresponding to the accessed logic unit based on the mapping relation between the logic unit and the LUN controller, processes the medium interface command by the LUN controller, generates a storage medium access command and sends the storage medium access command to the corresponding logic unit.
In the case where an electronic device (e.g., a host) sends an NVMe command to a storage device, the host command processing unit of the control circuit 801 receives the NVMe command, sends a plurality of corresponding storage commands to the storage command processing unit, and the process of processing the storage commands by the storage command processing unit is described above, which is not repeated herein.
Fig. 9 is a block diagram of a storage device according to another embodiment of the present application.
As an example, as shown in fig. 9, the control means includes a control circuit 901 and a control circuit 902, wherein the control circuit 901 includes a CPU group 1 and a media interface controller for processing an NVMe command such as an NVMe write command or an NVMe read command so that the NVM chip is accessed based on the NVMe command. The control circuit 902 includes a CPU group 2 for processing a user-specified operation or a file system basic operation. The electronic device interacts data with the control circuit 901 over the PCIe link. For example, to access data in the NVM chip, the user generates an NVMe read command or an NVMe write command through the host, and then sends the NVMe read command or the NVMe write command to the control circuit 901 through the PCIe link between the host and the control circuit 901. The host command processing unit, storage media management unit, and media interface controller in the control circuit 901 process NVMe read commands or NVMe write commands and access the NVM chip.
With continued reference to FIG. 9, for example, CPU group 1 includes four CPU cores CPU 1-0, CPU 1-1, CPU 1-2, CPU 1-3. Optionally, a real-time operating system (Real Time Operating System, RTOS), for example, runs on the four CPU cores of the CPU group 1, the CPU group 1 implementing a host command processing unit, a storage command processing unit, and a storage medium management unit with the support of the operating system, and also providing, for example, management FTL tables or garbage collection GC functions. It will be appreciated that the implementation of the host command processing unit, the storage command processing unit, and the storage medium management unit may be independent of the operating system.
As an example, as shown in fig. 9, the CPU group 2 includes four CPU cores of CPU 2-0, CPU 2-1, CPU 2-2, and CPU 2-3, and a standard operating system (such as Linux) is running on the four CPU cores of the CPU group 2, and a remote connection tool (such as a secure shell protocol SSH (Secure Shell)), an application program, a file system, and the like are provided on the standard operating system. Communication between the electronics and the control circuitry 902 in the control unit is via, for example, SSH.
In the example of fig. 9, the control circuit 901 and the control circuit 902 each include 4 CPU cores. It will be appreciated that each control circuit may include other numbers of CPU cores, which may be the same or different from each other. The number of CPU cores of each of the control circuit 901 and the control circuit 902 is not necessarily the same.
Still by way of example, when the host communicates with an operating system/application in the control circuitry 902, communication may be via a PCIe link between the control circuitry 901 and the host. For example, the host carries a communication or call request to the control circuit 902 via a custom command conforming to the NVMe protocol, and the control circuit 901 forwards such communication or call request to the control circuit 902, and a corresponding response by the control circuit 902 is also returned to the host via the control circuit 901 as a response to the custom command.
Since the control circuit 902 runs a standard operating system, it is possible for a user to develop a program running in the control circuit 902 based on the standard operating system, instead of the provider of the control unit. Even without assistance from the provider of the control circuit 902, a user is able to develop a program running in the control circuit 902 using a development tool of the prior art and/or a development tool provided by the provider of the control component. Such programs can also use storage device services based on, for example, NVMe protocols provided by the control circuit 901 in addition to services that can call a standard operating system through an API, and when using storage device services, the transmission path of read and write data occurs inside the storage device without being transmitted to the outside of the storage device via a PCIe link, so that the data throughput can exceed the bandwidth that can be provided by the PCIe link, and lower processing delay.
The hardware accelerator of the control circuit 902 can also be used by programs developed by a user that run in the control circuit 902. Applications that a hardware accelerator may be developed by a user as a device compatible with a standard operating system operate through a standard operating system, a hardware abstraction layer, and/or a board level support package (BSP).
The file system (such as the file system shown in fig. 5 and 6) is loaded on the standard operating system in the control circuit 902. The storage device may process a custom command indicating basic operation of the file system in addition to the NVMe command transmitted from the host, and the storage device processes the custom command in a file system manner to access file data to be accessed as indicated by the custom command. The host command processing unit in the control circuit 901 receives a custom command carrying a basic operation of a file system sent by, for example, an FPGA device, and in response to the custom command carrying the basic operation of the file system, forwards the basic operation of the file system carried by the custom command to the file system of the control circuit 902. In response to receiving the basic operation of the file system, the file system generates one or more storage commands, the storage commands are provided to a storage command processing unit in the control circuit 901, the storage command processing unit obtains PPA (PPA corresponding to the LBA carried by the storage command) provided by the storage media management unit, and the PPA is provided to the media interface controller in the form of a media interface command. The subsequent process flow is performed by the media interface controller.
In addition, the host may also send a custom command (not carrying the file system basic operation) to the storage device to access the storage device, e.g., the host instructs the user-specified operation through the custom command to control the storage device to perform the user-specified operation. For another example, the host may also send a custom command carrying a file system base operation to instruct the file system base operation to access the storage device in a file system manner. In chinese patent application nos. CN2023112835330 and CN2023112872541, the contents of the control unit, the process of interaction between the host and the storage device, and the process of accessing the storage device by the host through the custom command are described, and the contents thereof are incorporated herein in the present application. The following describes a process of the electronic device accessing the storage device to implement the RAID function based on the custom command provided by the embodiment of the present application.
FIG. 10 is a schematic diagram illustrating an implementation of RAID1 by an electronic device accessing two storage devices through custom commands according to an embodiment of the present application.
By way of example, in FIG. 10, a hard RAID mode is employed to implement RAID1 functionality. Fig. 10 includes an electronic device, a RAID module and two storage devices (storage device 1 and storage device 2), for example, the RAID module is a hardware module located outside the electronic device, and is a RAID control processing chip that implements a RAID function in a hard RAID manner, for example, a RAID card, a RAID chip integrated on a motherboard, and the like. The electronic equipment is connected with the RAID module through a PCIe link, and the RAID module is connected with the two storage devices through the PCIe link. The storage device 1 is a working storage device, and the storage device 2 is a mirror image storage device of the storage device 1 and is used for backing up data in the storage device 1. Because the RAID module is located outside the electronic device, the electronic device cannot perceive the RAID module.
For example, the electronic device generates a custom command (NVMe VU command) indicating basic operation of the file system. The file system basic operation carried by the custom command is a file writing operation fwrite (file_path_name, data), the electronic device sends the custom command to the RAID module through a PCIe link, the RAID module copies the custom command to obtain a custom command 1 (NVMe VU command 1) and a custom command 1' (NVMe VU command 1 ') in response to receiving the custom command, wherein the custom command 1 and the custom command 1' are identical, for example, carry the same file system basic operation, have the same format and the like. In addition, the basic operation of the file system carried by the custom command 1 and the custom command 1' is the same as the basic operation of the file system carried by the custom command sent to the RAID module by the electronic device, and is indicated to write the file operation fwrite (file_path_name, data).
The RAID module sends custom command 1 to storage device 1 and custom command 1' to storage device 2, respectively. The storage apparatus 1 processes the custom command 1, and writes data to a file system path and a file name indicated by the file_path_name according to a write file operation fwrite (file_path_name) indicated by the custom command 1. The storage device 2 processes the custom command 1', and writes data to the file system path and the file name indicated by the file_path_name according to the write file operation fwrite (file_path_name) indicated by the custom command 1'. Since the custom command 1 and the custom command 1 'are also identical, the file data written by the storage device 1 processing the custom command 1 is identical to the file data written by the storage device 2 processing the custom command 1', thereby realizing the function of RAID 1.
For another example, the electronic device generates a file system basic operation indicated by the custom command as a file_path_name (buf), the electronic device sends the custom command to the RAID module through the PCIe link, the RAID module does not copy the custom command after receiving the custom command, and sends the received custom command to the storage device 1, the storage device 1 processes the custom command, and reads corresponding file data from the storage device 1 according to the file_path_name indicated by the custom command. The RAID module is used for responding to the file data error read from the storage device 1 based on the custom command, copying the custom command to obtain a custom command 1', wherein the custom command 1' is identical to the custom command, the RAID module is used for sending the custom command 1 'to the storage device 2, and the storage device 2 is used for processing the custom command 1' to read the backed-up file data from the storage device 2. The RAID module completes the custom command processing in response to the file data read from the storage device 1 based on the custom command being correct. For either one of the storage device 1 or the storage device 2, the control section of the storage device is similar to that shown in fig. 9. The process of the control unit in the storage device 1 processing the custom command 1 or the process of the control unit in the storage device 2 processing the custom command 1' is shown in fig. 9, and will not be described herein. The foregoing describes, by way of example, the processing of a custom command by a storage device that indicates a write file operation, and a custom command by a storage device that indicates a read file operation, respectively. When a user-defined command indicating file writing operation is processed, the RAID module copies the user-defined command sent by the electronic device to obtain two identical user-defined commands, the two identical user-defined commands are respectively sent to the two storage devices, and the two storage devices store identical data into the two storage devices based on the received user-defined commands, so that data backup is realized, and high storage reliability of RAID1 is realized.
It should be understood that the RAID module shown in fig. 10 may also be located inside the electronic device, and is not limited herein, and is a hardware module inside the electronic device.
FIG. 11 is a schematic diagram illustrating a RAID1 implementation by an electronic device accessing two storage devices through custom commands according to another embodiment of the present application.
As an example, in fig. 11, a soft RAID scheme is used to implement the RAID1 function. Fig. 11 includes an electronic device and two storage devices (storage device 1 and storage device 2), where storage device 1 is a working storage device and storage device 2 is a mirror storage device of storage device 1 for backing up data in storage device 1. The storage device 1 comprises a file system 1, the storage device 2 comprises a file system 2, the file system 1 and the file system 2 are two independent file systems, and the file system 1 and the file system 2 can be the same or different in structure. The electronic device includes an application program, a RAID module, such as software inside the electronic device, and an NVMe driving circuit. File system basic operations are generated in the electronic device by running an application program (APP). The application program sends the file system basic operation to the RAID module in response to generating it. If the RAID module acquires the basic file system operation, and responds to the basic file system operation as a file writing operation, the RAID module copies the basic file system operation to obtain the same basic file system operation 1 and basic file system operation 1', and sends the basic file system operation 1 and the basic file system operation 1' to the NVMe driving circuit. The NVMe driving circuit generates a custom command 1 (NVMe VU command 1) carrying a file system basic operation 1 and generates a custom command 1' (NVMe VU command 1 ') carrying a file system basic operation 1', and the custom command 1 is the same as the custom command 1', and respectively sends the custom command 1 to the storage device 1 and the custom command 1' to the storage device 2. Storage device 1 processes custom command 1 to access LBA space 1 and storage device 2 processes custom command 1' to access LBA space 2, wherein LBA space 1 is the LBA space managed by storage device 1, LBA space 2 is the LBA space managed by storage device 2, LBA space 1 and LBA space 2 are two independent LBA spaces. The process by which the storage device processes custom commands carrying basic operations of the file system is referred to in the foregoing related description and will not be described herein. By sending the same NVMe VU commands to the storage device 1 and the storage device 2, respectively, the storage device 1 and the storage device 2 have the same file system structure (the file system data managed by the file system are the same), so that the backup of the data is realized.
When one of the two storage devices fails, file system data of the storage device which does not fail can be moved to the new storage device through file copy operation of the file system, and the RAID module is accessed (without copying the whole LBA space or the whole PPA space), so that the RAID module can continue to operate the two storage devices in a RAID1 mode to work normally.
For example, when the storage device 1 fails, the failed storage device 1 is replaced with a new storage device 3, the file system data of the storage device 2 is copied to the storage device 3, and the storage device 3 is connected to the RAID module, so that the storage device 3 and the storage device 2 can continue to operate in the RAID1 mode, and high reliability is further achieved.
In addition to the high reliability storage requirements, the demand for mass storage is increasing in practical applications. In order to meet the requirement of mass storage, the system provided by the embodiment of the application comprises the electronic device, the RAID module (such as positioned inside or outside the electronic device) and two storage devices (the storage device 1 and the storage device 2), and can realize RAID1 function and be compatible with RAID0 function on the basis of not changing the structure of the system. The RAID1 function and the RAID0 function can be switched according to actual requirements.
FIG. 12 is a schematic diagram illustrating implementation of RAID0 functionality according to an embodiment of the present application.
As an example, in fig. 12, the RAID0 function is implemented by a soft RAID method. Fig. 12 is the same as the system shown in fig. 11, and also includes an electronic device and two storage devices (a storage device 1 and a storage device 2), wherein the storage device 1 includes a file system 1, the storage device 2 includes a file system 2, and the file system 1 and the file system 2 are two independent file systems, and in addition, the structures of the file system 1 and the file system 2 may be the same or different. The electronic device includes an application program, a RAID module, such as software inside the electronic device, and an NVMe driving circuit. The file system basic operation 1 for accessing the file system 1 and the file system basic operation 2 for accessing the file system 2 are generated in the electronic device by running an application program (APP), wherein the file system basic operation 1 and the file system basic operation 2 may be different or the same. The application program transmits the generated file system basic operation 1 and file system basic operation 2 to the RAID module in response thereto. The RAID module acquires the file system basic operation 1 and the file system basic operation 2, and sends the file system basic operation 1 and the file system basic operation 2 to the NVMe driving circuit respectively. The NVMe driving circuit generates a custom command A carrying a file system basic operation 1 and generates a custom command B carrying a file system basic operation 2, and the custom command A and the custom command B are respectively sent to the storage device 1 and the storage device 2. The storage device 1 processes the custom command a and the storage device 2 processes the custom command B. The process by which the storage device processes custom commands carrying basic operations of the file system is referred to in the foregoing related description and will not be described herein.
For example, the file system basic operation 1 indicates writing file data 1, and the file system basic operation 2 indicates writing file data 2, where the file data 1 and the file data 2 are different file data, the storage device 1 processes the custom command a to write the file data 1 into the LBA space 1 corresponding to the storage device 1, and the storage device 2 processes the custom command B to write the file data 2 into the LBA space 2 corresponding to the storage device 2. Based on the above, on the basis of not changing the system structure for realizing the RAID1 function, different file data can be respectively written into the storage device 1 and the storage device 2 based on the system, so that the utilization of the storage space of the storage device 1 and the storage device 2 is realized, and the RAID0 function and the mass storage are further realized.
It should be understood that the file system structures managed by the file system 1 of the storage device 1 and the file system 2 of the storage device 2 may be different, or may have the same file system structure, for example, all the file system basic operations of accessing the file system 1 and the file system 2 issued by the application program are the same.
FIG. 13 is a schematic diagram of implementing RAID0 functionality according to another embodiment of the present application.
For example, in fig. 13, the RAID0 function is implemented by using a hard RAID scheme. Fig. 13 includes an electronic device, a RAID module and two storage devices (storage device 1 and storage device 2), for example, the RAID module is a hardware module located outside the electronic device, and is a RAID control processing chip that implements a RAID function in a hard RAID manner, for example, a RAID card, a RAID chip integrated on a motherboard, and the like. The electronic equipment is connected with the RAID module through a PCIe link, and the RAID module is connected with the two storage devices through the PCIe link. Because the RAID module is located outside the electronic device, the electronic device cannot perceive the RAID module.
For example, the electronic device generates a custom command a (NVMe VU command a) indicating the file system basic operation 1 and a custom command B (NVMe VU command B) indicating the file system basic operation 2, and the file system basic operation 1 may be different from or the same as the file system basic operation 2. The electronic device sends the custom command A and the custom command B to the RAID module through the PCIe link, and the RAID module sends the custom command A to the storage device 1 and the custom command B to the storage device 2 in response to receiving the custom command. The storage device 1 processes the custom command a, and the storage device 2 processes the custom command B, so that 2 storage devices are accessed based on 2 custom commands.
Under the condition that the basic file system operation 1 is different from the basic file system operation 2, different custom commands are sent to the two storage devices, so that file system data of the two storage devices are different, different storage devices can be used for storing different data, and high-capacity storage is realized.
While preferred embodiments of the present application have been described, additional variations and modifications in those embodiments may occur to those skilled in the art once they learn of the basic inventive concepts. It is therefore intended that the following claims be interpreted as including the preferred embodiments and all such alterations and modifications as fall within the scope of the application. It will be apparent to those skilled in the art that various modifications and variations can be made to the present application without departing from the spirit or scope of the application. Thus, it is intended that the present application also include such modifications and alterations insofar as they come within the scope of the appended claims or the equivalents thereof.

Claims (10)

1. A method of accessing a storage device, characterized by being applied to an electronic device that couples a first storage device and a second storage device and that is free of file system functionality, the method comprising:
Generating a first custom command and a second custom command carrying the file system basic operation and conforming to an NVMe protocol in response to a first file system basic operation indicating a file writing operation;
Transmitting the first custom command to the first storage device, and transmitting the second custom command to the second storage device to access a first logical address space managed by the first storage device and a second logical address space managed by the second storage device;
Wherein the first custom command and the second custom command both bear the first file system basic operation.
2. The method as recited in claim 1, further comprising:
generating a third custom command which carries the first file system basic operation and follows an NVMe protocol in response to the first file system basic operation indicating the file writing operation;
generating a fourth custom command which carries the second file system basic operation and follows an NVMe protocol in response to the second file system basic operation indicating the file writing operation;
And sending the third custom command to the first storage device and the fourth custom command to the second storage device to access a first logical address space managed by the first storage device and a second logical address space managed by the second storage device.
3. The method according to claim 1 or 2, wherein,
Generating the first custom command based on the first file system basic operation and copying the first custom command to obtain the second custom command identical to the first custom command, or
And copying the first file system basic operation to obtain a third file system basic operation which is the same as the first file system basic operation, and generating the same first custom command and second custom command according to the first file system basic operation and the third file system basic operation.
4. The method of claim 2, wherein in response to the first file system base operation and the second file system base operation being the same file system base operation, the third custom command and the fourth custom command carry the same file system base operation, or
And responding to the first file system basic operation and the second file system basic operation as different file system basic operations, wherein the third custom command and the fourth custom command bear different file system basic operations.
5. The method according to any one of claim 1 to 4, wherein,
And responding to the first custom command and the second custom command to bear the same basic operation of the file system, and respectively writing the same file data into a first logic address space of the first storage device and a second logic address space of the second storage device based on the first custom command and the second custom command.
6. The method according to claim 2 or 4, wherein,
And responding to the third custom command and the fourth custom command to bear different file system basic operations for indicating file writing operation, and respectively writing different file data into a first logic address space of the first storage device and a second logic address space of the second storage device based on the third custom command and the fourth custom command.
7. The method of any one of claims 1 to 6, wherein the first logical address space and the second logical address space are independent logical address spaces, wherein the first storage device manages file data written to the first logical address space through a first file system, and wherein the second storage device manages file data written to the second logical address space through a second file system.
8. The method according to any one of claims 1 to 7, wherein,
And in response to sending the first custom command to the first storage device, sending the second custom command to the second storage device, the second storage device being a mirrored storage device of the first storage device.
9. The method of claim 8, wherein,
Generating a fifth custom command which carries the fourth file system basic operation and follows an NVMe protocol in response to the fourth file system basic operation generating the file reading operation, and sending the fifth custom command to the first storage device;
and responding to the failure of the first storage device, generating a sixth custom command which carries the basic operation of the fourth file system and conforms to an NVMe protocol based on the failure of the fifth custom command to read data, and sending the sixth custom command to the second storage device so as to read the file data corresponding to the basic operation of the fourth file system from the second storage device based on the sixth custom command.
10. An electronic device, wherein the electronic device comprises an application program, an NVMe driving circuit and a RAID module;
the application program generates a first file system basic operation indicating a file writing operation;
The NVMe driving circuit is matched with the RAID module to generate a first custom command and a second custom command which bear the basic operation of the first file system and follow an NVMe protocol;
The NVMe drive circuit or the RAID module sends the first custom command to a first storage device and the second custom command to a second storage device to access a first logical address space of the first storage device and a second logical address space of the second storage device.
CN202410494437.9A 2024-04-23 2024-04-23 Method of implementing RAID using file system SSD Pending CN120832077A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202410494437.9A CN120832077A (en) 2024-04-23 2024-04-23 Method of implementing RAID using file system SSD

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202410494437.9A CN120832077A (en) 2024-04-23 2024-04-23 Method of implementing RAID using file system SSD

Publications (1)

Publication Number Publication Date
CN120832077A true CN120832077A (en) 2025-10-24

Family

ID=97400591

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202410494437.9A Pending CN120832077A (en) 2024-04-23 2024-04-23 Method of implementing RAID using file system SSD

Country Status (1)

Country Link
CN (1) CN120832077A (en)

Similar Documents

Publication Publication Date Title
US11086774B2 (en) Address translation for storage device
US8166258B2 (en) Skip operations for solid state disks
US10126959B2 (en) Systems and methods for a mass data storage system having a file-based interface to a host and a non-file-based interface to secondary storage
US6493811B1 (en) Intelligent controller accessed through addressable virtual space
KR101562781B1 (en) Self-journaling and hierarchical consistency for non-volatile storage
US8095577B1 (en) Managing metadata
US8539174B2 (en) Use by a host device having a first file system of a portable storage device having a second file system and supporting file segmentation
CN100405304C (en) Implementation method of high-speed solid-state storage device based on storage area network
EP3637242B1 (en) Data access method and apparatus
US8694563B1 (en) Space recovery for thin-provisioned storage volumes
EP2417530A1 (en) Method and apparatus for storing data in a flash memory data storage device
KR102270103B1 (en) Data storage device and operating method thereof
CN105808163A (en) Method and server for accessing shingled magnetic recording SMR hard disk
EP4386557A1 (en) Method and device for log structured merge-tree based key-value data storage
US9009440B2 (en) Adjustment of data storage capacity provided by a storage system
US20240045597A1 (en) Storage device and operation method thereof
CN113918089A (en) Key value storage device and method for sorting key values
CN120832077A (en) Method of implementing RAID using file system SSD
KR101756228B1 (en) Memory apparatus
KR102435910B1 (en) Storage device and operation method thereof
KR102509987B1 (en) Computing system including host and storage system
CN108369555B (en) Logical address history management in a memory device
CN115328819A (en) Solid-state memory device and method for writing/reading data
US11188511B2 (en) Offloading file-indexing to memory card firmware
CN120067064A (en) Method for responding to FPGA equipment access by using file system and related products

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination