CN118626008A - Metadata management method, device, electronic device and readable storage medium - Google Patents

Abstract

The application discloses a metadata management method, a device, an electronic device and a readable storage medium, which relate to the technical field of large-scale data storage and are applied to a server side configured on a metadata management system, wherein the server side and a memory pool on the metadata management system are in communication connection through a remote direct memory access network, and the method comprises the following steps: responding to an access request sent by a client, and acquiring path information of a node to be accessed from the access request; calling a remote direct memory access network, and searching the memory address of the node to be accessed in a memory pool of the metadata management system according to the path information; and acquiring access metadata of the node to be accessed through the memory address and performing data operation on the access metadata. The application overcomes the defect of too high access delay of data read-write and the like caused by too deep input/output software stack of the traditional file system.

Description

Metadata management method, device, electronic device and readable storage medium

Technical Field

The present application relates to the technical field of large-scale data storage, and in particular to a metadata management method, device, electronic device, and readable storage medium.

Background Art

The continuous development of digitalization and informatization is the driving force behind the continuous evolution and iteration of infrastructure such as computing, storage and networks. As global enterprises’ digital transformation progresses and scientific research becomes more dependent on big data processing, big data technologies represented by cloud computing and artificial intelligence are developing rapidly, and the amount of global data is also increasing exponentially. Data centers are facing the severe challenge of how to efficiently store and manage massive amounts of data.

Currently, most data centers use traditional storage devices (such as SSD (Solid-State Disk) and HDD (Hard Disk Drive and other block devices) and traditional file management systems. In the traditional file system for block storage devices such as SSD and HDD, the access request for data reading and writing initiated by the traditional file system will go through the processing of the too deep I/0 software stack composed of the virtual file system, block device file system, general block layer, I/O (Input/Output) scheduling layer and block device driver layer, etc., before the real reading and writing can be completed. In addition, the reading and writing speed of the block device itself is slow, so in general, the access delay of traditional block devices and traditional file systems is too high.

Although there are high-performance storage devices such as persistent memory that have better read and write performance than SSDs and HDDs, which can improve the read and write speed from the hardware level, when traditional file systems initiate access requests for data read and write to high-performance storage devices, they still need to complete data read and write through an overly deep I/O software stack, which restricts the read and write speed of high-performance storage devices and causes large access delays. Therefore, there is currently a technical problem of high access delays caused by the overly deep input/output software stack of traditional file systems.

The above contents are only used to assist in understanding the technical solution of the present application and do not constitute an admission that the above contents are prior art.

Summary of the invention

The main purpose of this application is to provide a metadata management method, device, electronic device and readable storage medium, aiming to solve the technical problem of excessive access delay caused by the too deep input/output software stack of the traditional file system.

To achieve the above object, the present application provides a metadata management method, which is applied to a server configured on a metadata management system, wherein the server establishes a communication connection with a memory pool on the metadata management system via a remote direct memory access network, and the method comprises:

In response to an access request sent by a client, path information of a node to be accessed is obtained from the access request; a remote direct memory access network is called to search for a memory address of the node to be accessed in a memory pool of the metadata management system according to the path information; access metadata of the node to be accessed is obtained through the memory address and data operations are performed on the access metadata.

In one embodiment, the step of calling a remote direct memory access network and searching for the memory address of the node to be accessed in the memory pool of the metadata management system according to the path information includes: determining a root directory node from the path information, calling the remote direct memory access network, and reading a root metadata bucket where the root directory node is located in the memory pool; taking the root directory node as a target node, searching for a target hash bucket matching a target hash value of the target node in the root metadata bucket, and searching for target metadata of the target node in the target hash bucket; when the access rights of the target metadata are open to the client, obtaining a secondary path node of the target node in the path information, and calculating a secondary hash value of the secondary path node according to the target hash value and a secondary name of the secondary path node; searching for a secondary hash bucket matching the secondary hash value in the root metadata bucket, and if the secondary hash bucket contains node metadata of the secondary path node, determining whether the secondary path node is a node to be accessed; if so, determining the memory address according to the secondary hash value.

In one embodiment, after the step of searching the root metadata bucket for a secondary hash bucket that matches the secondary hash value, it also includes: if the node metadata of the secondary path node does not exist in the secondary hash bucket, determining whether the parent node of the secondary path node has an extended metadata bucket; if the parent node has an extended metadata bucket, reading the extended metadata bucket in the memory pool via a remote direct memory access network, and searching for the extended hash bucket of the secondary path node in the extended metadata bucket; if the node metadata of the secondary path node exists in the extended hash bucket, then when the secondary path node is a node to be accessed, determining the memory address according to the secondary hash value.

In one embodiment, the step of performing data operations on the access metadata includes: when the type of the access request is a write request, calling a remote direct memory access replacement operation to determine whether the lock area of the access metadata bucket where the access metadata is located is empty; if it is empty, determining that the server is a modification master node, and through a remote direct memory access replacement operation, writing the master configuration information of the modification master node in the lock area of the access metadata bucket, obtaining write open permissions in the access metadata bucket, and performing a write operation on the access metadata in the access metadata bucket; if it is not empty, determining that the server is a modification slave node, and obtaining the master configuration information from the lock area of the access metadata bucket, establishing a remote direct memory access network communication connection with the modification master node according to the master configuration information, and sending the access metadata to the modification master node to perform a write operation on the access metadata through the modification master node.

In one embodiment, the step of writing the master configuration information of the modified master node in the lock area of the access metadata bucket through a remote direct memory access replacement operation, obtaining write-open permission in the access metadata bucket, and performing a write operation on the access metadata in the access metadata bucket includes: writing the node configuration information of the modified master node in the lock area of the access metadata bucket through the remote direct memory access replacement operation, creating a log space in the modified master node, and determining the remote direct memory access configuration information of the log space; registering the remote direct memory access configuration information of the log space on the modified master node, and writing the remote direct memory access configuration information into the lock area, obtaining write-open permission for the access metadata bucket; and performing a write operation on the access metadata when the write-open permission in the access metadata bucket is obtained.

In one embodiment, after the step of obtaining the write open permission in the access metadata bucket, it also includes: monitoring whether the index area of the log space is updated, and if the index area is updated, obtaining the index increment from the index area, obtaining the metadata to be modified and the modification information of the metadata to be modified from the log area associated with the index increment; modifying the metadata to be modified in the access metadata bucket according to the modification information, and updating the modification status of the metadata to be modified in the result area associated with the index increment.

In one embodiment, the access metadata includes metadata to be modified, and the step of sending the access metadata to the modification master node to perform a write operation on the access metadata through the modification master node includes: calling a remote direct memory access self-increment operation, performing index self-increment in the index area of the log space of the modification master node, and obtaining an index increment of the modification slave node; based on the index increment, writing the metadata to be modified into a log area associated with the index increment, so that the modification master node can modify the metadata to be modified in the log area; and polling a result area associated with the index increment, and upon detecting that the modification status of the metadata to be modified in the result area is updated, feeding back the modification result of the metadata to be modified to the client of the modification slave node.

In addition, to achieve the above-mentioned purpose, the present application provides a metadata management device, which is applied to a server configured on a metadata management system, wherein the server establishes a communication connection with a memory pool on the metadata management system through a remote direct memory access network, and the device comprises: a response module, which is used to respond to an access request sent by a client, and obtain path information of a node to be accessed from the access request;

A calling module, used for calling a remote direct memory access network, and searching for a memory address of the node to be accessed in a memory pool of the metadata management system according to the path information;

An operation module is used to obtain the access metadata of the node to be accessed through the memory address and perform data operations on the access metadata.

In addition, to achieve the above-mentioned purpose, the present application also provides an electronic device, the electronic device comprising: a memory, a processor, and a program of the metadata management method stored in the memory and executable on the processor, the program of the metadata management method can implement the steps of the metadata management method as described above when the program of the metadata management method is executed by the processor. In addition, to achieve the above-mentioned purpose, the present application also provides a computer-readable storage medium, the computer-readable storage medium stores a program for implementing the metadata management method, the program of the metadata management method can implement the steps of the metadata management method as described above when the program of the metadata management method is executed by the processor. In addition, to achieve the above-mentioned purpose, the present application also provides a computer program product, comprising a computer program, the computer program can implement the steps of the metadata management method as described above when the computer program is executed by the processor.

One or more technical solutions proposed in the present application have at least the following technical effects: the present application is applied to a server configured on a metadata management system, and a communication connection is established between the server and a memory pool on the metadata management system through a remote direct memory access (RDMA) network, so that the server can directly access the memory pool through the remote direct memory access network, so that when the server receives an access request sent by a client and obtains path information of a node to be accessed from the access request, it can call the remote direct memory access network to directly access the memory pool, so that the memory address of the node to be accessed in the memory pool can be quickly determined, so that it is convenient to obtain the access metadata of the node to be accessed through the memory address, and perform data operations on the access metadata, thereby eliminating the need to complete access through an I/O software stack composed of multiple layers such as a virtual file system, a block device file system, and a general block layer, thereby reducing access delay. Therefore, the present application does not need to complete access through an overly deep I/O software stack, and overcomes the defect of excessively high access delay caused by the overly deep input/output software stack (I/O software stack) of a traditional file system by calling a remote direct memory access network to directly access the memory pool.

BRIEF DESCRIPTION OF THE DRAWINGS

The accompanying drawings herein are incorporated in and constitute a part of the specification, illustrate embodiments consistent with the present application, and together with the description, are used to explain the principles of the present application.

In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, for ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative labor.

FIG1 is a schematic diagram of a flow chart of a first embodiment of a metadata management method of the present application;

FIG2 is a schematic diagram of a separate persistent memory architecture cluster in the metadata management method of the present application;

FIG3 is a schematic diagram of storage of a directory tree in a memory pool in the metadata management method of the present application;

FIG4 is a schematic diagram of a flow chart of a second embodiment of the metadata management method of the present application;

FIG5 is a schematic diagram of modifying the log space of the master node in the metadata management method of the present application;

FIG6 is a schematic diagram of the module structure of the metadata management device according to an embodiment of the present application;

FIG. 7 is a schematic diagram of the device structure of the hardware operating environment involved in the metadata management method in the embodiment of the present application.

The purpose, features and advantages of this application will be further described in conjunction with the embodiments and with reference to the accompanying drawings.

DETAILED DESCRIPTION

It should be understood that the specific embodiments described herein are only used to explain the technical solutions of the present application and are not used to limit the present application.

In order to better understand the technical solution of the present application, the following will be described in detail in conjunction with the drawings of the specification and the specific implementation methods. The main solution of the embodiment of the present application is: to provide a new file management system for the separated persistent memory architecture, that is, the metadata management system in the embodiment of the present application, and the metadata management system in the present application can be applied in the separated persistent memory architecture. Specifically, the metadata management system of the present application is configured with a server, a memory pool and a client. The memory pool is equipped with a large amount of persistent memory for persistent storage of file metadata, running on the memory node of the separated persistent memory architecture, and the server provides file metadata services, processes metadata read and write requests sent by the client, and returns a response, running on the computing node of the separated persistent memory architecture. The client is responsible for providing a file access interface for upper-level applications (such as applications on computers, etc.) to call, and runs on any server that can contact the server. The computing node where the server is located and the memory node where the memory pool is located are interconnected through the RDMA network, and the server is able to directly read and write the persistent memory on the memory pool through the RDMA network without the participation of the processor of the memory node. Servers can also be interconnected through the RDMA network, so that servers can directly access each other through the RDMA network without going through the server's respective processors.

To better understand this application, the following is an explanation of the separated persistent memory architecture: The separated persistent memory architecture is based on the combination of persistent memory technology and separated memory technology. In the context of "most data centers currently use traditional server architecture, where the computing, storage and network resources of independent servers are tightly bound and then interconnected through the network to achieve dynamic expansion. This traditional architecture results in data center hardware resources being able to increase or decrease only with servers as the smallest unit, and it is impossible to flexibly call a certain type of hardware resource, which may cause serious waste of certain resources when faced with businesses with significantly different resource requirements. Therefore, the storage systems of traditional data centers have shown shortcomings such as insufficient performance, low flexibility, and poor scalability, which has become a bottleneck for the further development of computer systems in the era of massive data. Breaking through the data I/O bottleneck has become one of the most urgent needs to improve the performance of computer systems today.", persistent memory technology and separated memory technology have emerged:

1. Persistent Memory Technology: Different persistent memory implementation schemes have been proposed by various research institutes in academia and industry, such as spin torque random access memory and 3D-XPoint. Persistent memory has many excellent storage properties, such as persistent storage, byte addressability, high storage density, low energy consumption, and low access latency. Like dynamic random access memory (DRAM), it can be directly mounted on the memory bus, allowing the CPU to directly address and access it through load/store instructions. Persistent memory technology enables computer systems to persist data in main memory with read and write performance close to that of DRAM. The byte addressability of persistent memory breaks the access restrictions of block devices such as SSD and HDD, and the more flexible read and write capabilities bring great optimization space for data access.

2. Disaggregated memory technology: In order to solve the problems of low resource utilization and inflexible expansion and contraction in traditional server architecture data centers, disaggregated data centers came into being. Its central idea is to classify and split the hardware resources of the data center into different hardware resource pools such as computing pool, memory pool and storage pool. Disaggregated memory (DM) technology is an important component of it. It separates DRAM from the server to form a memory pool that can be flexibly called by computing nodes, and interconnects with computing nodes through high-speed networks, such as RDMA, CXL (Compute ExpressLink) and NVMe-oF (NVMe over Fabric). At the same time, in order to avoid triggering network communication every time the computing node reads and writes memory, the computing node will be configured with a small amount of memory as a cache for the memory pool. Similarly, the memory pool will also be equipped with a small amount of computing resources for simple computing processing. Under the disaggregated memory architecture with storage and computing separation, hardware resources can be flexibly called and released, and can be expanded and contracted independently, which greatly avoids the problem of resource waste. The separation persistent memory obtained by combining the separation memory technology with the persistent memory technology breaks the performance limitations of traditional SSDs and HDDs. The separation architecture further improves resource utilization, can provide more efficient data storage services for various big data applications, and alleviate the shortcomings of traditional data center storage. The separation persistent memory architecture is a hardware improvement. If you need to access data on the hardware, you need to implement it through the software level such as the operating system. For example, the file system is a system-level application of the operating system. It provides a unified interface for applications running on the computer to access data on the storage medium. Most traditional file systems are designed for block devices such as SSDs and HDDs, such as EXT4 (Fourth extended filesystem). The access request initiated by the application will be processed by the I/O software stack composed of the virtual file system, block device file system, general block layer, I/O scheduling layer, and block device driver layer, and finally the block device can be read and written.

However, persistent memory technology has the characteristics of being byte-addressable and directly readable and writable through load/store instructions. The file system for block devices cannot give full play to the characteristics and performance of persistent memory. Furthermore, in the separated persistent memory architecture, the computing nodes and the memory pool communicate through the RDMA high-speed network. The traditional file system cannot be applied to the separated persistent memory architecture. The traditional network file system cannot take advantage of the excellent features of RDMA high-speed network zero copy, kernel bypass, and bypassing the remote CPU (Central Processing Unit). Therefore, it is difficult to give full play to the advantages of the high-performance separated persistent memory architecture. As a result, even if there is a high-performance separated persistent memory architecture, there is still a problem of high access data latency due to the overly deep I/O software stack of the traditional file system.

The present application provides a solution, so that the server on the metadata management system can directly access the memory pool on the metadata management system through the remote direct memory access network, so that when the server receives the access request sent by the client and obtains the path information of the node to be accessed from the access request, it can call the remote direct memory access network to directly access the memory pool, so that the memory address of the node to be accessed in the memory pool can be quickly determined, so that it is convenient to obtain the access metadata of the node to be accessed through the memory address, and perform data operations on the access metadata, thereby eliminating the need to complete the access through the I/O software stack composed of multiple layers such as the virtual file system, the block device file system, and the general block layer, thereby reducing the access delay. Therefore, the present application does not need to complete the access through an overly deep I/O software stack, and overcomes the defect of excessively high access delay caused by the overly deep I/O software stack of the traditional file system by calling the remote direct memory access network to directly access the memory pool.

Based on this, an embodiment of the present application provides a metadata management method, refer to Figure 1, which is a flow chart of the first embodiment of the metadata management method of the present application. In this embodiment, the metadata management method is applied to a server configured on a metadata management system, and the server establishes a communication connection with a memory pool on the metadata management system through a remote direct memory access network. The metadata management method includes steps S10 to S30: Step S10, in response to an access request sent by a client, obtaining path information of a node to be accessed from the access request;

It should be noted that the metadata management system is a system for a separated persistent memory architecture. In this embodiment, the execution subject is a server configured on the metadata management system. The server runs on a computing node in the separated persistent memory architecture. The server is used to provide metadata services. The metadata management system also includes a memory pool and a client. The memory pool runs on a memory node of the separated persistent memory architecture. The client provides a file access interface for upper-layer applications to call and runs on any server that can contact the server. The memory pool is used to store data such as file metadata, directory metadata, etc. In the metadata management system, there can be multiple servers and multiple memory pools. The server and the memory pool can communicate through the RDMA network, and the server and the server can also communicate through RDMA. The access request is used to instruct the server to provide metadata services for the client, including metadata read and write services, etc. The access request can include a write request and a read request. The node to be accessed is the node where the metadata that the client wants to read or modify is located, the path information is a plurality of path nodes spliced in sequence, the path information is characterized by the path to find the node to be accessed, and the node to be accessed is the last node spliced in the path information, for example, when the path information is 0/A/E/I, I is the last path node in the path information, and I is the node to be accessed. Exemplarily, step S10 includes: in response to an access request sent by the client, determining the path information of the node to be accessed from the access request, wherein the access request may also carry a client identifier of the client to distinguish different access clients.

In a feasible embodiment, the server and the memory pool may specifically include steps S01 to S02 through the remote direct memory access network communication connection:

Step S01, calling the UDP network, sending a connection request to the preset UDP port of the memory pool, so that the memory pool receives and sends RDMA connection information to the server according to the connection request;

Step S02: establishing a remote direct memory access network connection with the memory pool according to the received RDMA connection information.

It should be noted that the UDP network is used to provide a communication channel between the server and the memory pool when the server and the memory pool have not established a remote direct memory access network connection, so as to establish a communication connection of the RDAM network between the server and the memory pool and complete the initialization of the server. The preset UDP port is the port specified by the memory pool for listening to whether the server sends a connection request. The connection request is used to initiate a request for RDAM network connection to the memory pool. The RDMA connection information is the RDMA-related configuration information configured on the memory pool. The RDMA connection information includes data such as RDMA QP number and RDMA port for RDMA network connection. Among them, RDMA QP is a communication path in Remote Direct Memory Access, which allows direct memory access between two nodes to achieve zero-copy data transmission and low-latency interaction. Each QP (Queue Pair) contains a send queue (Send Queue) and a receive queue (Receive Queue). The RDMA QP number is used to distinguish different QPs.

Exemplarily, after receiving the RDMA connection information sent by the memory pool, the server obtains the RDMA QP number, RDMA port and other data used for RDMA network connection from the RDMA connection information, and establishes a connection with the RDMA network of the memory pool. While the server establishes a connection with the memory pool, the server can also obtain the address information of the open memory area in the memory pool and the access key Remote Key required for access. The open memory area refers to the area that can be directly accessed through the RDMA network. The server obtains and saves the address information of the open memory area and the access key required for accessing the memory area, so that when the server needs to access the data to be accessed later, the server can find the memory area in the memory pool based on the address information of the memory area, and successfully access the memory area through the access key corresponding to the memory area, thereby ensuring the security of the data and preventing the data in the memory pool from being maliciously accessed. Before the server is initialized, that is, before the server establishes an RDMA network connection with the memory pool, the memory pool also needs to be initialized. Specifically, the steps of initializing the memory pool include: the memory pool determines a memory area to which an RDMA network remote direct memory access network connection can be established, and configures an RDMA network remote direct memory access network for the memory area; and the memory pool defines a stored data structure.

Among them, the step of the memory pool determining the memory area where the RDMA network connection can be established can be: the memory pool can select any memory node as the memory area, the memory pool can select multiple memory areas, after determining the memory area, an RDMA MR can be established on the memory area, and registered on the network card to configure the RDMA network for the memory area, so as to support the server running on the computing node to directly access the memory area of the memory pool, and the memory pool can save the memory address of the memory area where the RDMA network is established and the access key corresponding to the memory area. After the memory pool is initialized, the memory pool will specify a UDP (User Datagram Protocol) port as a preset UDP port to monitor whether the preset UDP port receives a connection request sent by the server. After monitoring the connection request sent by the server, the memory pool sends the RDAM connection information to the server to establish an RDAM connection between the server and the memory pool.

The steps of defining the data structure stored in the memory pool include determining that the data structure includes FSHeader and MDBs. FSHeader represents the header of the metadata management system and is used to store global information; MDBs are data structures that store file and directory metadata. MDBs are composed of multiple MDBs (Metadata Buckets). MDB is the basic unit that the server reads from the memory pool and is also the basic unit that the server locks when modifying metadata. The MDB is composed of MDBHeader (MDB header) and multiple Hash Buckets. MDBHeader is used to store MDB status information, and HashBucket is used to store metadata. Metadata (Metadata) includes three types of metadata: RegularFile, Directory, and DirExtend. RegularFile represents general file metadata, Directory represents directory metadata, and DirExtend represents directory extension. Among them, an extended identification area can be preset in each metadata (the type of metadata can be RegularFile, Directory or DirExtend). The extended identification area is used to distinguish whether there is metadata of the DirExtend type. When the metadata extended identification area has the identification of the extended metadata bucket (extended MDB), it means that the metadata has metadata of the DirExtend type. When the metadata extended identification area is empty, it means that the metadata does not have metadata of the DirExtend type. The metadata of the DirExtend type is to solve the problem of the limited number of child nodes caused by the limited capacity of a single MDB. It can be understood that when the server creates a new child node under any directory in the memory pool, the server will calculate the hash value of the child node and place the metadata of the child node in the corresponding HashBucket in the MDB where the directory is located; if the Hash Bucket of the MDB is full, the server will create the DirExtend type metadata of the directory in a new MDB, that is, the extended metadata bucket, to indicate the extension, and place the metadata of the child node in the extended metadata bucket, and record the identifier of the extended metadata bucket in the extended identification area of the directory.

Since the memory pool supports DirExtend type metadata, when the server creates a new directory or a new file in the memory pool, it will try its best to place the new directory or new file in the same MDB as its corresponding parent directory, or in an MDB adjacent to the MDB of the parent directory. Therefore, when the server reads metadata from the memory pool, the number of network I/Os triggered for each access can be reduced. Understandably, whenever the server reads metadata from the memory pool, it uses MDB as the basic reading unit, and can quickly read back multiple MDBs with similar distances, thereby improving the success rate of finding the target metadata in the read-back MDB and avoiding frequent network I/Os, thereby reducing the number of network I/Os and achieving high-throughput and low-latency access. To better understand the present embodiment, refer to FIG. 2 , which is a schematic diagram of a separated persistent memory architecture cluster, illustrating the connection relationship between the server and the memory pool. The memory pool is distributed with many storage modules 1 for storing data, and also includes a processor 2, and is also provided with a communication component 3 for RDMA network communication. The server also has a storage module 1, multiple processors 2, and a component 3 for RDMA network communication. RDMA communication can be performed between multiple servers, and multiple servers can also perform RDMA communication with the memory pool.

Step S20, calling the RDMA network, and searching the memory address of the node to be accessed in the memory pool of the metadata management system according to the path information;

It should be noted that the memory address can provide the server with a basis for accessing the access metadata of the node to be accessed. Since an RDMA connection is established between the server and the memory pool, the server can directly access the memory pool through the RDMA network. Exemplarily, through the RDMA network, the server can send a memory access request to the memory pool, and the memory access request includes memory access information of the memory pool. The memory access information includes the memory addresses of multiple memory areas of the memory pool stored by the server and the memory keys of each memory area, so that the memory pool can determine the memory access information based on the memory access request, and determine whether there is a target memory area matching the memory access information in the memory pool. If there is a target memory area matching the memory access information in the memory pool, then determine the access right to the target memory area of the memory pool to open to the server, indicating that there is a memory area corresponding to the memory address of the memory access information in the memory pool, and the memory key sent by the server is also the same as the memory key of the memory area of the memory pool. If there is no target memory area matching the memory access information in the memory pool, then close the server's access right to the memory pool; there can be multiple target memory areas matching the memory access information. In the case where the memory pool opens access rights to the server, the remote direct memory access network is called, and the server performs step-by-step path hashing on the path information to find the memory address of the node to be accessed in the memory pool. The node that is ranked first in the path information is the root directory node, and the node that is ranked last is the node to be accessed. When the server parses the path information step by step, it starts from the root directory node and performs step-by-step parsing of the path nodes to locate the memory address of the node to be accessed in the memory pool. Among them, the step-by-step path hashing is: for each path node in the path information, the node hash value of the path node is calculated according to the node name of the path node and the hash value of the parent node of the path node, so as to find the memory address of the node to be accessed one by one through the node hash value of each path node. Among them, the path node that is ranked first in the path information is the root directory node, and the root directory node has no parent node, so the node hash value of the root directory node is preset in advance and can be 0. When the path information is 0/A/E/I, the parent node of A is the root directory node 0, the parent node of E is the path node A, and the parent node of I is the path node E. That is, in the path information, the previous path node is the parent node of the subsequent path node.

Furthermore, in a feasible embodiment, step S20 further includes steps S21 to S26:

Step S21, determining a root directory node from the path information, calling the remote direct memory access network, and reading a root metadata bucket where the root directory node is located in the memory pool;

It should be noted that the root directory node is sorted first in the path information, and the server directly reads the root metadata bucket (root MDB), so that the memory address of the node to be accessed can be directly searched in the root metadata bucket on the server. Since the server and the memory pool can communicate over a remote direct memory access network, the server can call the RDMA Read operation to read the root metadata bucket where the root directory is located from the memory pool. Among them, the RDMA Read operation is a unilateral operation, which means that the operation is mainly performed by the server, and the memory pool does not need to perform any operation. Exemplarily, the server determines the root directory node from the path information, and the server calls the RDMA Read operation to read the root metadata bucket where the root directory is located from the memory pool.

Step S22, taking the root directory node as the target node, searching the root metadata bucket for a target hash bucket that matches the target hash value of the target node, and searching the target metadata of the target node in the target hash bucket;

Step S23, when the access right of the target metadata is open to the client, obtaining the secondary path node of the target node in the path information, and calculating the secondary hash value of the secondary path node according to the target hash value and the secondary name of the secondary path node;

It should be noted that the target hash bucket is a structure for storing the target metadata of the target node, the target metadata is the metadata corresponding to the target node, the secondary path node is the node in the next order of the target node in the path information, the secondary name is the name of the secondary path node, and the secondary hash value is used to locate the storage position of the secondary path node in the root metadata bucket.

Step S24, searching the root metadata bucket for a secondary hash bucket that matches the secondary hash value, and if the secondary hash bucket contains node metadata of the secondary path node, determining whether the secondary path node is a node to be accessed;

It should be noted that the secondary hash bucket is a structure for storing secondary metadata of secondary path nodes. The node metadata is the metadata corresponding to the secondary path node. The secondary path node is the node in the next order of the target node in the path information. The secondary name is the name of the secondary path node. The secondary hash value is used to locate the storage position of the secondary path node in the root metadata bucket.

Step S25: If yes, determine the memory address according to the secondary hash value.

In this embodiment, it is determined step by step whether there is a corresponding hash bucket for each path node in the path information. The hash bucket includes a target hash bucket and a secondary hash bucket, and it is determined one by one whether there is metadata corresponding to each path node in the hash bucket corresponding to each path node. That is, in the path information, when there is a hash bucket for the path node in the front order, and the hash bucket contains metadata corresponding to the path node, and the access rights of the metadata are open to the client, the node hash value of the next path node will be calculated to determine whether there is corresponding metadata for the next path node. In this embodiment, if the secondary path node is not the node to be visited, the target node is updated to a secondary path node, the target hash value is updated to a secondary hash value, and the step of obtaining the secondary path node of the target node in the path information when the access rights of the target metadata are open to the client is returned.

Exemplarily, steps S22 to S25 include: taking the root directory node as the target node, searching the root metadata bucket for a target hash bucket that matches the target hash value of the target node, the target hash bucket containing the target metadata of the target node, and when the target node is a root directory node, the metadata corresponding to the root directory node can be found in the root metadata bucket. In the case where the access rights of the target metadata are open to the client, the secondary path node of the target node is determined in the path information, and the secondary hash value of the secondary path node is calculated according to the target hash value and the secondary name of the secondary path node; the secondary hash bucket matching the secondary hash value is searched in the root metadata bucket, and if the node metadata of the secondary path node exists in the secondary hash bucket, it is determined whether the secondary path node is a node to be visited; if the secondary path node is not a node to be visited, the target node is updated to a secondary path node, the target hash value is updated to a secondary hash value, and the process returns to step S23; if the secondary path node is a node to be visited, the secondary hash value is searched for the memory address of the node to be visited, and there is a mapping relationship between the address and the hash value, so the corresponding secondary hash bucket can be found through the secondary hash value, thereby facilitating the finding of the corresponding metadata. Among them, the step of determining whether the access rights of the target metadata are open to the client may be as follows: determining whether the access rights of the target metadata are open to the client based on the client identifier of the client carried in the access request and the permission viewing form preset in the target metadata; if the client identifier in the permission viewing form of the target metadata is a viewable type, determining that the access rights of the target metadata are open to the client; if the client identifier in the permission viewing form of the target metadata is a non-viewable type, determining that the access rights of the target metadata are not open to the client, and stopping access to the memory address of the node to be accessed.

In a feasible embodiment, after the step in step S24 in which the access right of the target metadata is open to the client, steps S241 to S243 are further included:

Step S241, if the node metadata of the secondary path node does not exist in the secondary hash bucket, determine whether the parent node of the secondary path node has an extended metadata bucket;

Step S242, when the parent node has an extended metadata bucket, read the extended metadata bucket in the memory pool through the remote direct memory access network, and search for the extended hash bucket of the secondary path node in the extended metadata bucket;

Step S243, if the node metadata of the secondary path node exists in the extended hash bucket, then when the secondary path node is a node to be accessed, the memory address is determined according to the secondary hash value.

It should be noted that the extended metadata bucket is used to store the metadata of the extended type of the parent node. In this embodiment, the parent node of the secondary path node is the target node, that is, it can be determined whether the target node has an extended metadata bucket, and the extended hash bucket is the structure of the node metadata of the secondary path node in the extended metadata bucket. In this embodiment, if the extended hash bucket exists in the extended metadata bucket, and the node metadata of the secondary path node does not exist in the extended hash bucket, the secondary path node is updated to the extended node where the extended hash bucket is located, and the step of determining whether the parent node of the secondary path node has an extended metadata bucket is returned to execute. Exemplarily, steps S241 to S243 include: if the node metadata of the secondary path node does not exist in the secondary hash bucket, it is determined whether the parent node of the secondary path node has an extended metadata bucket; if the parent node of the secondary path node does not have an extended metadata bucket, it is determined that the node to be accessed cannot be accessed in the memory pool, and the access to the node to be accessed is terminated, and feedback can be given to the client to prompt that the node metadata of the node to be accessed cannot be accessed in the memory pool. In the case where the parent node has an extended metadata bucket, the server reads the extended metadata bucket in the memory pool through an RDMA Read operation, and searches for the extended hash bucket of the secondary path node in the extended metadata bucket; if the extended hash bucket exists in the extended metadata bucket, and the node metadata of the secondary path node exists in the extended hash bucket, then the step of determining whether the secondary path node is a node to be accessed in step S24 is executed; if the extended hash bucket exists in the extended metadata bucket, and the node metadata of the secondary path node does not exist in the extended hash bucket, then the secondary path node is updated to the extended node where the extended hash bucket is located, and the step of determining whether the parent node of the secondary path node has an extended metadata bucket is returned to execute.

In combination with steps S21 to S25 and steps S241 to S243, it can be known that the path nodes in the path information are hashed step by step to search one by one in the memory pool for metadata corresponding to the path nodes. If there is any path node for which the corresponding metadata is not found, it is determined whether the parent node of the path node has an extended metadata bucket. If the parent node has an extended metadata bucket, the extended metadata bucket is read in the memory pool through a remote direct memory access network to search for metadata corresponding to the path node in the extended metadata bucket. If the parent node of the path node does not have an extended metadata bucket, access to the node to be accessed is stopped.

This embodiment makes full use of the remote direct memory access network to bypass the processor of the memory node and directly read and write the unilateral read and write capabilities of the memory in the memory pool. Most of the nodes in the traditional distributed file system cluster use remote procedure calls (RPC) to communicate, send requests to each other, and return the data after the other party completes the processing. However, due to the separated persistent memory architecture, the memory node where the memory pool is located has only a small amount of computing resources and cannot carry a large number of complex computing processes such as path resolution. If the server on the computing node directly sends the metadata request to the memory pool on the memory node, and the memory pool is allowed to perform complex computing processes, it is bound to cause the processor of the memory node to be overwhelmed. Therefore, in the embodiment of the present application, the server completes the computing operations such as path resolution, and all read and write processes are completed using RDMA unilateral read and write operations, that is, bypassing the processor of the memory node in the memory pool, which improves the speed of data reading and writing. For a better understanding of this embodiment, refer to Figure 3, which is a schematic diagram of the storage of the directory tree in the memory pool, and the process of path resolution of path information is explained. First, the content in FIG. 3 is explained. FIG. 3 includes a directory tree, and the directory tree includes nodes /, A, B, C, D, E, F, J, H, and I, where / represents the root directory node. In FIG. 3, different display forms are used to represent metadata of directory type, metadata of directory extension type, and metadata of file type, respectively. In FIG. 3, MDB0 is the root metadata bucket, and MDB1 is the extension metadata bucket. In MDB0 and MDB1, the total number of Hash Buckets is 11, and the sequence numbers of Hash Buckets are 0 to 10 from left to right. For example, in MDB0, the root node metadata of the root directory node "/" is in Hash Bucket No. 0 in MDB0, and the node metadata of A is in Hash Bucket No. 4. In MDB0, "/, A, F, B, C" correspond to metadata of directory type, "J, G" correspond to metadata of file type, in MDB1, "E, D" correspond to metadata of directory type, and "I, H" correspond to metadata of file type; A in MDB1 corresponds to metadata of directory extension type. Secondly, the path resolution process is described with reference to FIG3 : when the node to be accessed is I, the path information to be resolved is: root directory node/A/E/I; the server finds the root metadata bucket where the root directory node is located, which is MDB0, and the server finds the root node metadata corresponding to the root directory node in Hash Bucket No. 0 of MDB0 in MDB 0 according to the hash value of the root directory node (fixed as 0), and checks whether the root node metadata has open permissions to the client that initiates the access request. If there is open permissions, the server uses the hash value of the root directory node and the name of the next path node A as input, calculates the node hash value of path node A to be 4, and searches MDB0 for node metadata of path node A in Hash Bucket No. 4; and if the node metadata of path node A is open to the client, the server uses the node hash value of path node A to be 4 and the node name of the next path node E, calculates the node hash value of path node E to be 6, and searches MDB0 for node metadata of path node E in Hash Bucket No. 6; Bucket failed to search for the node metadata of path node E, and the server read back the extended metadata bucket of path node A, that is, MDB 1, from the memory pool through RDMA Read; the server searches for the metadata of path node E in Hash Bucket No. 6 of MDB 1 based on the node hash value 6 of path node E; and when the node metadata of path node E is open to the client, the server calculates the hash value of path node I as 1 based on the node hash value 6 of path node E and the name of the next path node I, and searches for the node metadata of path node I in Hash Bucket No. 1 of MDB1. When the access request type is a read request, the node metadata of path node I is returned to the client.

Step S30: obtaining access metadata of the node to be accessed through the memory address and performing data operations on the access metadata.

It should be noted that after determining the memory address corresponding to the node to be accessed, the access metadata of the node to be accessed can be read through the memory address, so as to use the read access metadata to perform data operations on the access metadata. Data operations include write operations and read operations. Write operations can include new operations and modify operations, etc. New operations can be new directory metadata, new file metadata, etc. Exemplarily, the access metadata of the node to be accessed is read from the memory address, and a read operation or a write operation is performed on the access metadata, which can be specifically determined based on the access request sent by the client. The type of access request can be a write request or a read request.

Further, referring to FIG4 , based on the above embodiment of the present application, in another embodiment of the present application, the same or similar contents as the above embodiment can be referred to the above introduction, and will not be repeated later. On this basis, referring to FIG4 , step S30 also includes steps S31 to S33:

Step S31, when the type of the access request is a write request, calling a remote direct memory access replacement operation to determine whether the lock area of the access metadata bucket where the access metadata is located is empty;

It should be noted that when the type of access request is a write request, a write operation will be performed on the access metadata. The access metadata bucket is a storage structure for storing access metadata. Each MDB has a lock area, and the access metadata bucket also has a corresponding lock area. When the lock area of the access metadata bucket is empty, it means that there is currently no server in the access metadata bucket responsible for writing the metadata in the access metadata bucket. When the lock area of the access metadata bucket is not empty, it means that there is currently a server in the access metadata bucket responsible for writing the metadata in the access metadata bucket. The remote direct memory access replacement operation is an RDMACAS operation. Exemplarily, step S31 includes: when the type of the access request is a write request, obtaining the lock area of the access metadata bucket where the access metadata is located and determining whether the lock area is empty. In this embodiment, the type of the access request can also be a read request, and step S30 can also include step S34: when the type of the access request is a read request, returning the read access metadata to the client.

Step S32: if it is empty, the server is determined to be the modification master node, and the server is determined to be the modification master node, and the master configuration information of the modification master node is written into the lock area of the access metadata bucket through a remote direct memory access replacement operation, and the write open permission in the access metadata bucket is obtained, and a write operation is performed on the access metadata in the access metadata bucket;

In this embodiment, if there are multiple servers that perform write operations on the same access metadata bucket, it is necessary to determine the modification master node and the modification slave node among the multiple servers. The write permission of the access metadata bucket is open to the modification master node. It can be understood that the modification master node can perform write operations on the metadata in the access metadata bucket, and is also used to receive the metadata expected to be modified sent by the modification slave node, and write the metadata expected to be modified by the modification slave node in the access metadata bucket. The write permission of the access metadata bucket is not directly open to the modification slave node. The modification slave node sends the metadata expected to be modified in the access metadata bucket to the modification master node, and the modification master node modifies the metadata expected to be modified in the access metadata bucket. The main configuration information is the configuration information that the modification master node needs to have when performing a write operation on the access metadata bucket. The main configuration information includes node configuration information and remote direct memory access configuration information. Exemplarily, step S32 includes, if the lock area is empty, determining that the server is a modified master node, and calling a remote direct memory access replacement operation, writing the main configuration information of the modified master node in the lock area of the access metadata bucket, obtaining write open permissions in the access metadata bucket, and performing a write operation on the access metadata in the access metadata bucket.

In a feasible implementation manner, step S32 further includes steps S321 to S324:

Step S321, writing the node configuration information of the modified master node into the lock area of the access metadata bucket through the remote direct memory access replacement operation, creating a log space in the modified master node, and determining the remote direct memory access configuration information of the log space;

It should be noted that the remote direct memory access replacement operation is an RDMA CAS (Compare-and-Swap) operation in the RDMA network. The RDMA CAS operation is used to write data in the lock area of the MDB. The node configuration information includes relevant information of the server's own node, such as modifying the identity of the master node, modifying the network address and open port of the master node, and the open port is used to provide the modification slave node with an entry to access the modification master node. The log space is used to record the data operation of modifying the metadata in the metadata bucket, and is also used to store and receive the metadata to be modified that is expected to be modified sent by the modification slave node. The log space includes an index area, a log area, and a result area. The index area is used to store an index increment. The index increment is associated with a log area and a result area. The log space can have multiple index increments. One index increment corresponds to a unique modification slave node. It can be understood that the modification master node can receive the metadata to be modified that is expected to be modified sent by multiple modification slave nodes. The remote direct memory access configuration information includes the address of the log space and the log key of the log space. The log key is the RKey (remote key) of the RDMA MR (Memory Region) configured in the log space.

Step S322, registering the remote direct memory access configuration information of the log space to the modification master node, and writing the remote direct memory access configuration information into the lock area, and obtaining the write-open permission for accessing the metadata bucket;

It should be noted that the remote direct memory access configuration information and the node configuration information are written into the lock area, so that the write permission in the access metadata bucket can be obtained. The address of the log space, the log key of the log space, and the node configuration information are written in the lock area, so that the modified slave node can obtain the network address of the modified master node from the lock area, so as to successfully access the modified master node through the network address, and also obtain the log space address and log key of the modified master node, so as to successfully access the log space of the modified master node. In this embodiment, the remote direct memory access configuration information of the log space is registered to the modified master node, so that the modified slave node can directly access the log space of the modified master node through the remote direct memory access network. Improve the access speed between servers.

Step S323: When the write permission in the access metadata bucket is obtained, a write operation is performed on the access metadata.

It should be noted that, when the server becomes the modification master node, it can directly perform write operations on the access metadata, and directly feed back the results of the write operations on the access metadata to the client. Exemplarily, the remote direct memory access replacement operation is called, the node configuration information of the modification master node is written in the lock area of the access metadata bucket, and a log space is created in the modification master node, and the remote direct memory access configuration information of the log space is determined; the remote direct memory access configuration information of the log space is registered on the network card of the modification master node, and the remote direct memory access configuration information is written in the lock area, and the write permission of the access metadata bucket is obtained; when the write permission in the access metadata bucket is obtained, the access metadata is written. After completing the write operation on the access metadata, the result of the write operation on the access metadata can be directly fed back to the client, and the modification master node can write to multiple access metadata.

In a feasible embodiment, after the step of obtaining the write open permission in the access metadata bucket in step S323, steps S324 to S325 are also included:

Step S324, monitoring whether the index area of the log space is updated, if the index area is updated, obtaining an index increment from the index area, obtaining metadata to be modified and modification information of the metadata to be modified from the log area associated with the index increment;

Step S325 , modifying the metadata to be modified in the access metadata bucket according to the modification information, and updating the modification status of the metadata to be modified in the result area associated with the index increment.

It should be noted that, when the server becomes the modification master node, it is necessary to monitor the index area of the log space of the modification master node to monitor whether the index area is updated. The update of the index area indicates that a modification slave node has accessed the log space of the modification master node, and the modification slave node has the possibility of sending the metadata to be modified that is expected to be modified to the modification master node. Therefore, when the index area update is monitored, the index increment of the index area is obtained, the metadata to be modified and its modification information are obtained from the log area associated with the index increment, and the metadata to be modified is modified in the access metadata bucket. The modification status is used to describe whether the metadata to be modified has been modified. Before the metadata to be modified is modified, the modification status can be the default status. After the metadata to be modified is modified, the modification status can be updated to a status such as completed modification or successfully modified. This embodiment does not make specific limitations here. The modification information is the content of the metadata to be modified that is expected to be modified and carried by the metadata to be modified. The log area can store multiple metadata to be modified in isolation, and the result area can also store multiple modification records of the metadata to be modified in isolation. The isolation storage order of the same metadata to be modified in the log area and the result area can be the same. Before the modification master node modifies the metadata to be modified, the modification status of the metadata to be modified displayed at the corresponding position in the result area is the default. After the modification master node successfully modifies the metadata to be modified, the modification status of the corresponding position of the metadata to be modified will be updated at the corresponding position in the result area, so as to remind the modification slave node that the metadata to be modified has been successfully modified.

Exemplarily, monitor whether the index area of the log space is updated. If the index area is updated, obtain the index increment from the index area, obtain the metadata to be modified and the modification information of the metadata to be modified from the log area associated with the index increment; modify the metadata to be modified in the access metadata bucket according to the modification information, and update the modification status of the metadata to be modified in the result area associated with the index increment. Wherein, when the index increment is updated, it means that the index area is also updated. This embodiment creates a log space that can perform remote direct memory access network communication in the modification master node, so that the modification slave node can directly access the modification master node, thereby improving the access speed without passing through the processor of each server. And open up an index area in the log space so that the modification slave node can perform index self-increment in the index area, and determine the index increment corresponding to the modification slave node, so as to facilitate the storage of the metadata to be modified of the modification slave node through the index increment and its associated log area and result area, and reduce the concurrency conflicts generated by each server.

Step S33, if it is not empty, determines that the server is a modification slave node, and obtains the master configuration information from the lock area of the access metadata bucket, establishes a communication connection of the remote direct memory access network with the modification master node according to the master configuration information, and sends the access metadata to the modification master node to perform a write operation on the access metadata through the modification master node.

It should be noted that the master configuration information is the information written into the lock area by the modification master node, and the master configuration information is used to provide the basis for the modification slave node to access the modification master node. The modification master node can send access metadata to the modification slave node, and the access metadata may include metadata to be modified, so as to modify the access metadata by modifying the master node. Exemplarily, if the lock area is not empty, it is determined that the server of the currently accessed access metadata bucket is the modification slave node, and the modification slave node fails to lock the lock area of the access metadata bucket. At this time, the master configuration information of the modification master node can be obtained from the lock area, and a communication connection of a remote direct memory access network is established with the modification master node based on the master configuration information, so that the access metadata can be sent to the modification master node through the remote direct memory access network, so as to perform a write operation on the access metadata through the modification master node.

In a feasible embodiment, step S33 further includes steps S331 to S333:

Step S331, calling a remote direct memory access self-increment operation, performing an index self-increment in the index area of the log space of the modified master node, and obtaining an index increment of the modified slave node;

Step S332, according to the index increment, writing the metadata to be modified into the log area associated with the index increment, so that the modification master node can modify the metadata to be modified in the log area;

Step S333: poll the result area of the indexed value-added association, and when it is detected that the modification status of the metadata to be modified in the result area is updated, the modification result of the metadata to be modified is fed back to the client of the modification slave node.

It should be noted that the remote direct memory access self-increment operation is an RDMA FAA operation, and the modification slave node can perform an RDMA FAA (Fetch-and-Add) operation through the remote direct memory access network to perform index self-increment in the index area of the log space of the modification master node, and obtain the index increment obtained by the index self-increment. The index increment can be a numerical value. The index increment corresponds to a unique modification slave node, and the modification slave node can correspond to multiple index increments. When the index area is updated, the index increment will also correspond to a new log area and a new result area, which are used to store the metadata to be modified of the modification slave node corresponding to the index increment and the modification status of the metadata to be modified. The modification result can be the data after the metadata to be modified is modified, or it can be the modification status of the metadata to be modified, etc. Exemplarily, the modification slave node can write the expected metadata to be modified in the log area associated with the index increment, the modification slave node can write multiple metadata to be modified in the log area, the modification master node will poll the log area, detect whether the modification slave node has newly written metadata to be modified, the modification master node will make corresponding modifications to the metadata to be modified in the log area, and the modification master node will also update the modification status in the result area corresponding to the metadata to be modified. The modification slave node will poll the result area to detect whether the modification of one or more metadata to be modified is completed. When the metadata to be modified is completed, the modification result of the metadata to be modified can be fed back to the client of the slave node to be modified. In this embodiment, the modification slave node can modify the corresponding metadata to be modified by modifying the master node, avoiding concurrency conflicts between servers.

Based on the above embodiments, it can be known that the server can autonomously coordinate concurrent write conflict methods (determine the modification of the master node and the modification of the slave node), and can efficiently complete the coordination and resolution of write conflicts without the participation of the memory pool, while ensuring that no conflicts occur and the efficiency of the write operation is not affected. Effectively avoid the waste of computing resources of the memory node. In the traditional single server architecture, both computing and storage are performed on the server. The server can receive modification requests from each end, and then coordinate the modifications and persist them uniformly. However, the separated persistent memory architecture is an architecture that separates storage and computing. The memory pool responsible for persistent storage cannot undertake the complex computing process of writing conflict resolution. The general approach is to let multiple servers compete for the same lock in the memory pool, and the server holding the lock writes its own modifications to the memory pool and then releases the lock. However, in the case of high concurrency, fierce lock competition will cause a significant increase in latency. This embodiment adopts a server-side autonomous coordination method, allowing many concurrently modified servers to autonomously confirm the modification of the master node and the modification of the slave node. The modification of the master node is responsible for collecting modifications, and the modification of the slave node submits the desired modified metadata to the modification of the master node without blocking. In this way, conflict resolution can be completed at the computing node (server) level, and low latency can be maintained under high concurrency.

To better understand this embodiment, refer to FIG5, which is a schematic diagram of modifying the log space of the master node, and explain the write operations of modifying the master node and modifying the slave node in this embodiment. First, the content shown in FIG5 is explained: the memory pool in FIG5 has an MDB, a server for modifying the master node and a server for modifying the slave node, and the index increment in the modification of the master node includes the log area, the result area, and the index area. The index area, the log area, and the result area constitute the log space of the modification of the master node. Secondly, the process of the write operation is described with reference to FIG5, including steps S1 to S5: Step S1: the modification master node locks the lock area of the MDB; Step S2: the modification slave node increments the index in the index area to obtain the index value 2 of the modified slave node; Step S3: the modification slave node writes the metadata to be modified that is expected to be modified in the log area; Step S4: the modification master node updates the modification status of the expected metadata to be modified in the result area, and the modification slave node can feedback the corresponding modification result to the client of the modified slave node after detecting that the modification status of the metadata to be modified is updated; Step S5: after the modification master node modifies the metadata to be modified that the modified slave node expects to be modified, if it is detected that the index area is not updated within the preset time, and the log area is not updated, the MDB can be unlocked to complete the concurrent modification of the MDB. Thus, the conflict caused by concurrent writing can be resolved at the computing node level, and low latency can be maintained under high concurrency.

The present application also provides a metadata management device, please refer to Figure 6, which is applied to a server configured on a metadata management system, and the server establishes a communication connection with the memory pool on the metadata management system through a remote direct memory access network. The device includes: a response module 10, which is used to respond to an access request sent by a client and obtain path information of the node to be accessed from the access request; a calling module 20, which is used to call a remote direct memory access network and search the memory address of the node to be accessed in the memory pool of the metadata management system according to the path information; an operation module 30, which is used to obtain the access metadata of the node to be accessed through the memory address and perform data operations on the access metadata. The metadata management device provided by the present application adopts the metadata management method in the above embodiment, aiming to solve the technical problem of high access delay caused by the deep input/output software stack of the traditional file system. Compared with the prior art, the beneficial effects of the metadata management method provided in the embodiment of the present application are the same as the beneficial effects of the metadata management method provided in the above embodiment, and other technical features in the metadata management device are the same as the features disclosed in the above embodiment method, which will not be repeated here.

The embodiment of the present application provides an electronic device, which can be a playback device, and the electronic device includes: at least one processor; and a memory connected to the at least one processor in communication; wherein the memory stores instructions that can be executed by at least one processor, and the instructions are executed by at least one processor so that at least one processor can execute the metadata management method in the above embodiment. Referring to FIG. 7 below, it shows a schematic diagram of the structure of an electronic device suitable for implementing the embodiment of the present disclosure. The electronic device in the embodiment of the present disclosure may include but is not limited to mobile terminals such as mobile phones, laptops, digital broadcast receivers, PDAs (personal digital assistants), PADs (tablet computers), PMPs (portable multimedia players), vehicle terminals (such as vehicle navigation terminals), etc., and fixed terminals such as digital TVs, desktop computers, etc. The electronic device shown in FIG. 7 is only an example and should not bring any restrictions to the functions and scope of use of the embodiment of the present disclosure. As shown in FIG. 7, the electronic device may include a processing device 1001 (such as a central processing unit, a graphics processor, etc.), which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 1002 or a program loaded from a storage device 1003 to a random access memory (RAM) 1004. In RAM 1004, various programs and data required for the operation of the electronic device are also stored. The processing device 1001, ROM 1002 and RAM 1004 are connected to each other via a bus 1005. An input/output (I/O) interface 1006 is also connected to the bus. Typically, the following systems can be connected to the I/O interface 1006: an input device 1007 including, for example, a touch screen, a touchpad, a keyboard, a mouse, an image sensor, a microphone, an accelerometer, a gyroscope, etc.; an output device 1008 including, for example, a liquid crystal display (LCD), a speaker, a vibrator, etc.; a storage device 1003 including, for example, a magnetic tape, a hard disk, etc.; and a communication device 1009. The communication device can allow the electronic device to communicate with other devices wirelessly or by wire to exchange data. Although the figure shows an electronic device with various systems, it should be understood that it is not required to implement or have all the systems shown. More or fewer systems may be implemented or have alternatively.

In particular, according to an embodiment of the present disclosure, the process described above with reference to the flowchart can be implemented as a computer software program. For example, an embodiment of the present disclosure includes a computer program product, which includes a computer program carried on a computer-readable medium, and the computer program includes a program code for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from the network through the communication device 1009, or installed from the storage device 1003, or installed from the ROM 1002. When the computer program is executed by the processing device 1001, the above functions defined in the method of the embodiment of the present disclosure are executed. The electronic device provided by the present application adopts the metadata management method in the above embodiment 1 to solve the technical problem of too high access delay caused by the input/output software stack of the traditional file system being too deep. Compared with the prior art, the beneficial effect of the product flow data allocation provided by the embodiment of the present application is the same as the beneficial effect of the metadata management method provided by the above embodiment, and the other technical features in the metadata management device are the same as the features disclosed in the above embodiment method, which will not be repeated here. It should be understood that the various parts of the present disclosure can be implemented with hardware, software, firmware or a combination thereof. In the description of the above implementation modes, specific features, structures, materials or characteristics may be combined in a suitable manner in any one or more embodiments or examples. The above is only a specific implementation mode of the present application, but the scope of protection of the present application is not limited thereto. Any technician familiar with the technical field can easily think of changes or replacements within the technical scope disclosed in the present application, which should be included in the scope of protection of the present application. Therefore, the scope of protection of the present application shall be based on the scope of protection of the claims. The present embodiment provides a computer-readable storage medium having computer-readable program instructions stored thereon, and the computer-readable program instructions are used to execute the metadata management method in the above-mentioned embodiment one.

The computer-readable storage medium provided in the embodiment of the present application may be, for example, a USB flash drive, but is not limited to electrical, magnetic, optical, electromagnetic, infrared, or semiconductor devices, equipment or devices, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable EPROM (Electrical Programmable Read Only Memory, read-only memory) or flash memory, an optical fiber, a portable compact disk CD-ROM (compact disc read-only memory, read-only memory), an optical storage device, a magnetic storage device, or any suitable combination of the above. In this embodiment, the computer-readable storage medium may be any tangible medium containing or storing a program, which may be used by or in combination with an instruction execution device, device or device. The program code contained on the computer-readable storage medium may be transmitted using any appropriate medium, including but not limited to: wires, optical cables, RF (Radio Frequency, radio frequency), etc., or any suitable combination of the above. The above-mentioned computer-readable storage medium may be contained in an electronic device; it may also exist alone without being assembled into an electronic device.

The computer-readable storage medium carries one or more programs. When the one or more programs are executed by the electronic device, the electronic device: in response to the access request sent by the client, obtains the path information of the node to be accessed from the access request; calls the remote direct memory access network, and searches the memory address of the node to be accessed in the memory pool of the metadata management system according to the path information; obtains the access metadata of the node to be accessed through the memory address and performs data operations on the access metadata. The computer program code for performing the operations of the present disclosure can be written in one or more programming languages or a combination thereof, and the programming languages include object-oriented programming languages such as Java, Smalltalk, C++, and conventional procedural programming languages such as "C" language or similar programming languages. The program code can be executed completely on the user's computer, partially on the user's computer, as an independent software package, partially on the user's computer and partially on the remote computer, or completely on the remote computer or server. In the case of a remote computer, the remote computer can be connected to the user computer through any type of network, including a LAN (local area network) or a WAN (Wide Area Network), or can be connected to an external computer (e.g., using an Internet service provider to connect through the Internet). The flowcharts and block diagrams in the accompanying drawings illustrate the possible architecture, functions and operations of the devices, methods and computer program products according to various embodiments of the present application. In this regard, each box in the flowchart or block diagram can represent a module, a program segment, or a part of a code, which contains one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in a different order from the order marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram and/or flowchart, and the combination of boxes in the block diagram and/or flowchart, can be implemented by a dedicated hardware-based device that performs a specified function or operation, or can be implemented by a combination of dedicated hardware and computer instructions. The modules involved in the embodiments described in the present disclosure can be implemented by software or by hardware. Among them, the name of the module does not constitute a limitation on the unit itself under certain circumstances. The computer-readable storage medium provided by the present application stores computer-readable program instructions for executing the above-mentioned metadata management method, which is intended to solve the technical problem of excessive access delay caused by the input/output software stack of the traditional file system that is too deep. Compared with the prior art, the beneficial effects of the computer-readable storage medium provided in the embodiment of the present application are the same as the beneficial effects of the metadata management method provided in the above-mentioned embodiment, and will not be repeated here. The present application also provides a computer program product, including a computer program, which implements the steps of the metadata management method as described above when the computer program is executed by a processor. The computer program product provided by the present application is intended to solve the technical problem of excessive access delay caused by the input/output software stack of the traditional file system that is too deep. Compared with the prior art, the beneficial effects of the computer program product provided by the embodiment of the present application are the same as the beneficial effects of the metadata management method provided by the above embodiment, which will not be described in detail here. The above are only preferred embodiments of the present application, and do not limit the patent scope of the present application. Any equivalent structure or equivalent process transformation made by using the contents of the specification and drawings of the present application, or directly or indirectly applied in other related technical fields, are also included in the patent processing scope of the present application.

Claims (10)

1. A metadata management method, characterized in that it is applied to a server configured on a metadata management system, the server and a memory pool on the metadata management system establish a communication connection through a remote direct memory access network, and the method comprises: In response to an access request sent by a client, obtaining path information of a node to be accessed from the access request; Calling a remote direct memory access network to search for a memory address of the node to be accessed in a memory pool of the metadata management system according to the path information; The access metadata of the node to be accessed is obtained through the memory address and data operations are performed on the access metadata. 2. The method according to claim 1, wherein the step of calling the remote direct memory access network and searching the memory address of the node to be accessed in the memory pool of the metadata management system according to the path information comprises: Determine a root directory node from the path information, call the remote direct memory access network, and read a root metadata bucket where the root directory node is located in the memory pool; Taking the root directory node as the target node, searching the root metadata bucket for a target hash bucket that matches the target hash value of the target node, and searching the target metadata of the target node in the target hash bucket; In a case where the access rights of the target metadata are open to the client, obtaining a secondary path node of the target node in the path information, and calculating a secondary hash value of the secondary path node according to the target hash value and a secondary name of the secondary path node; Searching the root metadata bucket for a secondary hash bucket that matches the secondary hash value, and if the secondary hash bucket contains node metadata of the secondary path node, determining whether the secondary path node is a node to be accessed; If so, the memory address is determined according to the secondary hash value. 3. The method of claim 2, wherein after the step of searching the root metadata bucket for a secondary hash bucket that matches the secondary hash value, the method further comprises: If the node metadata of the secondary path node does not exist in the secondary hash bucket, determining whether the parent node of the secondary path node has an extended metadata bucket; In the case where the parent node has an extended metadata bucket, reading the extended metadata bucket in the memory pool through a remote direct memory access network, and searching for an extended hash bucket of the secondary path node in the extended metadata bucket; If the node metadata of the secondary path node exists in the extended hash bucket, then when the secondary path node is a node to be accessed, the memory address is determined according to the secondary hash value. 4. The method according to claim 1, wherein the data operation comprises a write operation; and the step of performing data operation on the access metadata comprises: In the case where the type of the access request is a write request, calling a remote direct memory access replacement operation to determine whether a lock area of an access metadata bucket where the access metadata is located is empty; If it is empty, the server is determined to be a modification master node, and writes the master configuration information of the modification master node into the lock area of the access metadata bucket through a remote direct memory access replacement operation, obtains the write open permission in the access metadata bucket, and performs a write operation on the access metadata in the access metadata bucket; If it is not empty, the server is determined to be a modification slave node, and the master configuration information is obtained from the lock area of the access metadata bucket, a communication connection of a remote direct memory access network is established with the modification master node according to the master configuration information, and the access metadata is sent to the modification master node to perform a write operation on the access metadata through the modification master node. 5. The method according to claim 4, characterized in that the main configuration information includes node configuration information and remote direct memory access configuration information; the access metadata includes metadata to be modified; The steps of writing the master configuration information of the modified master node in the lock area of the access metadata bucket through remote direct memory access replacement operation, obtaining write open permission in the access metadata bucket, and performing a write operation on the access metadata in the access metadata bucket include: By means of the remote direct memory access replacement operation, the node configuration information of the modified master node is written into the lock area of the access metadata bucket, a log space is created in the modified master node, and the remote direct memory access configuration information of the log space is determined; Registering the remote direct memory access configuration information of the log space to the modification master node, and writing the remote direct memory access configuration information into the lock area, and obtaining the write-open permission for accessing the metadata bucket; When the write open permission in the access metadata bucket is obtained, a write operation is performed on the access metadata. 6. The method according to claim 5, characterized in that after the step of obtaining the write open permission in the access metadata bucket, it also includes: Monitoring whether the index area of the log space is updated, if the index area is updated, obtaining an index increment from the index area, obtaining metadata to be modified and modification information of the metadata to be modified from the log area associated with the index increment; The metadata to be modified is modified in the access metadata bucket according to the modification information, and the modification status of the metadata to be modified is updated in the result area associated with the index increment. 7. The method according to claim 4, wherein the access metadata includes metadata to be modified, and the step of sending the access metadata to the modification master node to perform a write operation on the access metadata through the modification master node comprises: Calling a remote direct memory access self-increment operation to perform index self-increment in the index area of the log space of the modified master node to obtain the index increment of the modified slave node; According to the index increment, the metadata to be modified is written into the log area associated with the index increment, so that the modification master node can modify the metadata to be modified in the log area; The result area of the indexed value-added association is polled, and when it is detected that the modification status of the metadata to be modified in the result area is updated, the modification result of the metadata to be modified is fed back to the client of the modification slave node. 8. A metadata management device, characterized in that it is applied to a server configured on a metadata management system, the server establishes a communication connection with a memory pool on the metadata management system via a remote direct memory access network, and the device comprises: A response module, used to respond to an access request sent by a client and obtain path information of a node to be accessed from the access request; A calling module, used for calling a remote direct memory access network, and searching for a memory address of the node to be accessed in a memory pool of the metadata management system according to the path information; The operation module is used to obtain the access metadata of the node to be accessed through the memory address and perform data operations on the access metadata. 9. An electronic device, characterized in that the electronic device comprises: at least one processor; and a memory communicatively coupled to the at least one processor; The memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can perform the steps of the metadata management method described in any one of claims 1 to 7. 10. A storage medium, characterized in that the storage medium is a computer-readable storage medium, and a program for implementing a metadata management method is stored on the computer-readable storage medium, and the program for implementing the metadata management method is executed by a processor to implement the steps of the metadata management method as described in any one of claims 1 to 7.