WO2018081960A1 - File management method, file system, and server system - Google Patents

Abstract

A file management method, a file system, and a server system. The method comprises: receiving a file write request; determining to-be-stored data of the file and metadata of the file according to the file write request; determining multiple data fragments of the to-be-stored data of the file; determining local metadata of each of the multiple data fragments; storing each of the multiple data fragments, the local metadata of each of the multiple data fragments and the metadata of the file into a data region; and storing the metadata of the file into a metadata region. By means of the solution, metadata can be effectively protected.

Description

Method of managing files, file system and server system Technical field

The present invention relates to the field of information technology and, more particularly, to a method, a file system, and a server system for managing files.

Background technique

A distributed file system is a relatively complex system, and the data it manages can be simplified into two clusters of data and metadata. Data is the real data that users see, basically referring to the real content of the file. Metadata, also known as "data data", is mainly the directory structure, attribute information of files and directories, and mapping relationships between directories. Metadata is used to implement functions such as quickly finding data, and is internal management information of the file system.

In the current file system mechanism, if you want to find a file or find its information, you should start from the root directory to find a file, and then open the file. The entire hierarchical relationship is recorded by metadata. If the metadata cluster crashes, even if the data is not corrupted, we can't organize the data anymore. We can't associate the data with its directory or even the file name, and we can't implement the file search function.

Currently, in order to ensure the reliability of the metadata cluster, each file system adopts various complicated logical or physical overheads. For example, a metadata backup scheme is adopted to ensure the reliability of the metadata, and some important metadata or all metadata in the distributed file system is backed up by using a database or another cluster; or, by using a log or The way snapshots are recorded, the changes in metadata are recorded, and when there is a problem, it can be played back or directly rolled back to the metadata at the most recent point in time to reduce the lost content. The above solution is more complicated or expensive, and therefore requires a more concise metadata protection mechanism.

Summary of the invention

Embodiments of the present invention provide a method, a file system, and a server system for managing files, which can effectively protect metadata.

The first aspect provides a method for managing a file in a file system, where the storage area managed by the file system includes a data area and a metadata area; the method includes:

Receiving a write file request;

Determining data to be stored of the file and metadata of the file according to the write file request;

Determining a plurality of data segments of the file to be stored;

Determining local metadata of each of the plurality of data segments;

Each of the plurality of data fragments, the local metadata of each of the plurality of data fragments, and the metadata of the file are stored in the data area;

The metadata for the file is stored in the metadata area.

The method for managing a file according to the embodiment of the present invention stores metadata of a file in a data area managed by the file system, in addition to storing metadata of the file in a metadata area managed by the file system, so that when a failure occurs in the metadata area, Metadata is restored through the data area, so that metadata can be effectively protected.

In some possible implementations, storing each of the plurality of data fragments, the local metadata of each of the plurality of data fragments, and the metadata of the file into the data area, include:

Expanding a local metadata area in the data area corresponding to each of the plurality of data fragments;

And storing each of the plurality of data fragments into a corresponding data fragment area in the data area;

And storing the local metadata of each of the plurality of data fragments into the pre-expansion sub-area in the corresponding local metadata area;

The metadata of the file is stored into an extended sub-region in a local metadata region corresponding to each of the plurality of data segments.

In some possible implementations, the extended sub-area is 512 bytes.

In some possible implementations, storing each of the plurality of data fragments, the local metadata of each of the plurality of data fragments, and the metadata of the file into the data area, include:

And storing each of the plurality of data fragments into a corresponding data fragment area in the data area;

And storing local metadata of each of the plurality of data fragments into a corresponding first sub-area in the local metadata area in the data area;

The metadata of the file is stored to a second sub-region in the local metadata region of the data region corresponding to each of the plurality of data segments.

In some possible implementations, the data fragmentation, the local metadata of the data fragmentation, and the metadata of the file are downloaded together.

The method for managing a file in the embodiment of the present invention stores the metadata of the file and the local metadata of the data fragment into the local metadata area together in the data area, and the added I/O overhead is small, and therefore, can be small. Overhead improves the reliability of metadata.

In some possible implementations, the method further includes:

When the metadata area fails, the metadata area is restored according to the metadata of the file in the data area.

In some possible implementations, the metadata area is restored according to the metadata of the file in the data area, including:

Reading metadata of a file in a local metadata area corresponding to each data slice in the data area;

The directory hierarchy relationship in the metadata area is restored according to the metadata of the read file.

In some possible implementation manners, when a failure occurs in the metadata area, the data fragmentation list may be scanned, and the metadata of the file in each data fragment local metadata area is read according to the data fragmentation list.

In some possible implementations, if the metadata of the file is stored in the last 512 bytes of the local metadata area, the metadata of the file in the 512 bytes is read.

In some possible implementations, the metadata of the file is scanned only by scanning the local metadata area in the data area.

In the metadata area recovery process of the embodiment of the present invention, only a specific area in the data area is scanned, so the recovery process is relatively simple.

In some possible implementations, each time a metadata of a file is obtained, a record is added at the directory level of the metadata area.

In some possible implementations, a hash table may be used to record record relationships that are not currently concatenated.

In some possible implementations, determining a plurality of data fragments of the file to be stored in the file includes:

The plurality of data fragments are determined according to the protection mode of the file system.

In some possible implementations, the protection mode of the file system is an EC protection mode.

In a second aspect, a file system is provided, the storage area managed by the file system including data a zone and a metadata zone; the file system includes: a file management unit, a data management unit, and a metadata management unit;

The file management unit is configured to receive a write file request, determine, according to the write file request, data to be stored of the file and metadata of the file, and determine a plurality of data fragments of the data to be stored of the file; The fragment and the metadata of the file are sent to the data management unit; the metadata of the file is sent to the metadata management unit;

The data management unit is configured to determine local metadata of each of the plurality of data fragments; each of the plurality of data fragments, each of the plurality of data fragments The local metadata of the slice and the metadata of the file are stored in the data area;

The metadata management unit is configured to store the metadata of the file into the metadata area.

The file system of the embodiment of the present invention stores the metadata of the file in the data area managed by the file system, in addition to storing the metadata of the file in the metadata area managed by the file system, so that when the metadata area fails, the data can be passed. The area recovers metadata so that metadata can be effectively protected.

In some possible implementation manners, the data management unit is specifically configured to:

Expanding a local metadata area in the data area corresponding to each of the plurality of data fragments;

And storing each of the plurality of data fragments into a corresponding data fragment area in the data area;

And storing the local metadata of each of the plurality of data fragments into the pre-expansion sub-area in the corresponding local metadata area;

The metadata of the file is stored into an extended sub-region in a local metadata region corresponding to each of the plurality of data segments.

In some possible implementation manners, the data management unit is specifically configured to:

And storing each of the plurality of data fragments into a corresponding data fragment area in the data area;

And storing local metadata of each of the plurality of data fragments into a corresponding first sub-area in the local metadata area in the data area;

The metadata of the file is stored to a second sub-region in the local metadata region of the data region corresponding to each of the plurality of data segments.

In some possible implementations, when a failure occurs in the metadata area,

The data management unit is further configured to read metadata of the file in the data area, and send metadata of the read file to the metadata management unit;

The metadata management unit is further configured to recover the metadata area according to the metadata of the file sent by the data management unit.

In some possible implementations, the data management unit is configured to: read metadata of a file in a local metadata area corresponding to each data segment in the data area, and send metadata of the read file to The metadata management unit;

The metadata management unit is specifically configured to restore the directory hierarchy relationship in the metadata area according to the metadata of the file sent by the data management unit.

In some possible implementations, the file management unit is specifically configured to determine the multiple data fragments according to the protection mode of the file system.

In a third aspect, a server system is provided, comprising:

a file system in any of the possible implementations of the second aspect or the second aspect above;

A storage resource that includes a storage area managed by the file system.

In a fourth aspect, a server system is provided, including:

a storage resource, where the storage resource includes a storage area managed by a file system, where the storage area includes a data area and a metadata area;

The management unit is configured to perform the method in any of the above possible implementation manners of the first aspect or the first aspect.

DRAWINGS

In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings used in the embodiments of the present invention will be briefly described below. The drawings in the following description are only some embodiments of the present invention, and may also be based on these. The drawings obtain other figures.

FIG. 1 is a schematic diagram of a server system to which the technical solution of the embodiment of the present invention is applicable.

2 is a block diagram of a system in accordance with an embodiment of the present invention.

FIG. 3 is a schematic flowchart of a method for managing a file according to an embodiment of the present invention.

4 is a flow chart of a write file in accordance with an embodiment of the present invention.

FIG. 5 is a flow chart of restoring metadata according to an embodiment of the present invention.

Figure 6 is a schematic block diagram of a file system in accordance with an embodiment of the present invention.

Figure 7 is a schematic block diagram of a server system in accordance with one embodiment of the present invention.

Figure 8 is a schematic block diagram of a server system in accordance with another embodiment of the present invention.

detailed description

The technical solutions in the embodiments of the present invention are clearly and completely described in the following with reference to the accompanying drawings in the embodiments of the present invention. It is obvious that the described embodiments are a part of the embodiments of the present invention, but not all embodiments. All other embodiments obtained based on the embodiments of the present invention are intended to fall within the scope of the present invention.

The technical solutions of the embodiments of the present invention can be applied to various file systems, including a distributed file system and a non-distributed file system.

In the embodiment of the present invention, the metadata of the file is additionally saved in the data area in addition to the metadata area. The data area and the metadata area may belong to different fault domains, such as at different logical locations, or at different physical locations. Therefore, the protection of the metadata of the file can be enhanced. For example, the data fragmentation, the local metadata of the data fragment, and the metadata of the file are sent to a storage medium (such as a hard disk, a solid state hard disk, etc.) in the same IO request. As each data slice is stored in the data area, metadata for a file can also be stored in the data area. Therefore, the more the number of data fragments, the more times the metadata is backed up.

Since the metadata of the file is downloaded along with the data fragmentation, the method has a very small impact on system performance, and can greatly improve the reliability of the metadata of the file.

FIG. 1 is a schematic diagram of a server system to which the technical solution of the embodiment of the present invention is applicable.

As shown in FIG. 1, server system 100 can include at least one server 110, each server 110 including a storage resource 101, such as a disk, solid state drive, or other storage resource, to provide storage space. The server 110 may further include a management unit 102, and implements a function of the file system through the management unit 102 to provide an access service of the file system.

For a distributed file system, the data of the file can be stored in the storage resource 101 of any server 110 in a distributed manner; for the non-distributed file system, the data of the file is stored in the storage resource 101 of the corresponding server 110.

Each server 110 can be a data and/or metadata node, i.e., can store only data or metadata, as well as both data and metadata. The area of all servers 110 for storing data constitutes a data area managed by the file system, and the area of all servers for storing metadata constitutes a metadata area managed by the file system. The data area and the metadata area are logically separated, physically located on the same node (such as a server), or on different nodes.

It should be understood that in various embodiments of the present invention, various management units may be functional modules implemented by a processor, for example, may be programs running in a processor; various management units may also be hardware, for example, Is one or more processors. The various storage resources may be hardware, such as a disk, a solid state hard disk, or a volatile memory; various storage resources may also be logical storage spaces, such as logic generated for virtualizing physical storage resources and managed by the management unit. storage. For the sake of brevity, the details are not described below.

FIG. 2 shows a logical system architecture diagram of an embodiment of the present invention.

As shown in FIG. 2, system 200 includes a metadata cluster 210 and a data cluster 220.

The metadata cluster 210 and the data cluster 220 may be physically dispersed in different physical nodes, or may share the same physical nodes, but are logically divided.

The metadata cluster 210 includes a metadata area 211 for storing metadata of files. The metadata area 211 can be located on one or more nodes. The metadata cluster 210 also includes one or more metadata management units 212 for managing metadata for files in the metadata cluster. Similarly, one or more metadata management units 212 may also be located on one or more nodes.

The data cluster 220 includes a data area 221 for storing metadata of files in addition to data for storing files and local metadata. Local metadata can also be called local file system metadata, including information such as the size and creation time of data fragments.

The metadata management unit 212 is a functional unit and can be located in a distributed file system composed of a single server or a plurality of servers. The metadata management unit 212 may be software, for example, distributed, may be composed of distributed software running on multiple servers; or may be hardware, for example, a processor composed of multiple servers, which can run The distributed software. The metadata area 211 accepts the management of the metadata management unit 212 and provides a storage space for the data read/written by the metadata management unit 212. The metadata area 211 may be a physical storage medium such as a hard disk or a solid state hard disk of a plurality of servers; or may be a logical storage space such as a storage space logically composed of a plurality of servers.

The data management unit 222 and the data area 221 in the data cluster 220 are similar to the metadata management unit 212 and the metadata area 211, respectively, and are not described herein again.

Alternatively, the metadata of the local metadata and the file may be stored in one piece in the local metadata area in the data area 221. For example, the local metadata area can be extended, and the extended area is called an extended sub-area of the local metadata area, and the local metadata is still stored in the original area, which is called the sub-area before the expansion, and the metadata of the file is stored in Expand in the sub-area. In addition, if the local metadata area The space provided by the domain is sufficient to store the metadata of the local metadata and files, or the local metadata area has been extended in advance, and the metadata of the local metadata and the file can be stored separately in the local metadata area. In the sub-areas, that is, the first sub-area and the second sub-area respectively stored in the local metadata area, for example, the first sub-area may be a predetermined byte in the local metadata area, and the second sub-area may be The next predetermined byte in the local metadata area.

Similarly, data area 221 can be located on one or more nodes. For example, each node provides a portion of storage space to collectively form a data area 221. The data cluster 220 also includes one or more data management units 222 for managing data for files in the data cluster, local metadata, and metadata for the files. Similarly, one or more data management units 222 can also be located on one or more nodes. For example, a plurality of nodes all run data management software, and the whole of these softwares constitutes a data management unit 222.

In some implementations, metadata management unit 212 and data management unit 222 are functional units in a file system. For example, it may be a functional unit in the management unit 102 in FIG. It should be understood that the functional units in the file system may also include other functional units for managing the files, which are not limited by the present invention.

The metadata area 211 and the data area 221 are storage areas managed by the file system.

The storage area managed by the file system and the file system can be set in the server system, for example, the server system 100 in FIG.

FIG. 3 is a schematic flowchart of a method 300 for managing files according to an embodiment of the present invention.

In the embodiment of the present invention, the storage area managed by the file system includes a data area and a metadata area, for example, the data area 221 and the metadata area 211 in FIG.

The method 300 may be performed by a file system in a server system or a server system, for example, may be performed by a management unit in a server system or a file system, and the management unit may specifically be one or more management units. For convenience of description, the following describes a server system as an example.

310, receiving a write file request.

When writing a file, the client sends a write file request to the server system. The write file request may include identification information of the file, information of the data to be stored and the access address, and the like.

The data to be stored of the file is the real data of the file, that is, the data carrying the content of the file.

The identification information of the file indicates which file is to be written, for example, the directory name and the file name.

The information of the access address indicates the address to be stored in the data to be stored, for example, specifically The length of the address and address space.

320. Determine, according to the write file request, data to be stored of the file and metadata of the file.

Upon receiving the write file request, the server system determines the data to be stored of the file and the metadata of the file. The data to be stored in the file can be directly carried in the write file request; the metadata of the file can be generated according to the information in the write file request, which is equivalent to being carried in the write file request indirectly. The metadata for this file includes information such as the directory structure of the file, the attributes of the files and directories, and so on.

330. Determine a plurality of data fragments of the file to be stored.

Optionally, the plurality of data fragments may be determined according to a protection mode of the file system.

Determining the data fragment of the data to be stored of the file may logically divide the data to be stored into a plurality of data fragments. For example, an erasure coding (EC) protection mode can be used to divide a file into multiple data fragments.

It should be understood that the present invention does not limit the protection mode, that is, other protection modes may also be employed.

Optionally, the foregoing steps 310-330 may be specifically performed by a file management unit in the system, and then the file management unit sends the plurality of data fragments and the metadata of the file to the data management unit in the system.

340. Determine local metadata of each of the plurality of data fragments.

When writing a file, you need to process two types of data, one is the data to be stored in the file, and the other is the metadata of the file. The data to be stored in the file is divided into data fragments and stored in the data area managed by the file system. In addition, each data fragment has corresponding local metadata, and the local metadata includes information such as the size and creation time of the data fragment. Local metadata is also stored in the data area, and can be stored locally in the local metadata area of the data area.

After obtaining a plurality of data fragments of the file, the server system determines local metadata for each data slice. Specifically, if there is no corresponding local metadata before, local metadata is generated according to the new data fragment; if local metadata is already present, the local metadata is modified according to the new data fragment.

350. Store, in each of the plurality of data fragments, local metadata of each of the plurality of data fragments and metadata of the file into the data area.

In this step, each data slice, the local metadata of each data slice, and the metadata of the file are stored in a data area.

Optionally, the local metadata of each data slice and the metadata of the file may be stored in one piece into a local metadata area in the data area.

Specifically, the data area may also be divided into two parts, one part for storing data fragments, which may be referred to as a data fragment area, and the other part for storing metadata, which may be referred to as a local metadata area. It is to be understood that the above-described two regions may be other embodiments, and the specific description is not to be construed as limiting the invention.

In the embodiment of the present invention, the metadata of the file and the local metadata of each data slice are stored in a local metadata area. The local metadata area in the embodiment of the present invention is large compared to the case of storing only local metadata of data fragments. In other words, the technical solution of the embodiment of the present invention uses a larger local metadata area or extends the local metadata area.

Optionally, as an embodiment of the present invention, a local metadata area in the data area corresponding to each data fragment of the plurality of data fragments may be expanded;

And storing each of the plurality of data fragments into a corresponding data fragment area in the data area;

And storing the local metadata of each of the plurality of data fragments into the pre-expansion sub-area in the corresponding local metadata area;

The metadata of the file is stored into an extended sub-region in a local metadata region corresponding to each of the plurality of data segments.

Specifically, in this embodiment, the local metadata area corresponding to the data fragment is expanded, and the metadata of the file is stored in the extended sub-area in the local metadata area. That is to say, the local metadata is still stored in the original area, and the metadata of the file is stored in the extended sub-area.

For example, the local metadata area corresponding to each data slice can be extended by 512 bytes, and the metadata of the file is filled in the extended area.

It should be understood that the above 512 bytes are only examples, and should not be construed as limiting the present invention. The specific implementation may be reasonably set according to the size of the metadata of the file.

Optionally, as another embodiment of the present invention, each of the plurality of data fragments may be stored into a corresponding data fragment area in the data area;

And storing local metadata of each of the plurality of data fragments into a corresponding first sub-area in the local metadata area in the data area;

The metadata of the file is stored to a second sub-region in the local metadata region of the data region corresponding to each of the plurality of data segments.

Specifically, in the present embodiment, the local metadata area can be set large enough in advance, so that the metadata of the file can be directly stored to a specific sub-area in the local metadata area.

For example, the local metadata area can be divided into two sub-areas, the former sub-area stores local metadata, and the latter sub-area stores metadata of the file.

In the specific implementation, before the data fragmentation, the local metadata of the data fragmentation and the metadata of the file are downloaded, the corresponding data may be processed in the cache, and then the disk is written down to the disk. In this way, the metadata of the file and the local metadata are stored in the local metadata area without increasing the I/O overhead.

Optionally, the above steps 340 and 350 may be specifically performed by a data management unit in the system. Then, the data management unit sends a write success message to the file management unit, the write success message indicating that the data fragmentation and the metadata of the file are successfully written. The file management unit then transmits the metadata of the file to the metadata management unit in the system to request the metadata management unit to update the metadata of the file.

360, storing the metadata of the file into the metadata area.

The server system updates the metadata of the file in the metadata area of the file system management. Optionally, the step may be performed after the data area is successfully written, or may be performed in parallel with the write processing of the data area, which is not limited by the present invention.

Optionally, the foregoing step 360 may be specifically performed by a metadata management unit in the system. Then, the metadata management unit sends a write success message to the file management unit, the write success message indicating that the metadata of the file is successfully written. The file management unit then sends a file write success message to the client.

The method for managing a file according to the embodiment of the present invention stores metadata of a file in a data area managed by the file system, in addition to storing metadata of the file in a metadata area managed by the file system, so that when a failure occurs in the metadata area, Metadata is restored through the data area, so that metadata can be effectively protected.

Further, the method for managing a file in the embodiment of the present invention stores the metadata of the file and the local metadata of the data fragment into the local metadata area in the data area, and the added I/O overhead is small, so A small overhead increases the reliability of metadata.

The storage process is described above, and the metadata recovery process is described below.

Optionally, as shown in FIG. 3, the method 300 may further include:

370. When the metadata area fails, the metadata area is restored according to metadata of the file in the data area.

Since the metadata of the file is also stored in the data area, when the metadata area fails, the metadata of the metadata area can be restored by scanning the metadata of the data area.

Specifically, the local metadata area corresponding to each data fragment in the data area can be read. The metadata of the file; based on the metadata of the read file, restore the directory hierarchy in the metadata area.

For example, when a failure occurs in the metadata area, the data fragment list may be scanned, and the metadata of the files in each data fragment local metadata area is read according to the data fragmentation list. For example, if the metadata of the file is stored in the last 512 bytes of the local metadata area, the metadata of the file in the 512 bytes is read.

Because the data area is fully available, you only need to scan the local metadata area in the data area when scanning the metadata for the file.

Each time you get the metadata for a file, a record is added to the directory hierarchy of the metadata area. Alternatively, a hash table may be used to record record relationships that are not currently concatenated. After all the records are collected, the directory hierarchy of the metadata area is established.

Optionally, the scanning process may be performed by the data management unit, and the metadata of the read file is sent to the metadata management unit; the directory hierarchy relationship establishing process may be performed by the metadata management unit, and after the establishment is completed, A recovery success message is sent to the file management unit, indicating that the metadata area has been restored.

As can be seen from the above technical solution, in the metadata area recovery process of the embodiment of the present invention, only a specific area in the data area is scanned, so the recovery process is relatively simple.

The technical solution of the embodiment of the present invention is described in detail from the perspective of each management unit in conjunction with FIG. 4 and FIG. It should be understood that each management unit in FIG. 4 and FIG. 5 is a functionally divided management unit, which may be physically disposed on the same node or on different nodes. In addition, the mutual relationship and function of each management unit in FIG. 4 and FIG. 5, and the relationship with other components in the system, etc., can be referred to the description in the foregoing various embodiments.

4 is a flow chart of writing a file according to an embodiment of the present invention.

401. The client sends a write file request.

The client sends a write file request to the server system and sends it to the file management unit. The write file request may include identification information of the file, information of the data to be stored and the access address, and the like.

402. The file management unit determines data to be stored of the file and metadata of the file.

The metadata for this file includes information such as the directory structure of the file, the attributes of the files and directories, and so on.

403. The file management unit determines a plurality of data fragments of the file to be stored.

Optionally, the file may be divided into multiple data fragments according to a protection mode of the file system, such as an EC protection mode.

404. The file management unit sends the plurality of data fragments and the metadata of the file to the data management unit.

405. The data management unit writes the received data into the cache.

That is, the data fragment and the metadata of the file are written to the cache.

406. The data management unit determines local metadata.

The local metadata includes information such as the size of the data slice, the creation time, and the like. The data management unit determines local metadata of each data slice according to the plurality of data fragments.

407. The data management unit fills in the metadata of the file in the local metadata area.

For example, the data management unit expands the local metadata area of each data slice by 512 bytes, and fills in the metadata of the file in the extended area; or, the data management unit is in the back 512 of the pre-expanded local metadata area. Fill in the metadata of the file in bytes.

408. The data management unit discards the data fragmentation, the local metadata of the data fragment, and the metadata of the file.

In this way, the data management unit stores the data fragmentation, the local metadata of the data fragmentation, and the metadata of the file together in the data area.

409. The data management unit responds to the file management unit data fragmentation and the metadata of the file is successfully written.

410. The file management unit sends a request to the metadata management unit to request metadata of the updated file.

411. The metadata management unit stores the metadata of the file into the metadata area.

412. The metadata management unit returns a metadata write success message to the file management unit.

413. The file management unit returns a file write success message to the client.

FIG. 5 is a flow chart of restoring metadata according to an embodiment of the present invention.

501. The data management unit scans the data fragment list.

When a failure occurs in the metadata area, the metadata of the metadata area is restored according to the metadata of the data area. The data management unit scans the data slice list, and for each data slice, the following operations are performed.

502. The data management unit reads the metadata of the file in each of the fragment local metadata areas according to the data fragmentation list.

For example, if the metadata of the file is stored in the last 512 bytes of the local metadata area, the metadata of the file in the 512 bytes is read.

Because the data area is fully available, you only need to scan the local metadata area in the data area when scanning the metadata for the file.

503. The data management unit sends the metadata of the read file to the metadata management unit.

504. The metadata management unit restores the directory hierarchy relationship according to the metadata of the received file.

Each time the metadata management unit receives the metadata of a file, it adds a record at the directory level of the metadata area. Optionally, a hash table may be used to record record relationships that are not currently connected in series. After all the records are collected, the directory hierarchy of the metadata area is established.

505. The metadata management unit sends a recovery success message to the file management unit.

After the metadata area is restored, the metadata management unit notifies the file management unit that the metadata area has been restored, so that the file service is restored.

It should be understood that, in various embodiments of the present invention, the size of the sequence numbers of the above processes does not mean the order of execution, and the order of execution of each process should be determined by its function and internal logic, and should not be taken to the embodiments of the present invention. The implementation process constitutes any limitation.

It should be understood that the specific examples of the embodiments of the present invention are only to be understood by those skilled in the art.

The method of managing files according to an embodiment of the present invention is described in detail above, and a file system and a server system according to an embodiment of the present invention will be described below.

FIG. 6 shows a schematic block diagram of a file system 600 in accordance with an embodiment of the present invention.

The storage area managed by the file system 600 includes a data area and a metadata area. As shown in FIG. 6, the file system 600 can include a file management unit 610, a data management unit 620, and a metadata management unit 630.

The file management unit 610 is configured to receive a write file request, determine, according to the write file request, data to be stored of the file and metadata of the file, and determine a plurality of data fragments of the file to be stored; The data fragment and the metadata of the file are sent to the data management unit 620; the metadata of the file is sent to the metadata management unit 630.

The data management unit 620 is configured to determine local metadata of each of the plurality of data fragments, and slice each of the plurality of data fragments, and each of the plurality of data fragments The local metadata of the slice and the metadata of the file are stored in the data area.

The metadata management unit 630 is configured to store the metadata of the file into the metadata area.

The file system of the embodiment of the present invention stores the metadata of the file in the data area managed by the file system, in addition to storing the metadata of the file in the metadata area managed by the file system, so that when the metadata area fails, the data can be passed. The area recovers metadata so that metadata can be effectively protected.

Optionally, in an embodiment of the present invention, the data management unit 620 is specifically configured to:

Expanding a local metadata area in the data area corresponding to each of the plurality of data fragments;

And storing each of the plurality of data fragments into a corresponding data fragment area in the data area;

And storing the local metadata of each of the plurality of data fragments into the pre-expansion sub-area in the corresponding local metadata area;

The metadata of the file is stored into an extended sub-region in a local metadata region corresponding to each of the plurality of data segments.

Optionally, in an embodiment of the present invention, the data management unit 620 is specifically configured to:

And storing each of the plurality of data fragments into a corresponding data fragment area in the data area;

And storing local metadata of each of the plurality of data fragments into a corresponding first sub-area in the local metadata area in the data area;

The metadata of the file is stored to a second sub-region in the local metadata region of the data region corresponding to each of the plurality of data segments.

Optionally, in an embodiment of the present invention, when the metadata area fails,

The data management unit 620 is further configured to read the metadata of the file in the data area, and send the metadata of the read file to the metadata management unit 630;

The metadata management unit 630 is further configured to recover the metadata area according to the metadata of the file sent by the data management unit 620.

Optionally, in an embodiment of the present invention, the data management unit 620 is specifically configured to: read metadata of a file in a local metadata area corresponding to each data fragment in the data area, and read the file. Metadata is sent to the metadata management unit 630;

The metadata management unit 630 is specifically configured to restore the directory hierarchy relationship in the metadata area according to the metadata of the file sent by the data management unit 620.

Optionally, in an embodiment of the present invention, the file management unit 610 is specifically configured to determine the multiple data fragments according to the protection mode of the file system.

The file system 600 of the embodiment of the present invention may perform the method for managing files according to the embodiment of the present invention, and the foregoing and other operations and/or functions of the respective units in the file system 600 may be referred to the foregoing in order to implement the corresponding processes of the foregoing respective methods. The description in the various embodiments, for the sake of brevity, I will not repeat them here.

FIG. 7 shows a schematic block diagram of a server system 700 in accordance with one embodiment of the present invention. As shown in FIG. 7, the server system 700 can include:

The file system 600 of the foregoing embodiment of the present invention;

A storage resource 710 includes a storage area managed by the file system 600.

For a detailed description of the file system 600 of the embodiment of the present invention, reference may be made to the foregoing embodiments, and details are not described herein for brevity.

FIG. 8 shows a schematic block diagram of a server system 800 in accordance with another embodiment of the present invention. As shown in FIG. 8, the server system 800 can include:

a storage resource 810, which includes a storage area managed by a file system, the storage area including a data area and a metadata area;

The management unit 820 is configured to:

Receiving a write file request;

Determining data to be stored of the file and metadata of the file according to the write file request;

Determining a plurality of data segments of the file to be stored;

Determining local metadata of each of the plurality of data segments;

Each of the plurality of data fragments, the local metadata of each of the plurality of data fragments, and the metadata of the file are stored in the data area;

The metadata for the file is stored in the metadata area.

In the server system of the embodiment of the present invention, in addition to storing the metadata of the file in the metadata area managed by the file system, the metadata of the file is also stored in the data area managed by the file system, so that when the metadata area fails, the data can be passed. The area recovers metadata so that metadata can be effectively protected.

Optionally, in an embodiment of the present invention, the management unit 820 is specifically configured to:

Expanding a local metadata area in the data area corresponding to each of the plurality of data fragments;

And storing each of the plurality of data fragments into a corresponding data fragment area in the data area;

And storing the local metadata of each of the plurality of data fragments into the pre-expansion sub-area in the corresponding local metadata area;

Storing the metadata of the file to a local area corresponding to each of the plurality of data fragments An extended sub-area in the metadata area.

Optionally, in an embodiment of the present invention, the management unit 820 is specifically configured to:

And storing each of the plurality of data fragments into a corresponding data fragment area in the data area;

And storing local metadata of each of the plurality of data fragments into a corresponding first sub-area in the local metadata area in the data area;

The metadata of the file is stored to a second sub-region in the local metadata region of the data region corresponding to each of the plurality of data segments.

Optionally, in an embodiment of the present invention, the management unit 820 is further configured to:

When the metadata area fails, the metadata area is restored according to the metadata of the file in the data area.

Optionally, in an embodiment of the present invention, the management unit 820 is specifically configured to:

Reading metadata of a file in a local metadata area corresponding to each data slice in the data area;

The directory hierarchy relationship in the metadata area is restored according to the metadata of the read file.

Optionally, in an embodiment of the present invention, the management unit 820 is specifically configured to:

The plurality of data fragments are determined according to the protection mode of the file system.

The management unit 820 in the server system 800 of the embodiment of the present invention may perform the various processes in the foregoing method embodiments. For the specific description, reference may be made to the foregoing embodiments, and details are not described herein for brevity.

It should be understood that in the embodiment of the present invention, the term "and/or" is merely an association relationship describing an associated object, indicating that there may be three relationships. For example, A and/or B may indicate that A exists separately, and A and B exist simultaneously, and B cases exist alone. In addition, the character "/" in this article generally indicates that the contextual object is an "or" relationship.

Those of ordinary skill in the art will appreciate that the elements and algorithm steps of the various examples described in connection with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of both, for clarity of hardware and software. Interchangeability, the composition and steps of the various examples have been generally described in terms of function in the above description. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the solution. A person skilled in the art can use different methods for implementing the described functions for each particular application, but such implementation should not be considered to be beyond the scope of the present invention.

A person skilled in the art can clearly understand that, for the convenience and brevity of the description, the specific working process of the system, the device and the unit described above can refer to the corresponding process in the foregoing method embodiment, and details are not described herein again.

In the several embodiments provided by the present application, it should be understood that the disclosed systems, devices, and methods may be implemented in other manners. For example, the device embodiments described above are merely illustrative. For example, the division of the unit is only a logical function division. In actual implementation, there may be another division manner, for example, multiple units or components may be combined or Can be integrated into another system, or some features can be ignored or not executed. In addition, the mutual coupling or direct coupling or communication connection shown or discussed may be an indirect coupling or communication connection through some interface, device or unit, or an electrical, mechanical or other form of connection.

The units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, may be located in one place, or may be distributed to multiple network units. Some or all of the units may be selected according to actual needs to achieve the objectives of the embodiments of the present invention.

In addition, each functional unit in each embodiment of the present invention may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above integrated unit can be implemented in the form of hardware or in the form of a software functional unit.

The integrated unit, if implemented in the form of a software functional unit and sold or used as a standalone product, may be stored in a computer readable storage medium. Based on such understanding, the technical solution of the present invention contributes in essence or to the prior art, or all or part of the technical solution may be embodied in the form of a software product stored in a storage medium. A number of instructions are included to cause a computer device (which may be a personal computer, server, or network device, etc.) to perform all or part of the steps of the methods described in various embodiments of the present invention. The foregoing storage medium includes: a U disk, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disk, and the like. .

The above is only the specific embodiment of the present invention, but the scope of the present invention is not limited thereto, and any equivalent person can be easily conceived within the technical scope of the present invention by any person skilled in the art. Modifications or substitutions are intended to be included within the scope of the invention. Therefore, the scope of protection of the present invention should be determined by the scope of the claims.

Claims (19)

A method for managing a file in a file system, wherein the storage area managed by the file system includes a data area and a metadata area; and the method includes: Receiving a write file request; Determining data to be stored of the file and metadata of the file according to the write file request; Determining a plurality of data segments of the file to be stored; Determining local metadata of each of the plurality of data segments; And storing, in the data area, each data fragment of the plurality of data fragments, local metadata of each of the plurality of data fragments, and metadata of the file; Metadata of the file is stored to the metadata area. The method according to claim 1, wherein said each of said plurality of data segments is sliced, said local metadata of each of said plurality of data segments, and said The metadata of the file is stored in the data area, including: Expanding a local metadata area in the data area corresponding to each of the plurality of data fragments; And storing each of the plurality of data fragments into a corresponding data fragment area in the data area; And storing local metadata of each of the plurality of data fragments into an extended pre-region in the corresponding local metadata area; Metadata of the file is stored to an extended sub-region in a local metadata region corresponding to each of the plurality of data segments. The method according to claim 1, wherein said each of said plurality of data segments is sliced, said local metadata of each of said plurality of data segments, and said The metadata of the file is stored in the data area, including: And storing each of the plurality of data fragments into a corresponding data fragment area in the data area; Storing local metadata of each of the plurality of data fragments into a corresponding first sub-area in a local metadata area in the data area; Metadata of the file is stored to a second sub-region in a local metadata region in the data region corresponding to each of the plurality of data segments. Method according to any one of claims 1 to 3, characterized in that the method Also includes: When the metadata area fails, the metadata area is restored according to metadata of files in the data area. The method according to claim 4, wherein the recovering the metadata area according to the metadata of the file in the data area comprises: Reading metadata of a file in a local metadata area corresponding to each data slice in the data area; The directory hierarchy relationship in the metadata area is restored according to the metadata of the read file. The method according to any one of claims 1 to 5, wherein the determining a plurality of data fragments of the file to be stored of the file comprises: The plurality of data fragments are determined according to a protection mode of the file system. A file system, wherein the storage area managed by the file system comprises a data area and a metadata area; the file system comprises: a file management unit, a data management unit and a metadata management unit; The file management unit is configured to receive a write file request, determine, according to the write file request, data to be stored of the file and metadata of the file, and determine a plurality of data fragments of the file to be stored; Transmitting the plurality of data fragments and the metadata of the file to the data management unit; and transmitting the metadata of the file to the metadata management unit; The data management unit is configured to determine local metadata of each of the plurality of data fragments; and slice each of the plurality of data fragments into the plurality of data fragments Local metadata of each data slice and metadata of the file are stored to the data area; The metadata management unit is configured to store metadata of the file to the metadata area. The file system according to claim 7, wherein said data management unit is specifically configured to: Expanding a local metadata area in the data area corresponding to each of the plurality of data fragments; And storing each of the plurality of data fragments into a corresponding data fragment area in the data area; And storing local metadata of each of the plurality of data fragments into an extended pre-region in the corresponding local metadata area; Metadata of the file is stored to an extended sub-region in a local metadata region corresponding to each of the plurality of data segments. The file system according to claim 7, wherein said data management unit is specifically configured to: And storing each of the plurality of data fragments into a corresponding data fragment area in the data area; Storing local metadata of each of the plurality of data fragments into a corresponding first sub-area in a local metadata area in the data area; Metadata of the file is stored to a second sub-region in a local metadata region in the data region corresponding to each of the plurality of data segments. A file system according to any one of claims 7 to 9, wherein when the metadata area fails, The data management unit is further configured to read metadata of a file in the data area, and send metadata of the read file to the metadata management unit; The metadata management unit is further configured to recover the metadata area according to metadata of the file sent by the data management unit. The file system according to claim 10, wherein the data management unit is configured to read metadata of a file in a local metadata area corresponding to each data slice in the data area, and read the data. The metadata of the fetched file is sent to the metadata management unit; The metadata management unit is specifically configured to restore a directory hierarchy relationship in the metadata area according to the metadata of the file sent by the data management unit. The file system according to any one of claims 7 to 11, wherein the file management unit is specifically configured to determine the plurality of data fragments according to a protection mode of the file system. A server system, comprising: A file system according to any one of claims 7 to 12; A storage resource, the storage resource including a storage area managed by the file system. A server system, comprising: a storage resource, where the storage resource includes a storage area managed by a file system, where the storage area includes a data area and a metadata area; Management unit for: Receiving a write file request; Determining data to be stored of the file and metadata of the file according to the write file request; Determining a plurality of data segments of the file to be stored; Determining local metadata of each of the plurality of data segments; And storing, in the data area, each data fragment of the plurality of data fragments, local metadata of each of the plurality of data fragments, and metadata of the file; Metadata of the file is stored to the metadata area. The server system according to claim 14, wherein the management unit is specifically configured to: Expanding a local metadata area in the data area corresponding to each of the plurality of data fragments; And storing each of the plurality of data fragments into a corresponding data fragment area in the data area; And storing local metadata of each of the plurality of data fragments into an extended pre-region in the corresponding local metadata area; Metadata of the file is stored to an extended sub-region in a local metadata region corresponding to each of the plurality of data segments. The server system according to claim 14, wherein the management unit is specifically configured to: And storing each of the plurality of data fragments into a corresponding data fragment area in the data area; Storing local metadata of each of the plurality of data fragments into a corresponding first sub-area in a local metadata area in the data area; Metadata of the file is stored to a second sub-region in a local metadata region in the data region corresponding to each of the plurality of data segments. The server system according to any one of claims 14 to 16, wherein the management unit is further configured to: When the metadata area fails, the metadata area is restored according to metadata of files in the data area. The server system according to claim 17, wherein said management unit is specifically configured to: Reading metadata of a file in a local metadata area corresponding to each data slice in the data area; The directory hierarchy relationship in the metadata area is restored according to the metadata of the read file. The server system according to any one of claims 14 to 18, wherein the management unit is specifically configured to: The plurality of data fragments are determined according to a protection mode of the file system.

Images