CN106777394B - a cluster file system - Google Patents

Abstract

The invention discloses a cluster file system, comprising: an lvm management module and an nfs management module; wherein, the lvm management module creates a corresponding block device according to the block device creation information and provides it to a cluster user, so that the cluster user can directly perform operations in the block device. IO operation; nfs management module, mount the nfs file system on the node where the lvm storage pool is mounted, and manage the lvm storage pool through sanlock; the cluster file system provides a block device so that the virtual device can format the block device according to the required file format Converting it into a file system for its own needs reduces the process of each IO operation; at the same time, it provides a relatively fast IO operation, improves the efficiency of the cluster file system, and improves the IO speed of the virtual device of the cluster.

Description

a cluster file system

technical field

The invention relates to the field of computer technology, in particular to a cluster file system.

Background technique

With the rapid development of computer technology and network technology, the cluster system has gradually become the focus of the computer industry with its powerful computing power and robust fault-tolerant mechanism. However, the development of the supporting cluster file system is affected by distribution, and the development is slow. Most of the cluster file systems recreate the file system format required by the virtual machine on the basis of the original file system. This type of file system deepens the process of each IO operation and reduces the access rate of the file system. Therefore, how to improve the efficiency of the cluster file system is a technical problem to be solved by those skilled in the art.

SUMMARY OF THE INVENTION

The purpose of the present invention is to provide a cluster file system, which provides a block device so that the virtual device can format the block device into a file system required by itself according to the required file format, which reduces the process of each IO operation .

In order to solve the above technical problems, the present invention provides a cluster file system, comprising: an lvm management module and an nfs management module; wherein,

The lvm management module is configured to create a corresponding block device according to the block device creation information and provide it to a cluster user, so that the cluster user can directly perform IO operations in the block device;

The nfs management module is used to mount the nfs file system on the node where the lvm storage pool is mounted, and manage the lvm storage pool through sanlock.

Optionally, the nfs management module includes:

The initialization module is used to create a corresponding lvm storage pool lock information file in the nfs file system when the lvm storage pool is created.

Optionally, the initialization module specifically executes initlockspace through sanlock when creating the lvm storage pool, and executes the add lockspace operation on each host to create a file named with the uuid of vg, so that the file has a corresponding Resource lock for the lvm storage pool.

Optionally, the nfs management module includes:

The metadata consistency unit is used for, if a metadata operation request is received, obtain the corresponding vg lock resource through sanlock, execute the corresponding metadata operation, and release the vg lock resource after the metadata operation is completed.

Optionally, the nfs management module includes:

The block device lock information creation unit is configured to create a corresponding block device lock information file in the nfs file system when a block device is created.

Optionally, the block device lock information creation unit specifically creates a file named after the uuid of the block device through sanlock when creating a block device, and executes the init resource operation through sanlock, so that the file has the Resource lock for block devices.

Optionally, the nfs management module includes:

The block device operation unit is configured to, if a block device operation request is received, acquire the corresponding lock resource through sanlock and execute the corresponding block device operation, and release the lock resource after the block device operation is completed.

Optionally, the nfs management module includes:

The block device lock information deletion unit is used to delete the block device when it is detected through sanlock that the lock resource corresponding to the block device is not occupied when a block device deletion request is received, and delete the block after the deletion is successful. The block device lock information file corresponding to the device.

Optionally, this solution also includes:

The lock information detection module is used to regularly detect the lock information in the nfs management module and the block device information in the kernel, and delete the kernel information corresponding to the deleted block device.

A cluster file system provided by the present invention includes: an lvm management module and an nfs management module; wherein, the lvm management module creates a corresponding block device according to the block device creation information and provides it to a cluster user, so that the cluster user can directly access the block device. Perform IO operations; nfs management module, mount the nfs file system on the node where the lvm storage pool is mounted, and manage the lvm storage pool through sanlock;

It can be seen that the cluster file system provides a block device, so that the virtual device can directly format the block device into the file system required by the virtual device according to the required file format, that is, the user directly performs IO operations on the block device, reducing the number of problems in the prior art. First, the steps of performing IO operations in the file system reduce the process of each IO operation; at the same time, it provides relatively fast IO operations, improves the efficiency of the cluster file system, and improves the IO speed of the virtual devices of the cluster.

Description of drawings

In order to explain the embodiments of the present invention or the technical solutions in the prior art more clearly, the following briefly introduces the accompanying drawings that need to be used in the description of the embodiments or the prior art. Obviously, the accompanying drawings in the following description are only It is an embodiment of the present invention. For those of ordinary skill in the art, other drawings can also be obtained according to the provided drawings without creative work.

FIG. 1 is a structural block diagram of a cluster file system provided by an embodiment of the present invention;

FIG. 2 is a structural block diagram of another cluster file system provided by an embodiment of the present invention.

Detailed ways

The core of the present invention is to provide a cluster file system, which provides a block device so that the virtual device can format the block device into a file system required by its own according to the required file format, which reduces the process of each IO operation .

In order to make the purposes, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments These are some embodiments of the present invention, but not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative efforts shall fall within the protection scope of the present invention.

In this embodiment, lvm is the logical volume manager, nfs (network filesystem) is the network file system, and sanlock is a lightweight and flexible distributed lock manager implemented by software. It is used in open source virtualization projects such as ovirt and libvirt. There are applications. In the prior art, the IO operation is first performed in the file system, and then the corresponding IO operation is written to the block device by the file system, that is, the process of performing the IO operation twice, which reduces the IO operation speed. In this embodiment, the block device is directly provided, so that the IO operation is directly performed in the block device, so the process of the IO operation is saved and the speed of the IO operation is improved. For details, please refer to FIG. 1. FIG. 1 is a structural block diagram of a cluster file system provided by an embodiment of the present invention. The system may include: an lvm management module 100 and an nfs management module 200; wherein,

The lvm management module 100 is configured to create a corresponding block device according to the block device creation information and provide it to a cluster user, so that the cluster user can directly perform IO operations in the block device;

Specifically, the lvm management module 100 provides a specific block device to be used by the cluster to increase the access speed of the cluster file system.

The nfs management module 200 is configured to mount the nfs file system on the node where the lvm storage pool is mounted, and manage the lvm storage pool through sanlock.

Specifically, the nfs management module 200 provides a dependency file for cluster management, and determines the storage target as a sanlock lock management through nfs. In addition, the purpose of sanlock management in the nfs management module 200 is to better suit the lvm for cluster management files, and at the same time to prevent common access by cluster members, resulting in a split-brain phenomenon. That is, the nfs file system provides the storage dependency managed by sanlock, that is, the nodes that mount the lvm storage pool need to mount the nfs file system, and the lvm storage pool is managed by sanlock to make it suitable for the cluster file system.

Further, sanlock can not only provide a lock management solution; at the same time, Sanlock can also provide a weak cluster management solution, that is, cluster lvm storage pool access node control. In order to improve the management efficiency and management reliability of the nfs management module 200 .

The system manages and controls the cluster file system through the lvm management module 100 and the nfs management module 200 . Shared readable and writable files are provided through the file system provided by nfs that cluster members can access each other, and the operation of mutually exclusive locks is provided through sanlock to prevent resources from being reused by different machines.

Based on the above technical solution, in the cluster file system provided by the embodiment of the present invention, the system provides a virtual device with a specific block file, and the virtual device formats the block device into a file system required by itself according to the required file format, Reduced the flow of each IO operation. At the same time, it provides a relatively fast IO operation, improves the efficiency of the cluster file system, and at the same time improves the IO speed of the virtual devices of the cluster.

Based on the above embodiment, the metadata of the lvm is operated on when the lvm is created, because multiple hosts can access the lvm storage pool at the same time. Therefore, in order to ensure the security of the lvm metadata, each time the cluster lvm Data operations must be mutually exclusive. Therefore, sanlock is used to ensure that the cluster's operations on metadata are mutually exclusive. Therefore, sanlock+nfs provides vg locks, that is, operations on metadata must obtain vg locks through sanlock. , then do the operation, and release that vg lock when the operation is complete. That is to say, in this embodiment, the nfs management module 200 may include:

The initialization module is used to create the corresponding lvm storage pool lock information file in the nfs file system when the lvm storage pool is created.

Specifically, when the lvm storage pool is created, the initialization module executes initlockspace through sanlock, and performs the add lockspace operation on each host to create a file named with the uuid of vg, so that the file has resources corresponding to the lvm storage pool Lock.

That is, first configure the iscsi device or FC block device, and the host can access the LUN normally, and mount an nfs file system that each host can access to each host that uses the LUN. Deploy the sanlock management software, and then execute the init lockspace operation on a host (with the uuid of the vg to be created as the lockspace name of the lock information), that is, initialize the distributed lock information, and then execute the addlockspace operation on each host , that is, perform a connection lock operation, perform lock management operations on each host; create a vg (the uuid of the above lockspace name), and then create the lock information of the vg on the nfs file system (a file named after the uuid of vg) , and then initialize.

The metadata consistency unit is used to obtain the corresponding vg lock resource through sanlock and execute the corresponding metadata operation when a metadata operation request is received, and release the vg lock resource after the metadata operation is completed.

Among them, several disk partitions are connected into a whole volume group (volume group, vg) to form an lvm storage pool.

Based on the above embodiment, in order to prevent the split-brain problem caused by simultaneous access to the cluster, the nfs management module in this embodiment may include:

The block device lock information creation unit is used to create a corresponding block device lock information file in the nfs file system when a block device is created.

Specifically, the block device lock information creation unit is, when creating a block device, create a file named after the uuid of the block device through sanlock, and execute the init resource operation through sanlock, so that the file has a resource lock of the block device.

That is, each time the operation of creating an lv (ie, a block device) must create a file named by the uuid of the lv on the nfs file system, and at the same time perform the init resource operation on the file, that is, the file is the lock information of the lv.

The block device operation unit is used for, if a block device operation request is received, obtain the corresponding lock resource through sanlock, execute the corresponding block device operation, and release the lock resource after the block device operation is completed.

The block device lock information deletion unit is used to delete the block device when the lock resource corresponding to the block device is detected by sanlock when a block device deletion request is received, and delete the block device lock corresponding to the block device after the deletion is successful. information file.

Specifically, each time the lv is used, the resource lock of the lv must be acquired, and the resource lock must be deleted at the same time when the lv is deleted. That is, for the residual data of the lv of the kernel, each operation of creating an lv must create a lock of the sanlock resource through sanlock, that is, the resource information named with the uuid of the lv, and the lock of the lv is obtained every time the lv is used. , and then it can be used normally. For the deletion operation of lv, it is necessary to check whether the lv is already occupied, that is, the lv is in use; when the lv can be deleted normally, perform the deletion operation. After the deletion is completed, the lock information of the lv must be deleted. Avoid residual information.

Based on any of the above embodiments, please refer to FIG. 2 , this embodiment may further include:

The lock information detection module is used to regularly detect the lock information in the nfs management module and the block device information in the kernel, and delete the kernel information corresponding to the deleted block device.

Specifically, other cluster member nodes need to have scheduled tasks to detect the content of the lock information and the information of the kernel lv, and delete the kernel information of the deleted lv. Make sure that system resources are not occupied.

The cluster file system provided by the present invention has been described in detail above. The principles and implementations of the present invention are described herein by using specific examples, and the descriptions of the above embodiments are only used to help understand the method and the core idea of the present invention. It should be pointed out that for those skilled in the art, without departing from the principle of the present invention, several improvements and modifications can also be made to the present invention, and these improvements and modifications also fall within the protection scope of the claims of the present invention.

Claims (7)

1. A clustered file system, comprising: the lvm management module and the nfs management module; wherein,

the lvm management module is used for creating corresponding block equipment according to the block equipment creation information and providing the block equipment for a cluster user, so that the cluster user directly performs IO operation in the block equipment;

the nfs management module is used for mounting an nfs file system on a node of a mounted lvm storage pool and managing the lvm storage pool through sanlock;

the nfs management module comprises:

a block device lock information deleting unit, configured to delete a block device when detecting, through sanlock, that a lock resource corresponding to the block device is not occupied when a block device deletion request is received, and delete a block device lock information file corresponding to the block device after the deletion is successful;

further comprising: and the lock information detection module is used for detecting the lock information in the nfs management module and the block device information in the kernel at regular time and deleting the kernel information corresponding to the deleted block device.

2. The cluster file system of claim 1, wherein the nfs management module comprises:

and the initialization module is used for creating a corresponding lvm storage pool lock information file in the nfs file system when the lvm storage pool is created.

3. The cluster file system of claim 2, wherein the initialization module is specifically configured to execute an init lockspace through sanlock when the lvm storage pool is created, and execute add lockspace operation on each host to create a file named uuid of vg, so that the file has a resource lock corresponding to the lvm storage pool.

4. The cluster file system of claim 3, wherein the nfs management module comprises:

and the metadata consistency unit is used for acquiring the corresponding vg lock resources through sanlock and then executing the corresponding metadata operation if the metadata operation request is received, and releasing the vg lock resources after the metadata operation is completed.

5. The cluster file system of claim 4, wherein the nfs management module comprises:

and the block device lock information creating unit is used for creating a corresponding block device lock information file in the nfs file system when creating the block device.

6. The cluster file system according to claim 5, wherein the block device lock information creating unit is specifically configured to create a file named uuid of the block device by sanlock when creating the block device, and perform an init resource operation by sanlock to make the file have a resource lock of the block device.

7. The cluster file system of claim 6, wherein the nfs management module comprises:

and the block device operation unit is used for acquiring the corresponding lock resource through sanlock and then executing the corresponding block device operation if the block device operation request is received, and releasing the lock resource after the block device operation is completed.

Images