Disclosure of Invention
In view of this, embodiments of the present application provide a data processing method and apparatus, so as to improve performance of a storage device while implementing concurrent reading and writing.
In order to achieve the above purpose, the technical solutions provided in the embodiments of the present application are as follows:
in a first aspect of an embodiment of the present application, a data processing method is provided, where the method includes:
receiving a plurality of operation requests sent by a client aiming at a target volume through a plurality of paths, wherein each operation request in the plurality of operation requests comprises a read request and/or a write request;
and applying for a sharing lock for any operation request in the operation requests, so that the client executes the operation request through the sharing lock.
In a specific implementation manner, when the operation request includes a write request, the method further includes:
applying for a sharing lock for the bitmap corresponding to the target volume;
and responding to the execution of the write request to write data to the target volume, and updating the information in the bitmap through the sharing lock.
In a specific implementation manner, when the operation request includes a read request, the method further includes:
and sending the reading request to the object storage equipment corresponding to the target volume so as to acquire the data to be read from the object storage equipment.
In a specific implementation, before the operation request includes a write request and before responding to the write request, the method further includes:
in response to creating the snapshot, applying for an exclusive lock;
a snapshot is created of the data object indicated by the exclusive lock bitmap.
In a specific implementation, the method further includes:
in response to failure to apply for an exclusive lock, sending a first notification message to the client, the first notification message being used to instruct the client to release the shared lock;
the exclusive lock is assigned to the create snapshot process.
In a specific implementation, the method further includes:
and responding to the completion of snapshot creation, and sending a second notification message to the client, wherein the second notification message is used for instructing the client to re-add the sharing lock.
In a specific implementation, the method further includes:
applying for an exclusive lock in response to performing a snapshot rollback and/or target volume deletion operation;
and executing snapshot rollback operation and/or target volume deleting operation after the application is successful.
In a second aspect of embodiments of the present application, there is provided a data processing apparatus, including:
the device comprises a receiving unit, a processing unit and a processing unit, wherein the receiving unit is used for receiving a plurality of operation requests sent by a client aiming at a target volume through a plurality of paths, and each operation request in the plurality of operation requests comprises a read request and/or a write request;
and the application unit is used for applying for a sharing lock for any operation request in the operation requests so that the client executes the operation request through the sharing lock.
In a third aspect of embodiments of the present application, there is provided an electronic device, including: a processor, a memory;
the memory for storing computer readable instructions or a computer program;
the processor is configured to read the computer readable instructions or the computer program to enable the apparatus to implement the data processing method according to the first aspect.
In a fourth aspect of embodiments of the present application, there is provided a computer-readable storage medium including instructions or a computer program, which when run on a computer, cause the computer to perform the data processing method of the first aspect.
In a fifth aspect of embodiments of the present application, there is provided a computer program product, which when run on a processor, implements the data processing method of the first aspect.
Therefore, the embodiment of the application has the following beneficial effects:
in the embodiment of the application, when the client performs multiple operation requests on the target volume through multiple paths, the server receives the multiple operation requests through the multiple paths. Each operation request of the plurality of operation requests may include a read request and/or a write request. Based on the received multiple operation requests, the server applies for a sharing lock for each operation request, so that the client executes the operation request through the sharing lock. That is, the server applies for a shared lock for the process corresponding to each operation request, and then concurrent reading and writing is realized through the shared lock, so that the performance of the device is improved.
Detailed Description
In order to make the technical solutions of the present application better understood, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application, and it is obvious that the described embodiments are only a part of the embodiments of the present application, and not all of the embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present application.
In order to facilitate understanding of technical solutions provided by the embodiments of the present application, technical terms related to the present application will be described first.
Block storage is a type of data storage used in storage area networks in which data is stored in blocks in volumes that are attached to nodes. The volume formed by these blocks will map into the operating system and be controlled by the file system layer. The essence of block storage is that a bare disk or similar bare disk device is mapped to a host for use, the host can format and store and read data, and the block device has a fast reading speed but does not support sharing.
The logical volume support may support exclusive lock (exclusive lock), bitmap (object map), and other features. The exclusive lock protects the consistency of the image data, and when the image is modified, the lock needs to be held. When this feature is turned on, it is ensured that only one client is accessing the image, otherwise lock contention would cause the I/O to drop dramatically. object map, dependent on exclusive lock. When the property is on, the states of all data objects of the image are recorded to mark whether the data objects really exist or not.
The Object Storage Device (OSD) is responsible for storing the actual user data (data will be stored in OSD in the form of objects); it is responsible for responding to client read requests, usually an osd daemon binds only to one disk in the ceph cluster.
Snapshot rollback refers to when some change results in a data error or data loss, the snapshot data may be rolled back to the state at the time the snapshot was created, so that the data is restored.
The inventor finds that in an iscsi multi-path scene, when an iscsi client accesses a logical volume through multiple paths, an iscsi server opens the logical volume. Due to the exclusive-lock characteristic, only one process is supported to hold a lock at the same time so as to access the logical volume, and therefore, multiple processes compete with each other to cause performance reduction. The conventional solution is to disable exclusive locks, but since bitmaps rely on exclusive locks, the bitmap feature needs to be disabled when exclusive locks are disabled, thereby affecting many of the functions of the logical volume.
Based on this, an embodiment of the present application provides a data processing method, where when a client accesses a target volume through multiple paths, a server receives multiple operation requests of the client for the target volume through the multiple paths, where the multiple operation requests include a read request and/or a write request. Aiming at a plurality of operation requests of a client to a target volume, a server applies for a shared lock for any operation request, so that a process corresponding to each operation request utilizes the shared lock to perform access and/or write operation on the target volume, other characteristics of the target volume are not affected, and the storage performance is improved.
In order to facilitate understanding of the technical solutions provided by the embodiments of the present application, the following description will be made with reference to the accompanying drawings.
Referring to fig. 1, which is a flowchart of a data processing method provided in an embodiment of the present application, as shown in fig. 1, the method may include:
s101: and receiving a plurality of operation requests sent by a client aiming at the target volume through a plurality of paths.
In this embodiment, when the client accesses the target volume on the server side through multiple paths, multiple operation requests are sent to the server side through the multiple paths. One path may correspond to one operation request, and the operation request may include a read request and/or a write request.
S102: and applying for a sharing lock for any operation request in the operation requests so that the client executes the operation request through the sharing lock.
The server applies for a sharing lock for each operation request aiming at any operation request in the operation requests, so that the client executes the operation request through the sharing lock. Specifically, a shared lock is allocated to the process executing each operation request, so that multiple processes can simultaneously access the target volume through the shared lock.
In a specific implementation manner, when the operation request includes a write request, since a data object needs to be written into the target volume and a bitmap is used for recording a data object state, a shared lock needs to be applied for the bitmap corresponding to the target volume; the information in the bitmap is updated with the shared lock in response to executing the write request to write data to the target data. That is, when a write operation is performed, in order to ensure that the written data can be judged whether the data exists through the bitmap, the bitmap is updated, the bitmap can be updated simultaneously in order to realize the write operation of multiple paths, a shared lock is applied for the bitmap corresponding to the target volume, and when the data is written, the state of the written data is added to the bitmap, so that whether the written data exists is judged through the bitmap in the following process. Specifically, before updating the bitmap, it is first determined whether the bitmap is already locked, and if not, a shared lock needs to be applied for the bitmap.
In a specific implementation manner, when the operation request includes a read request, the bitmap may be directly ignored, and the read request is sent to the object storage device osd corresponding to the target volume, so as to obtain data to be read from the object storage device.
In a specific implementation, before the operation request includes a write request and the write request is executed, a snapshot creation operation may be performed, and when the snapshot operation is executed, an exclusive lock is applied, and a snapshot is created for the data object indicated by the bitmap according to the exclusive lock. That is, when creating a snapshot for a data object in a target volume, in order to avoid the writing operation of a business process on the target volume, an exclusive lock is applied for a snapshot creating process, and then a snapshot is created for the data object in the target volume according to the state indicated by a bitmap.
Further, if the exclusive lock application based on the snapshot operation fails, a first notification message is sent to the client, and the first notification message is used for instructing the client to release the shared lock. That is, in case of failure of applying for an exclusive lock, all business processes are first informed to release their own locks, and then allocate an exclusive lock to the process creating the snapshot.
It should be noted that, for the creation, deletion, protection and de-protection of the snapshot, an exclusive lock is applied first, so as to perform the relevant operation of the snapshot by using the exclusive lock.
Further, after the snapshot creation is completed, a second notification message is sent to the client, and the second notification message is used for instructing the client to re-add the sharing lock. That is, after the snapshot is created on the data in the target volume, a second notification message may be sent to the client, so that each service process adds a shared lock again and executes each service.
In one particular implementation, an exclusive lock is applied when a snapshot rollback and/or target volume deletion operation is performed; in response to the application success, a snapshot rollback operation and/or a target volume deletion operation is performed.
It can be seen that, when the client performs multiple operation requests on the target volume through multiple paths, the server receives the multiple operation requests through the multiple paths. Each operation request of the plurality of operation requests may include a read request and/or a write request. Based on the received multiple operation requests, the server applies for a sharing lock for each operation request, so that the client executes the operation request through the sharing lock. That is, the server applies for a shared lock for the process corresponding to each operation request, and then concurrent reading and writing is realized through the shared lock, so that the performance of the device is improved.
For ease of understanding, reference is made to the frame diagram shown in FIG. 2, which includes operations on the target volume. The first part is that the client performs read-write operation on the target volume through a plurality of paths. While executing the first part, the normal connection between the object store osd is maintained through the heartbeat mechanism by making a connection between the register (watch) and the object store osd. Meanwhile, in order to ensure concurrent reading and writing, a shared lock is applied for the target volume and the bitmap object map, so that concurrent reading and writing operations are executed through the shared lock, and information in the bitmap is updated through the shared lock.
When the second part is executed, i.e. when operations are performed on the snapshot, such as snapshot creation, protection, deprotection and deletion, a connection is established with the osd through registration and an exclusive lock is applied. If the application fails, the current business process is informed to release the sharing lock if the application fails, which indicates that the business process holds the sharing lock, thereby ensuring that the exclusive lock application succeeds. After the application is successful, the exclusive lock is used to execute the operations of the snapshot, and simultaneously, the full bitmap can be loaded. Loading the full bitmap means reading all data objects corresponding to the bitmap from the target volume and loading the data objects into the value memory.
When the third part is executed, namely snapshot rollback or volume deletion operation is executed, connection is established between registration and osd, and exclusive lock is applied for the data object indicated by the bitmap.
When the fourth part is executed, namely the operation of exporting/importing/counting the capacity of the target volume is executed, the operation of loading the full bitmap is directly executed without applying any lock through home connection between the registration and the osd.
Based on the above method embodiments, the present application provides a data processing apparatus, which will be described below with reference to the accompanying drawings.
Referring to fig. 3, which is a block diagram of a data processing apparatus according to an embodiment of the present disclosure, as shown in fig. 3, the apparatus 300 may include: a receiving unit 301 and an applying unit 302.
A receiving unit 301, configured to receive multiple operation requests sent by a client for a target volume through multiple paths, where each operation request in the multiple operation requests includes a read request and/or a write request;
an applying unit 302, configured to apply for a shared lock for any operation request in the multiple operation requests, so that the client executes the operation request through the shared lock.
In a specific implementation manner, when the operation request includes a write request, the apparatus further includes: an update unit;
the application unit is further configured to apply for a shared lock for the bitmap corresponding to the target volume;
and the updating unit is used for responding to the write request to write data to the target volume and updating the information in the bitmap through the sharing lock.
In a specific implementation manner, when the operation request includes a read request, the apparatus further includes: a transmitting unit;
the sending unit is further configured to send the read request to an object storage device corresponding to the target volume, so as to obtain data to be read from the object storage device.
In a specific implementation manner, the apparatus further includes: a creating unit;
the application unit is further configured to apply for an exclusive lock in response to creating a snapshot before responding to the write request when the operation request includes a write request;
the creating unit is used for creating a snapshot for the data object indicated by the exclusive lock bitmap.
In a specific implementation manner, the apparatus further includes: a transmitting unit and a distributing unit;
the sending unit is used for sending a first notification message to the client in response to failure of applying for the exclusive lock, wherein the first notification message is used for indicating the client to release the shared lock;
an allocation unit for allocating the exclusive lock to the process of creating a snapshot.
In a specific implementation manner, the sending unit is further configured to send, in response to completion of snapshot creation, a second notification message to the client, where the second notification message is used to instruct the client to add the shared lock again.
In a specific implementation, the method further includes: an execution unit;
the application unit is further used for responding to the snapshot rollback and/or target volume deletion operation and applying for an exclusive lock;
and the execution unit is used for responding to the application success and then executing snapshot rollback operation and/or target volume deleting operation.
It should be noted that, for implementation of each unit in this embodiment, reference may be made to relevant descriptions in the foregoing method embodiments, and details of this embodiment are not described herein again.
In addition, an embodiment of the present application further provides an electronic device, including: a processor, a memory;
the memory for storing computer readable instructions or a computer program;
the processor is configured to read the computer readable instructions or the computer program to enable the apparatus to implement the data processing method.
Embodiments of the present application provide a computer-readable storage medium including instructions or a computer program, which when run on a computer, cause the computer to execute the data processing method.
The embodiment of the application provides a computer program product, and when the computer program product runs on a processor, the data processing method is realized.
It should be noted that, in the present specification, the embodiments are described in a progressive manner, each embodiment focuses on differences from other embodiments, and the same and similar parts among the embodiments may be referred to each other. For the system or the device disclosed by the embodiment, the description is simple because the system or the device corresponds to the method disclosed by the embodiment, and the relevant points can be referred to the method part for description.
It should be understood that in the present application, "at least one" means one or more, "a plurality" means two or more. "and/or" for describing an association relationship of associated objects, indicating that there may be three relationships, e.g., "a and/or B" may indicate: only A, only B and both A and B are present, wherein A and B may be singular or plural. The character "/" generally indicates that the former and latter associated objects are in an "or" relationship. "at least one of the following" or similar expressions refer to any combination of these items, including any combination of single item(s) or plural items. For example, at least one (one) of a, b, or c, may represent: a, b, c, "a and b", "a and c", "b and c", or "a and b and c", wherein a, b, c may be single or plural.
It is further noted that, herein, relational terms such as first and second, and the like may be used solely to distinguish one entity or action from another entity or action without necessarily requiring or implying any actual such relationship or order between such entities or actions. Also, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising an … …" does not exclude the presence of other identical elements in a process, method, article, or apparatus that comprises the element.
The steps of a method or algorithm described in connection with the embodiments disclosed herein may be embodied directly in hardware, in a software module executed by a processor, or in a combination of the two. A software module may reside in Random Access Memory (RAM), memory, Read Only Memory (ROM), electrically programmable ROM, electrically erasable programmable ROM, registers, hard disk, a removable disk, a CD-ROM, or any other form of storage medium known in the art.
The previous description of the disclosed embodiments is provided to enable any person skilled in the art to make or use the present application. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the generic principles defined herein may be applied to other embodiments without departing from the spirit or scope of the application. Thus, the present application is not intended to be limited to the embodiments shown herein but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.