CN119441244B - Data processing method, device, equipment and medium based on distributed optimistic lock - Google Patents
Data processing method, device, equipment and medium based on distributed optimistic lockInfo
- Publication number
- CN119441244B CN119441244B CN202510026372.XA CN202510026372A CN119441244B CN 119441244 B CN119441244 B CN 119441244B CN 202510026372 A CN202510026372 A CN 202510026372A CN 119441244 B CN119441244 B CN 119441244B
- Authority
- CN
- China
- Prior art keywords
- data
- version number
- optimistic lock
- distributed
- database
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/23—Updating
- G06F16/2308—Concurrency control
- G06F16/2336—Pessimistic concurrency control approaches, e.g. locking or multiple versions without time stamps
- G06F16/2343—Locking methods, e.g. distributed locking or locking implementation details
-
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/23—Updating
- G06F16/2365—Ensuring data consistency and integrity
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02D—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN INFORMATION AND COMMUNICATION TECHNOLOGIES [ICT], I.E. INFORMATION AND COMMUNICATION TECHNOLOGIES AIMING AT THE REDUCTION OF THEIR OWN ENERGY USE
- Y02D10/00—Energy efficient computing, e.g. low power processors, power management or thermal management
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Data Mining & Analysis (AREA)
- Databases & Information Systems (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Computer Security & Cryptography (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
本申请公开了一种基于分布式乐观锁的数据处理方法、装置、设备及介质，涉及分布式系统技术领域，包括：当接收到数据写入请求时，从缓存中获取分布式乐观锁；将数据写入请求中携带的请求版本号与分布式乐观锁记录的乐观锁版本号进行匹配；若匹配成功，则更新分布式乐观锁记录的乐观锁版本号，并在更新完成之后，释放分布式乐观锁；从数据写入请求中获取目标写入数据，并根据目标写入数据更新结构化数据库中的数据库数据和缓存中的缓存数据。本申请能够维护缓存数据和数据库数据的一致性。
The present application discloses a data processing method, apparatus, device, and medium based on distributed optimistic locking, relating to the field of distributed system technology. The method comprises: upon receiving a data write request, obtaining a distributed optimistic lock from a cache; matching the request version number carried in the data write request with the optimistic lock version number of the distributed optimistic lock record; if the match is successful, updating the optimistic lock version number of the distributed optimistic lock record, and releasing the distributed optimistic lock after the update is complete; obtaining target write data from the data write request, and updating database data in a structured database and cache data in a cache according to the target write data. The present application can maintain the consistency of cached data and database data.
Description
Technical Field
The present application relates to the field of distributed systems, and in particular, to a data processing method, apparatus, device, and medium based on distributed optimistic lock.
Background
Multi-version concurrency control (MVCC, multi-Version Concurrency Control) in MySQL is a typical way to implement optimistic locks. It allows read operations to be completed without waiting for write operations to complete, thereby improving the concurrency performance of the system. To support this mechanism, a version field is typically added to the table structure in the database design to identify version information for each record. However, in a distributed system environment, a cache layer may exist between a client and a server in a distributed system due to the communication between multiple nodes and the collaboration between different services involved. If the cache is not updated in time, the client side can read out the outdated data, and the problem of inconsistent data between the cache and the database is caused. Thus, optimistic locks that rely on the database level are prone to cache and database data inconsistencies.
The foregoing is provided merely for the purpose of facilitating understanding of the technical solutions of the present application and is not intended to represent an admission that the foregoing is prior art.
Disclosure of Invention
The application mainly aims to provide a data processing method, device, equipment and medium based on a distributed optimistic lock, aiming at solving the technical problem that cache and database data inconsistency are easy to occur when the optimistic lock is applied.
In order to achieve the above object, the present application provides a data processing method based on a distributed optimistic lock, applied to a current thread, the data processing method based on the distributed optimistic lock includes:
when a data writing request is received, acquiring a distributed optimistic lock from a cache;
matching a request version number carried in the data writing request with an optimistic lock version number of the distributed optimistic lock record;
if the matching is successful, updating the optimistic lock version number of the distributed optimistic lock record, and releasing the distributed optimistic lock after the updating is completed;
And acquiring target write data from the data write request, and updating database data in a structured database and cache data in the cache according to the target write data.
In an embodiment, the step of updating the optimistic lock version number of the distributed optimistic lock record comprises:
Determining key value pair data corresponding to the distributed optimistic lock, wherein a primary key of the key value pair data is a device identifier, and the value of the key value pair data records an optimistic lock version number;
Extracting an optimistic lock version number from the key pair data;
increasing the optimistic lock version number based on a preset version update rule to obtain an update version number;
and storing the updated version number as an optimistic lock version number in key value pair data corresponding to the distributed optimistic lock.
In one embodiment, the step of updating database data in the structured database and cache data in the cache according to the target write data includes:
Analyzing the target writing data and determining the table name, the field and the condition to be written;
constructing a database update statement based on the table name, the field and the condition;
updating database data in the structured database based on the database update statement, incrementally updating the version number corresponding to the version number of the database corresponding to the version number field of each database data, and determining the updated version number of each database data;
And updating the cache data in the cache according to the updated database data, and incrementally updating the corresponding cache version number of the version number field of each database data to determine the updated cache version number of each database data, wherein the optimistic lock version number, the cache version number and the database version number have the same initial value.
In one embodiment, the step of obtaining the distributed optimistic lock from the cache when a data write request is received further comprises:
Acquiring a preset synchronous lock, wherein when a thread acquires the preset synchronous lock, only the current thread is allowed to execute operations of acquiring a distributed optimistic lock and updating an optimistic lock version number of a record of the distributed optimistic lock;
The step of updating the optimistic lock version number of the distributed optimistic lock record and releasing the distributed optimistic lock after the updating is completed further comprises:
Releasing the preset synchronous lock.
In an embodiment, the data processing method based on the distributed optimistic lock further includes:
When a data reading request is received, acquiring a distributed optimistic lock from the cache;
Matching a request version number carried in the data reading request with the updated optimistic lock version number of the distributed optimistic lock record;
if the matching is successful, acquiring target data corresponding to each data reading request from the cached data;
if the matching fails, returning matching failure information.
In an embodiment, after the step of obtaining the target data corresponding to each data read request from the cached data, the method further includes:
And if the target data does not exist in the cached data, acquiring target data corresponding to each data reading request from the structured database, and updating the cached data in the cache according to each target data.
In addition, to achieve the above object, the present application also proposes a data processing apparatus based on a distributed optimistic lock, including:
the lock taking module is used for obtaining the distributed optimistic lock from the cache when a data writing request is received;
The matching module is used for matching the request version number carried in the data writing request with the optimistic lock version number of the distributed optimistic lock record;
The updating module is used for updating the optimistic lock version number of the distributed optimistic lock record if the matching is successful, and releasing the distributed optimistic lock after the updating is completed;
And the writing module is used for acquiring target writing data from the data writing request, and updating database data in the structured database and cache data in the cache according to the target writing data.
Furthermore, to achieve the above object, the present application also proposes a data processing device based on a distributed optimistic lock, comprising a memory, a processor and a computer program stored on said memory and executable on said processor, said computer program being configured to implement the steps of the data processing method based on a distributed optimistic lock as described above.
Furthermore, to achieve the above object, the present application also proposes a storage medium being a computer-readable storage medium, having stored thereon a computer program which, when executed by a processor, implements the steps of a distributed optimistic lock-based data processing method as described above.
Furthermore, to achieve the above object, the present application also provides a computer program product comprising a computer program which, when executed by a processor, implements the steps of a distributed optimistic lock based data processing method as described above.
The method comprises the steps of obtaining a distributed optimistic lock from a cache when a data writing request is received, matching a request version number carried in the data writing request with an optimistic lock version number of a distributed optimistic lock record, updating the optimistic lock version number of the distributed optimistic lock record if the matching is successful, releasing the distributed optimistic lock after the updating is completed, obtaining target writing data from the data writing request, and updating database data in a structured database and cache data in the cache according to the target writing data. The application sets optimistic locks in the cache, but not in the structured database, improves concurrency performance and reading and writing speed, ensures that updating is allowed only when the version number carried by the request is consistent with the version number recorded currently through a version number matching mechanism, effectively prevents the problem of data coverage caused by that a plurality of threads modify the same data item at the same time, and maintains the consistency of the data.
Drawings
The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the application and together with the description, serve to explain the principles of the application.
In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings that are required to be used in the description of the embodiments or the prior art will be briefly described below, and it will be obvious to those skilled in the art that other drawings can be obtained from these drawings without inventive effort.
FIG. 1 is a flowchart of a data processing method based on a distributed optimistic lock according to an embodiment of the present application;
FIG. 2 is a schematic flow chart of a data processing method based on a distributed optimistic lock according to a second embodiment of the present application;
FIG. 3 is a flowchart of a second embodiment of a distributed optimistic lock-based data processing method according to the present application;
FIG. 4 is a block diagram of a distributed optimistic lock-based data processing apparatus according to an embodiment of the present application;
FIG. 5 is a schematic diagram of a device architecture of a hardware operating environment involved in a distributed optimistic lock-based data processing method according to an embodiment of the present application.
The achievement of the objects, functional features and advantages of the present application will be further described with reference to the accompanying drawings, in conjunction with the embodiments.
Detailed Description
It should be understood that the specific embodiments described herein are merely illustrative of the technical solution of the present application and are not intended to limit the present application.
For a better understanding of the technical solution of the present application, the following detailed description will be given with reference to the drawings and the specific embodiments.
It should be noted that, the execution body of the embodiment may be a computing service device with functions of data processing, network communication and program running, such as a tablet computer, a personal computer, a mobile phone, etc., or an electronic device, a thread, etc. capable of implementing the above functions. The present embodiment and the following embodiments will be described below by taking the current thread as an example.
Based on this, an embodiment of the present application provides a data processing method based on a distributed optimistic lock, and referring to fig. 1, fig. 1 is a flowchart of a first embodiment of the data processing method based on a distributed optimistic lock according to the present application.
In this embodiment, the data processing method based on the distributed optimistic lock includes steps S10 to S40:
Step S10, when a data writing request is received, a distributed optimistic lock is acquired from a cache.
It should be noted that, in the distributed system, when the application program receives a data writing request, it allocates the request to an available thread for processing, and the thread calls an API (Application Programming Interface ) provided by a Redis (Remote Dictionary Server, remote dictionary service) client library to attempt to acquire the lock. Each thread is independent in processing write requests, and each attempts to acquire a lock without interfering with each other. Even if multiple threads attempt to acquire locks on the same resource at the same time, only the first successful thread can continue to perform the update operation, and other threads may be blocked or require retries. The Redis is an open-source, high-performance memory database, which is usually used as a data structure server, and all data are stored in the memory, so that the Redis provides extremely high read-write speed, and is suitable for application scenes requiring quick response.
It is first necessary to determine whether there is a lock key in the cache that corresponds to the target resource. The lock key is typically composed of a resource ID (Identity document, identification number) and a version number. Redis provides SETNX (Set if Not Exists) commands that set key-value pairs only when a key is not present. If return 1 indicates that the lock was successfully acquired, and if return 0 indicates that the lock has been held by other clients. By acquiring the lock, it can be ensured that only one thread can modify the database at the same time, and data collision or coverage caused by simultaneous updating of a plurality of threads is prevented.
Step S20, the request version number carried in the data writing request is matched with the optimistic lock version number of the distributed optimistic lock record.
It should be noted that, the request version number is extracted from the data writing request, and the version number is usually obtained together when the data is read, and is sent to the server side together with the subsequent writing request for data updating. An optimistic lock version number of the distributed optimistic lock record is obtained from the cache, the version number reflecting the latest state of the current resource. The request version number is compared to the optimistic lock version number by a strict numerical value or string. If the two are equal, the versions are considered to be matched, otherwise, the versions are not matched. If the version numbers match, then the update operation is allowed to be performed and the version number is incremented to reflect the new state, and if the version numbers do not match, then the update request is denied, typically with an error returned to the client prompting the user to retry or take other action.
If there is a mismatch between version numbers, the optimistic lock version numbers may change due To other threads having updated the same resource, or the requested version number read by the client may be out of date due To network delay or other factors, and if the lock has a TTL (Time To Live) set, it may be automatically released due To a timeout, resulting in other threads acquiring the lock and updating the version number.
By matching the version numbers, the updating is allowed only when the request version number carried by the request is consistent with the currently recorded optimistic lock version number, and the problem of data coverage caused by that a plurality of threads modify the same data item at the same time is effectively prevented. The version number mechanism is not only used for controlling concurrent access, but also can help track historical changes of data, and is convenient for audit and rollback operations.
Step S30, if the matching is successful, updating the optimistic lock version number of the distributed optimistic lock record, and releasing the distributed optimistic lock after the updating is completed.
It should be noted that updating the version number of the optimistic lock and releasing the optimistic lock are both performed by the current thread. The optimistic lock version number of the distributed optimistic lock record is updated to reflect the latest state of the resource. The version number is incremented after each successful update, ensuring that each version is unique, thereby maintaining data consistency and historical traceability.
By updating the version number and then releasing the lock, it is ensured that only one thread can modify a particular resource at the same time, avoiding the problem of data coverage caused by simultaneous updating of multiple threads.
In a possible implementation manner, in step S30, steps a10 to a40 may be included:
and step A10, determining key value pair data corresponding to the distributed optimistic lock, wherein a main key of the key value pair data is a device identifier, and the value of the key value pair data records the optimistic lock version number.
It should be noted that the distributed optimistic lock exists in the form of a key-value pair data, e.g., id_1, (id_1+v_2), where the primary key id_1 is the device identifier and the value (id_1+v_2) is a combination of the device identifier and the optimistic lock version number.
Step a20, extracting the optimistically locked version number from the key pair data.
Step A30, increment the optimistic lock version number based on a preset version update rule to obtain an update version number.
It should be noted that the most common case of the preset version update rule is to simply add 1 to second_version. This rule applies to most scenarios because it can effectively track the number of changes in a resource.
And step A40, storing the updated version number as an optimistic lock version number in key value pair data corresponding to the distributed optimistic lock.
It should be noted that, for the original distributed optimistic lock, the key value needs to be replaced by the updated update version number to the original optimistic lock version number in the data, so as to update the distributed optimistic lock.
In a possible embodiment, step S10 may be performed while step S01 is further included:
In step S01, a preset synchronization lock is acquired, where, in the case where the thread acquires the preset synchronization lock, only the current thread is allowed to execute the operations of acquiring the distributed optimistic lock and updating the optimistic lock version number of the distributed optimistic lock record.
It should be noted that, in this embodiment, the preset synchronization lock may be implemented by synchronized. synchronized may be applied to a method or code block that restricts access to shared resources by multiple threads, only one thread may enter the synchronized protected method or code block at a time, and the other threads must wait until the current thread releases the lock.
Then, after step S30, step S02 is further included:
Step S02, releasing the preset synchronous lock.
In Java, when a thread leaves a synchronized protected method or code block, the Java virtual machine automatically releases the lock held by the thread. The synchronized method locks on the current instance object, and the synchronized code block may designate any object as the lock object. Once the thread leaves these methods or code blocks, the lock is automatically released.
In the distributed system or the high concurrency environment, the method acquires a preset synchronous lock before the thread tries to execute the key operation, so that only one thread can execute the operations at the same time, and the thread safety and the data consistency can be further ensured.
Step S40, obtaining target writing data from the data writing request, and updating database data in the structured database and cache data in the cache according to the target writing data.
Specifically, the target data to be updated is parsed from the data writing request, and typically, the request includes the data item to be updated and its version number information. According to the analyzed target data, a corresponding SQL (Structured Query Language ) update statement is constructed, and all relevant changes are ensured to be either all successful or all rolled back. The version number is incremented to reflect the latest data state while the structured database is updated. Immediately after updating the structured database, the latest data is read from the database instead of directly using the data in the write request, which can ensure that the data in the cache is up to date, avoiding data inconsistencies due to concurrent updates. Updating the latest data into the cache, and ensuring that the cache is kept synchronous with the persistence layer.
It will be appreciated that if the cache is updated before the structured database is updated, partial updates may occur for some reason (e.g., network failure, server crash), i.e., only the cache is updated but the database is not updated, or only the database is updated but the cache is not updated. Therefore, the structured database is always updated first and then the cache is updated, and even if the cache is not updated, the data consistency of the persistent layer is not affected. Because the version number matching mechanism is adopted, each update is ensured to be carried out based on the latest data state, and the problem of concurrency conflict caused by multiple threads which almost try to update the same resource does not occur.
Specifically, the cache contains a distributed optimistic lock, the distributed optimistic lock is key value pair data, a primary key of the key value pair data is a device identifier, and the value of the key value pair data records an optimistic lock version number.
It should be noted that the distributed optimistic lock is a concurrency control mechanism to ensure that multiple clients update the same data in a high concurrency environment without collision. It tracks changes to the data by way of a version number or other identifier, allowing the system to check whether other transactions have modified the data before performing an update operation. A device identifier for uniquely identifying each device, the optimistic lock version number recording a version number used by the optimistic lock.
The cache also comprises a plurality of cache data, the cache data is a cache of database data, each cache data comprises a version number field, and the version number field of the cache data is used for recording a cache version number.
It should be noted that, the cache data is a copy of the data in the structured database, and is used for accelerating the reading operation, and the cache version number records the version of the cache data, so as to ensure that the version of the cache data is consistent with the data of the structured database. The cache version number ensures that the data in the cache is up-to-date and consistent with the data in the structured database.
The structured database comprises a plurality of database data, each database data comprises a version number field, the version number field of the database data is used for recording the version number of the database, and the optimistic lock version number, the cache version number and the initial value of the version number of the database are the same.
It should be noted that the database data stores the actual business data, and includes a plurality of fields, such as id, name, value, etc., and the database version number records the current version of the record, which is used to support the optimistic locking mechanism. By incrementing the database version number, it is ensured that each update is based on the latest data version, preventing concurrency conflicts.
The initial values of the request version number, the cache version number, and the database version number are the same, typically 1 or other specified starting value. The optimistic lock version number is incremented when a distributed optimistic lock in the cache is requested to write data, ensuring that only one thread is allowed to perform a data write operation, the cache version number is incremented when data is cached in the cache, ensuring consistency with the structured database, and the database version number is incremented when each update in the structured database, ensuring that each update is based on the latest version of data.
In a possible implementation manner, step S40 may include steps B10 to B40:
And step B10, analyzing the target writing data and determining the table name, the field and the condition which need to be written.
It should be noted that, a user submits an update request through some interface, carries data to be updated, and typically sends the data in JSON (JavaScript Object Notation, javaScript, object notation) or XML (Extensible Markup Language ) format, which contains specific update information. The target write data contains various information including table names, namely database table names needing to be updated, fields, specific fields and new values thereof, and conditions, namely which records should be updated, usually a WHERE clause in an SQL query, for accurately matching the records needing to be updated.
And step B20, constructing a database update statement based on the table names, the fields and the conditions.
It should be noted that, starting from a basic SQL UPDATE statement template, it includes an UPDATE key, a table name, a SET clause, and a WHERE clause. And dynamically constructing the SET clause according to the fields obtained by analysis and the new values thereof. And dynamically constructing a WHERE clause according to the conditions obtained by analysis. The SET clause and the WHERE clause are combined together to form a complete SQL update statement.
And step B30, updating database data in the structured database based on the database update statement, and incrementally updating the version number field of each database data corresponding to the database version number to determine the updated database version number of each database data.
It should be noted that, after the complete SQL update statement has been constructed in step B20, it is now necessary to execute this statement through JDBC (Java Database Connectivity, java database connection) or other database connection library. An increment operation including a version number field in the SET clause, such as fifth _version= fifth _version+1, ensures that the version number is incremented automatically each time a successful update occurs.
And step B40, updating the cache data in the cache according to the updated database data, and incrementally updating the version number field of each database data corresponding to the cache version number to determine the updated cache version number of each database data, wherein the optimistic lock version number, the cache version number and the initial value of the database version number are the same.
Specifically, a query is executed in a structured database, the data which is just updated is obtained, the data obtained from the database is mapped to a cache object in a cache, and the data in the cache is updated through a command of Redis or a client side library. An additional version number field (e.g., a four version) is maintained in the cache for each record for synchronous updating, the value of the four version field is incremented each time the cache is updated, and after the update operation is completed, the latest four version is queried and validated.
The method and the device can ensure the consistency and the integrity of the data in a high concurrency environment, improve the performance and the user experience of the system, solve the problems of potential data conflict and inconsistency, and greatly improve the performance and the reliability of the system.
The embodiment obtains the distributed optimistic lock from the cache when detecting that the current thread receives the data writing request, matches the request version number carried in the data writing request with the optimistic lock version number of the distributed optimistic lock record, updates the optimistic lock version number of the distributed optimistic lock record if the matching is successful, releases the distributed optimistic lock after the updating is completed, obtains target writing data from the data writing request, and updates the structured database and the cache according to the target writing data. According to the embodiment, the optimistic lock is arranged in the cache instead of the structured database, so that concurrency performance and reading and writing speed are improved, updating is allowed only when the version number carried by the request is consistent with the version number recorded currently through a version number matching mechanism, the problem of data coverage caused by that a plurality of threads modify the same data item at the same time is effectively prevented, and therefore data consistency is maintained.
For an example, to facilitate understanding of the implementation flow of the data processing method based on the distributed optimistic lock according to the present embodiment obtained in connection with the first embodiment, please refer to fig. 2, fig. 2 provides a schematic flow chart of a data processing method based on the distributed optimistic lock, specifically, the flow chart is started from a "start" node. Thread A gets the lock, matches the version, passes, and simultaneously, thread B also tries to acquire the lock, but because the lock is occupied by thread A, thread B enters a blocking state, thread A changes the version to v_2, thread A temporarily releases the lock, enters the blocking state, waits for subsequent operation, thread B tries to acquire the lock again, but the version is updated to v_2 at the moment and does not match the expected version of thread B, so that thread B returns directly to indicate failure, thread A releases the blocking, continues to execute, data in the structured database is modified, thread A updates cache data in the unstructured database (such as Redis), and the flow is ended.
In the second embodiment of the present application, the same or similar content as in the first embodiment of the present application may be referred to the above description, and will not be repeated. On this basis, referring to fig. 3, the data processing method based on the distributed optimistic lock further includes steps S50 to S80:
In step S50, when a data read request is received, a distributed optimistic lock is acquired from the cache.
It should be noted that, after the data read request is sent, if the optimistic lock version number is modified, it means that there are other threads that have updated the resource. At this point, reading data directly from the cache may result in reading out stale data. By acquiring the optimistic lock and performing version number matching, it is ensured that the read operation is performed based on the latest data state. Although the read operation itself does not require locking, in this case, acquiring an optimistic lock can ensure that only one thread can successfully read the latest data at the same time, and that subsequent write operations can also be performed based on the latest version.
Step S60, the request version number carried in the data reading request is matched with the updated optimistic lock version number of the distributed optimistic lock record.
It should be noted that the request version number is a version number carried by the client when the client initiates a read request, and is generally obtained from a previous read or write operation. The optimistic lock version number is the latest version number of the distributed optimistic lock records in the cache reflecting the latest state of the resource.
Step S70, if the matching is successful, the target data corresponding to each data reading request is obtained from the cached data.
It should be noted that, the cache is generally used as a cache layer, and stores copies of data in the structured database. Each cache data contains a version number field for recording an optimistically locked version number that is synchronized with the structured database. When the version number is successfully matched, the server side searches corresponding data from the cache according to the resource identifier in the request. If there is corresponding data in the cache, the data is directly returned to the client.
In addition, after the data is successfully read, the read version number is returned to the client side at the same time, so that the client side can be used as a request version number or an optimistic lock version number when the data is written or read next time.
In a possible embodiment, after step S70, step C10 may further be included:
And step C10, if the cached data does not contain target data, acquiring target data corresponding to each data reading request from the structured database, and updating the cached data in the cache according to each target data.
It should be noted that if there is no corresponding data in the cache, it may be because the data has not been cached or the cache has expired. At this time, the latest data needs to be read again from the structured database and updated into the cache, so as to ensure that the cache is consistent with the persistence layer.
Step S80, if the matching fails, returning matching failure information.
Other threads or clients update the same resource within the same time window, resulting in a change in version number, or the request version number read by the client may be out of date due to network delays or other factors.
The method comprises the steps of obtaining a distributed optimistic lock from a cache when a plurality of threads are detected to receive a data reading request, matching a request version number carried in the data reading request with an updated optimistic lock version number recorded by the distributed optimistic lock, obtaining target data corresponding to each data reading request from cached data if the matching is successful, returning matching failure information and the updated optimistic lock version number if the matching is failed, receiving the updated data reading request based on the updated optimistic lock version number, and returning to an execution step according to the updated data reading request. According to the embodiment, through a version number matching mechanism, each reading is ensured to be carried out based on the latest data state, the dirty reading problem is effectively prevented, the consistency and the integrity of data can be ensured under a high concurrency environment, and meanwhile, the performance and the user experience of a system are improved.
It should be noted that the above examples are only for understanding the present application, and do not constitute a limitation of the data processing method of the present application based on the distributed optimistic lock, and that it is within the scope of the present application to make more simple transformations based on this technical idea.
The present application also provides a data processing device based on a distributed optimistic lock, referring to fig. 4, the data processing device based on a distributed optimistic lock includes:
the lock fetching module 10 is configured to obtain a distributed optimistic lock from a cache when a data writing request is received;
the matching module 20 is configured to match a request version number carried in the data write request with an optimistic lock version number of the distributed optimistic lock record;
An updating module 30, configured to update the optimistic lock version number of the distributed optimistic lock record if the matching is successful, and release the distributed optimistic lock after the updating is completed;
And the writing module 40 is configured to obtain target writing data from the data writing request, and update database data in the structured database and cache data in the cache according to the target writing data. The data processing device based on the distributed optimistic lock provided by the application adopts the data processing method based on the distributed optimistic lock in the embodiment, and can solve the technical problem that cache and database data are inconsistent easily when the optimistic lock is applied. Compared with the prior art, the data processing device based on the distributed optimistic lock has the same beneficial effects as the data processing method based on the distributed optimistic lock provided by the above embodiment, and other technical features in the data processing device based on the distributed optimistic lock are the same as the features disclosed in the method of the above embodiment, which are not repeated herein.
The application provides a data processing device based on a distributed optimistic lock, which comprises at least one processor and a memory in communication with the at least one processor, wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the data processing method based on the distributed optimistic lock in the first embodiment.
With reference now to FIG. 5, a block diagram of a distributed optimistic lock-based data processing apparatus suitable for use in implementing an embodiment of the present application is shown. The distributed optimistic lock-based data processing apparatus in an embodiment of the application may include, but is not limited to, mobile terminals such as mobile phones, notebook computers, digital broadcast receivers, PDAs (Personal DIGITAL ASSISTANT: personal digital assistants), PADs (Portable Application Description: tablet computers), PMPs (Portable MEDIA PLAYER: multimedia players), vehicle terminals (e.g., vehicle navigation terminals), and the like, and fixed terminals such as digital TVs, desktop computers, and the like. The distributed optimistic lock-based data processing apparatus illustrated in fig. 5 is merely an example and should not be taken as limiting the functionality and scope of use of embodiments of the present application.
As shown in fig. 5, the distributed optimistic lock-based data processing apparatus may include a processing device 1001 (e.g., a central processor, a graphics processor, etc.) that may perform various appropriate actions and processes according to a program stored in a Read Only Memory 1002 or a program loaded from a storage device 1003 into a random access Memory (RAM: random Access Memory) 1004. Also stored in the RAM1004 are various programs and data required for operation of the distributed optimistic lock based data processing apparatus. The processing device 1001, the ROM1002, and the RAM1004 are connected to each other by a bus 1005. An input/output (I/O) interface 1006 is also connected to the bus. In general, a system including an input device 1007 such as a touch screen, a touch pad, a keyboard, a mouse, an image sensor, a microphone, an accelerometer, a gyroscope, etc., an output device 1008 including a Liquid crystal display (LCD: liquid CRYSTAL DISPLAY), a speaker, a vibrator, etc., a storage device 1003 including a magnetic tape, a hard disk, etc., and a communication device 1009 may be connected to the I/O interface 1006. The communication means 1009 may allow the distributed optimistic lock-based data processing device to communicate wirelessly or by wire with other devices to exchange data. While a distributed optimistic lock-based data processing apparatus having various systems is illustrated in the figures, it should be understood that not all illustrated systems are required to be implemented or provided. More or fewer systems may alternatively be implemented or provided.
In particular, according to embodiments of the present disclosure, the processes described above with reference to flowcharts may be implemented as computer software programs. For example, embodiments of the present disclosure include a computer program product comprising a computer program embodied on a computer readable medium, the computer program comprising program code for performing the method shown in the flow chart. In such an embodiment, the computer program may be downloaded and installed from a network through a communication device, or installed from the storage device 1003, or installed from the ROM 1002. The above-described functions defined in the method of the disclosed embodiment of the application are performed when the computer program is executed by the processing device 1001.
The data processing equipment based on the distributed optimistic lock provided by the application adopts the data processing method based on the distributed optimistic lock in the embodiment, and can solve the technical problem that cache and database data are inconsistent easily when the optimistic lock is applied. Compared with the prior art, the data processing device based on the distributed optimistic lock has the same beneficial effects as the data processing method based on the distributed optimistic lock provided by the above embodiment, and other technical features in the data processing device based on the distributed optimistic lock are the same as the features disclosed in the method of the previous embodiment, which are not described herein in detail.
It is to be understood that portions of the present disclosure may be implemented in hardware, software, firmware, or a combination thereof. In the description of the above embodiments, particular features, structures, materials, or characteristics may be combined in any suitable manner in any one or more embodiments or examples.
The foregoing is merely illustrative of the present application, and the present application is not limited thereto, and any person skilled in the art will readily recognize that variations or substitutions are within the scope of the present application. Therefore, the protection scope of the present application shall be subject to the protection scope of the claims.
The present application provides a computer readable storage medium having computer readable program instructions (i.e., a computer program) stored thereon for performing the distributed optimistic lock-based data processing method of the above embodiment.
The computer readable storage medium provided by the present application may be, for example, a USB flash disk, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system or device, or a combination of any of the foregoing. More specific examples of a computer-readable storage medium may include, but are not limited to, an electrical connection having one or more wires, a portable computer diskette, a hard disk, a random access Memory (RAM: random Access Memory), a Read-Only Memory (ROM), an erasable programmable Read-Only Memory (EPROM: erasable Programmable Read Only Memory or flash Memory), an optical fiber, a portable compact disc Read-Only Memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing. In this embodiment, the computer-readable storage medium may be any tangible medium that can contain, or store a program for use by or in connection with an instruction execution system or device. Program code embodied on a computer readable storage medium may be transmitted using any appropriate medium, including but not limited to electrical wiring, fiber optic cable, RF (Radio Frequency) and the like, or any suitable combination of the foregoing.
The computer readable storage medium may be contained in the distributed optimistic lock-based data processing apparatus or may exist alone without being incorporated into the distributed optimistic lock-based data processing apparatus.
The computer readable storage medium is loaded with one or more programs, and when the one or more programs are executed by the data processing device based on the distributed optimistic lock, the data processing device based on the distributed optimistic lock obtains the distributed optimistic lock from a cache when a data writing request is received, matches a request version number carried in the data writing request with an optimistic lock version number of a distributed optimistic lock record, updates the optimistic lock version number of the distributed optimistic lock record if the matching is successful, releases the distributed optimistic lock after the updating is completed, obtains target writing data from the data writing request, and updates database data in a structured database and cache data in the cache according to the target writing data.
Computer program code for carrying out operations of the present application may be written in any combination of one or more programming languages, including an object oriented programming language such as Java, smalltalk, C ++ and conventional procedural programming languages, such as the "C" programming language or similar programming languages. The program code may execute entirely on the user's computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer and partly on a remote computer or entirely on the remote computer or server. In the case of remote computers, the remote computer may be connected to the user's computer through any kind of network, including a local area network (LAN: local Area Network) or a wide area network (WAN: wide Area Network), or may be connected to an external computer (for example, through the Internet using an Internet service provider).
The flowcharts and block diagrams in the figures illustrate the architecture, functionality, and operation of possible implementations of systems, methods and computer program products according to various embodiments of the present application. In this regard, each block in the flowchart or block diagrams may represent a module, segment, or portion of code, which comprises one or more executable instructions for implementing the specified logical function(s). It should also be noted that, in some alternative implementations, the functions noted in the block may occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently, or the blocks may sometimes be executed in the reverse order, depending upon the functionality involved. It will also be noted that each block of the block diagrams and/or flowchart illustration, and combinations of blocks in the block diagrams and/or flowchart illustration, can be implemented by special purpose hardware-based systems which perform the specified functions or acts, or combinations of special purpose hardware and computer instructions.
The modules involved in the embodiments of the present application may be implemented in software or in hardware. Wherein the name of the module does not constitute a limitation of the unit itself in some cases.
The readable storage medium provided by the application is a computer readable storage medium, and the computer readable storage medium stores computer readable program instructions (namely a computer program) for executing the data processing method based on the distributed optimistic lock, so that the technical problem that cache and database data are inconsistent easily caused by application of the optimistic lock can be solved. Compared with the prior art, the beneficial effects of the computer readable storage medium provided by the application are the same as those of the data processing method based on the distributed optimistic lock provided by the above embodiment, and are not described herein in detail.
The application also provides a computer program product comprising a computer program which, when executed by a processor, implements the steps of a distributed optimistic lock-based data processing method as described above.
The computer program product provided by the application can solve the technical problem that the application optimistic lock is easy to cause cache and database data inconsistency. Compared with the prior art, the beneficial effects of the computer program product provided by the application are the same as those of the data processing method based on the distributed optimistic lock provided by the above embodiment, and are not described herein in detail.
The foregoing description is only a partial embodiment of the present application, and is not intended to limit the scope of the present application, and all the equivalent structural changes made by the description and the accompanying drawings under the technical concept of the present application, or the direct/indirect application in other related technical fields are included in the scope of the present application.
Claims (8)
1. A data processing method based on a distributed optimistic lock, applied to a current thread, comprising:
When a data writing request is received, a distributed optimistic lock is obtained from a cache, wherein the distributed optimistic lock is key value pair data, a main key of the key value pair data is a device identifier, and the value of the key value pair data records an optimistic lock version number;
Acquiring a preset synchronous lock, wherein when a thread acquires the preset synchronous lock, only the current thread is allowed to execute operations of acquiring a distributed optimistic lock and updating an optimistic lock version number of a record of the distributed optimistic lock;
matching a request version number carried in the data writing request with an optimistic lock version number of the distributed optimistic lock record;
If the matching is successful, the optimistic lock version number of the distributed optimistic lock record is increased, and after the updating is completed, the distributed optimistic lock is released;
Releasing the preset synchronous lock;
Acquiring target writing data from the data writing request, and updating database data in a structured database and cache data in the cache according to the target writing data;
The step of updating database data in the structured database and cache data in the cache according to the target write data comprises the following steps:
Analyzing the target writing data and determining the table name, the field and the condition to be written;
constructing a database update statement based on the table name, the field and the condition;
updating database data in the structured database based on the database update statement, incrementally updating the version number corresponding to the version number of the database corresponding to the version number field of each database data, and determining the updated version number of each database data;
And updating the cache data in the cache according to the updated database data, and incrementally updating the corresponding cache version number of the version number field of each database data to determine the updated cache version number of each database data, wherein the optimistic lock version number, the cache version number and the database version number have the same initial value.
2. The distributed optimistic lock-based data processing method of claim 1, wherein the step of updating an optimistic lock version number of the distributed optimistic lock record comprises:
Determining key value pair data corresponding to the distributed optimistic lock, wherein a primary key of the key value pair data is a device identifier, and the value of the key value pair data records an optimistic lock version number;
Extracting an optimistic lock version number from the key pair data;
increasing the optimistic lock version number based on a preset version update rule to obtain an update version number;
and storing the updated version number as an optimistic lock version number in key value pair data corresponding to the distributed optimistic lock.
3. The method for distributed optimistic lock-based data processing according to claim 1, the data processing method based on the distributed optimistic lock is characterized by further comprising the following steps:
When a data reading request is received, acquiring a distributed optimistic lock from the cache;
Matching a request version number carried in the data reading request with the updated optimistic lock version number of the distributed optimistic lock record;
if the matching is successful, acquiring target data corresponding to each data reading request from the cached data;
if the matching fails, returning matching failure information.
4. The method for processing data based on distributed optimistic lock according to claim 3, further comprising, after the step of obtaining target data corresponding to each of the data read requests from the cached data:
And if the target data does not exist in the cached data, acquiring target data corresponding to each data reading request from the structured database, and updating the cached data in the cache according to each target data.
5. A distributed optimistic lock-based data processing apparatus, the distributed optimistic lock-based data processing apparatus comprising:
The lock taking module is used for obtaining a distributed optimistic lock from a cache when a data writing request is received, wherein the distributed optimistic lock is key value pair data, a main key of the key value pair data is a device identifier, and the value of the key value pair data records an optimistic lock version number;
The matching module is used for matching the request version number carried in the data writing request with the optimistic lock version number of the distributed optimistic lock record;
the updating module is used for increasing the optimistic lock version number of the distributed optimistic lock record if the matching is successful, and releasing the distributed optimistic lock after the updating is completed;
The writing module is used for acquiring target writing data from the data writing request and updating database data in the structured database and cache data in the cache according to the target writing data;
The data processing device based on the distributed optimistic lock is further used for acquiring a preset synchronous lock, wherein when a thread acquires the preset synchronous lock, only the current thread is allowed to execute operations of acquiring the distributed optimistic lock and updating the optimistic lock version number of the distributed optimistic lock record;
The writing module is specifically configured to:
Analyzing the target writing data and determining the table name, the field and the condition to be written;
constructing a database update statement based on the table name, the field and the condition;
updating database data in the structured database based on the database update statement, incrementally updating the version number corresponding to the version number of the database corresponding to the version number field of each database data, and determining the updated version number of each database data;
And updating the cache data in the cache according to the updated database data, and incrementally updating the corresponding cache version number of the version number field of each database data to determine the updated cache version number of each database data, wherein the optimistic lock version number, the cache version number and the database version number have the same initial value.
6. A distributed optimistic lock-based data processing apparatus comprising a memory, a processor and a computer program stored on the memory and executable on the processor, the computer program being configured to implement the steps of the distributed optimistic lock-based data processing method according to any one of claims 1 to 4.
7. A storage medium, characterized in that the storage medium is a computer-readable storage medium, on which a computer program is stored, which computer program, when being executed by a processor, carries out the steps of the distributed optimistic lock-based data processing method according to any one of claims 1 to 4.
8. A computer program product, characterized in that the computer program product comprises a computer program which, when executed by a processor, implements the steps of the distributed optimistic lock-based data processing method according to any one of claims 1 to 4.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202510026372.XA CN119441244B (en) | 2025-01-08 | 2025-01-08 | Data processing method, device, equipment and medium based on distributed optimistic lock |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202510026372.XA CN119441244B (en) | 2025-01-08 | 2025-01-08 | Data processing method, device, equipment and medium based on distributed optimistic lock |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN119441244A CN119441244A (en) | 2025-02-14 |
| CN119441244B true CN119441244B (en) | 2025-07-29 |
Family
ID=94516551
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202510026372.XA Active CN119441244B (en) | 2025-01-08 | 2025-01-08 | Data processing method, device, equipment and medium based on distributed optimistic lock |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN119441244B (en) |
Citations (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113220669A (en) * | 2021-06-09 | 2021-08-06 | 京东科技控股股份有限公司 | Service data processing method and device and electronic equipment |
Family Cites Families (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US7774319B2 (en) * | 2004-08-11 | 2010-08-10 | Sap Ag | System and method for an optimistic database access |
| US20110276549A1 (en) * | 2010-05-04 | 2011-11-10 | Microsoft Corporation | Optimistic locking in a distributed file system replication environment |
| JP6199808B2 (en) * | 2014-06-06 | 2017-09-20 | 株式会社東芝 | Database apparatus and data access method |
-
2025
- 2025-01-08 CN CN202510026372.XA patent/CN119441244B/en active Active
Patent Citations (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113220669A (en) * | 2021-06-09 | 2021-08-06 | 京东科技控股股份有限公司 | Service data processing method and device and electronic equipment |
Also Published As
| Publication number | Publication date |
|---|---|
| CN119441244A (en) | 2025-02-14 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US9268810B2 (en) | Locking across multiple RID spaces | |
| CN113761294B (en) | Data management method, device, storage medium and electronic equipment | |
| US8655859B2 (en) | Concurrency control for extraction, transform, load processes | |
| US7107267B2 (en) | Method, system, program, and data structure for implementing a locking mechanism for a shared resource | |
| US9164911B2 (en) | Atomic execution over accesses to multiple memory locations in a multiprocessor system | |
| US9658900B2 (en) | Reentrant read-write lock algorithm | |
| WO2017181931A1 (en) | Method and device for processing distributed transaction | |
| US10191936B2 (en) | Two-tier storage protocol for committing changes in a storage system | |
| US9063887B2 (en) | Restoring distributed shared memory data consistency within a recovery process from a cluster node failure | |
| US20150074070A1 (en) | System and method for reconciling transactional and non-transactional operations in key-value stores | |
| US12032560B2 (en) | Distributed transaction execution management in distributed databases | |
| KR20060050223A (en) | Verification method and system for dynamically generated operations in data store | |
| CN112579307A (en) | Physical lock resource allocation detection method and device and electronic equipment | |
| CN109933606B (en) | Database modification method, device, equipment and storage medium | |
| US10565184B2 (en) | Method and system for committing transactions in a semi-distributed manner | |
| CN115422188A (en) | Table structure online changing method and device, electronic equipment and storage medium | |
| US10489374B2 (en) | In-place updates with concurrent reads in a decomposed state | |
| CN110546609A (en) | Hardware transactional memory (HTM) assists database transactions | |
| CN119441244B (en) | Data processing method, device, equipment and medium based on distributed optimistic lock | |
| US10776344B2 (en) | Index management in a multi-process environment | |
| US10685014B1 (en) | Method of sharing read-only data pages among transactions in a database management system | |
| US20080082533A1 (en) | Persistent locks/resources for concurrency control | |
| CN118916430B (en) | Data processing method, device, equipment, storage medium and program product | |
| CN115826880B (en) | Local caching method, device, system, medium and equipment of object storage system | |
| US9846695B2 (en) | Information processing apparatus, control method, and storage medium storing program |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |