Disclosure of Invention
The application aims to provide an anti-timeout method and device for backup log data, an electronic device and a computer readable storage medium, so that the backup timeout probability is effectively reduced, the backup efficiency is further improved, and the service processing burden and the network communication burden of a source device are reduced.
In order to solve the above technical problem, in a first aspect, the present application discloses an anti-timeout method for backing up log data, which is applied to a backup device, and includes:
receiving log data to be backed up sent by source equipment;
storing IO tasks corresponding to log data which cannot be processed currently to a delay processing queue;
and judging whether the overtime remaining time of each IO task in the delay processing queue is lower than the preset time length or not through preset process polling, and executing the IO tasks with the remaining time lower than the preset time length.
Optionally, the preset process is a preset daemon process.
Optionally, the preset time period is 2 seconds.
Optionally, the executing the IO task with the remaining time being less than the preset time includes:
and calling the wake-up function of the IO task with the residual time being lower than the preset time length.
Optionally, the method further comprises:
recording the timeout rate of the IO tasks in the delay processing queue;
judging whether the timeout rate is higher than a preset threshold value or not;
and if so, increasing the preset duration.
Optionally, after the determining whether the timeout rate is higher than a preset threshold, the method further includes:
if not, judging whether the CPU occupancy rate of the backup equipment is higher than a preset ratio or not;
and if the CPU occupancy rate is higher than the preset ratio, reducing the preset time.
In a second aspect, the present application further discloses an anti-timeout device for backing up log data, which is applied to a backup device and includes:
the receiving module is used for receiving the log data to be backed up sent by the source equipment;
the cache module is used for storing IO tasks corresponding to the log data which cannot be processed currently to the delay processing queue;
and the monitoring module is used for judging whether the overtime remaining time of each IO task in the delay processing queue is lower than the preset time length or not through preset process polling and executing the IO tasks with the remaining time lower than the preset time length.
Optionally, the system further includes a modification module, configured to record a timeout rate of the IO task in the delay processing queue; judging whether the timeout rate is higher than a preset threshold value or not; and if so, increasing the preset duration.
In a third aspect, the present application also discloses an electronic device, including:
a memory for storing a computer program;
a processor for executing said computer program to implement the steps of any of the above described anti-timeout methods of backing up log data.
In a fourth aspect, the present application further discloses a computer-readable storage medium, in which a computer program is stored, which, when being executed by a processor, is configured to implement the steps of any one of the above-mentioned anti-timeout methods for backing up log data.
The anti-timeout method for backup log data provided by the application is applied to backup equipment and comprises the following steps: receiving log data to be backed up sent by source equipment; storing IO tasks corresponding to log data which cannot be processed currently to a delay processing queue; and judging whether the overtime remaining time of each IO task in the delay processing queue is lower than the preset time length or not through preset process polling, and executing the IO tasks with the remaining time lower than the preset time length.
Therefore, the method and the device can automatically monitor the overtime condition of each IO task in the delay processing queue by utilizing the preset process, and further process the IO task which is about to overtime in time before overtime, thereby effectively reducing the backup overtime probability, avoiding repeated data transmission between the source device and the backup device, further improving the backup efficiency, and effectively relieving the business processing burden and the network communication burden of the source device. The anti-timeout device for backup log data, the electronic device and the computer-readable storage medium have the same beneficial effects.
Detailed Description
The core of the application is to provide an anti-timeout method and device for backup log data, an electronic device and a computer readable storage medium, so as to effectively reduce backup timeout probability, further improve backup efficiency and reduce service processing burden and network communication burden of a source device.
In order to more clearly and completely describe the technical solutions in the embodiments of the present application, the technical solutions in the embodiments of the present application will be described below with reference to the drawings in the embodiments of the present application. It is to be understood that the embodiments described are only a few embodiments of the present application and not all embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present application.
Currently, the importance of data backup storage is also more and more emphasized by people. Among other things, remote backup storage is an effective means of providing data redundancy security. Through remote backup, the source device can send local important data such as log data to the remote backup device for backup storage. When the backup device is busy and cannot be processed in a timely manner, the backup task is temporarily queued to delay processing. However, in the prior art, it often happens that the data backup task put into the queue has exceeded the effective time before it has not been processed. Thus, the backup device needs to resend the request to the source device in order for the source device to resend the log data. This is not only inefficient, but also increases the traffic processing burden and network communication burden on the source device. In view of this, the present application provides an anti-timeout scheme for backing up log data, which can effectively solve the above problem.
Referring to fig. 1, an embodiment of the present application discloses a timeout prevention method for backup log data, which is applied to a backup device and mainly includes:
s101: and receiving log data to be backed up sent by the source equipment.
First, in remote backup, a source device refers to a device that generates source data, and a backup device refers to a device that stores backup data. After the source data generated by the source device is backed up, the backup data is transmitted to and stored in the backup device through the telecommunication service.
Log data backup (Journal Replication) is an important data backup operation. The method comprises the steps that log logging (JREC) operation and log sending (JSND) operation can be executed at a source equipment end; and log receiving (JRCV) operation and log dropping (JETR) operation can be performed at the backup device side.
When the source device sends log data to request log data backup, the backup device may perform a JRCV operation, that is, receive the log data to be backed up sent by the source device.
S102: and storing the IO tasks corresponding to the log data which cannot be processed currently to the delay processing queue.
When storing received log data, i.e., performing JETR operation, a certain IO throughput capacity is required. If the backup device has more current IO operations and cannot process the disk-dropping operation of the log data in time, the log backup task which cannot be processed in time can be delayed.
Specifically, the maximum number of bytes that can be handled by the storage device for one IO operation is fixed, and a single IO operation may be referred to as an IO task. For received log data, the disk-dropping operation can be divided into a plurality of IO tasks, and the delayed processing queue can be used for storing the IO tasks which cannot be processed in time, so that the backup equipment can process the IO tasks again in the absence.
S103: and judging whether the overtime remaining time of each IO task in the delay processing queue is less than the preset time length or not through preset process polling, and executing the IO tasks with the remaining time less than the preset time length.
Specifically, the embodiment of the application is provided with a preset process for preventing timeout. During the execution process, the process continuously and periodically polls each IO task in the delay processing queue to obtain the remaining time of the IO task when the distance is overtime. If the IO task with the distance overtime remaining time being lower than the preset time length is found, the preset process can immediately execute the IO task to prevent overtime.
Moreover, as a specific embodiment, the preset process may be specifically a preset daemon process, and automatically runs in the background, so as to timely perform timeout prevention.
It should be further explained that, as a specific embodiment, the preset process may specifically execute the IO task which is about to time out by invoking a wake-up function of the corresponding IO task.
On the basis of the above, as a specific embodiment, the preset time period may be specifically set to 2 seconds. Since the execution time of an IO task is very short, typically on the order of milliseconds. Thus, the 2 seconds left until the timeout is sufficient for an IO task to complete. Of course, the preset time period can be set to other suitable values by those skilled in the art according to the actual application requirements.
In conjunction with the above, the process of polling judgment of the default process can be seen in fig. 2. Fig. 2 is a flowchart of a method for polling a delay processing queue according to an embodiment of the present application, which mainly includes:
s201: the IO task currently polled is determined.
S202: and acquiring the residual time of the IO task distance overtime.
S203: judging whether the remaining time is less than 2 seconds; if yes, entering S204; if not, the process proceeds to S205.
S204: calling a wake-up function of the IO task; the process proceeds to S205.
S205: the pointer points to the next IO task; the process proceeds to S202.
The anti-timeout method for backup log data provided by the embodiment of the application comprises the following steps: receiving log data to be backed up sent by source equipment; storing IO tasks corresponding to log data which cannot be processed currently to a delay processing queue; and judging whether the overtime remaining time of each IO task in the delay processing queue is less than the preset time length or not through preset process polling, and executing the IO tasks with the remaining time less than the preset time length.
Therefore, the method and the device can automatically monitor the overtime condition of each IO task in the delay processing queue by utilizing the preset process, and further process the IO task which is about to overtime in time before overtime, thereby effectively reducing the backup overtime probability, avoiding repeated data transmission between the source device and the backup device, further improving the backup efficiency, and effectively relieving the business processing burden and the network communication burden of the source device.
As a specific embodiment, on the basis of the foregoing, the anti-timeout method for backup log data provided in the embodiment of the present application further includes:
recording the timeout rate of IO tasks in the delay processing queue;
judging whether the timeout rate is higher than a preset threshold value or not;
if yes, increasing the preset time length;
if not, judging whether the CPU occupancy rate of the backup equipment is higher than a preset ratio;
and if the CPU occupancy rate is higher than the preset ratio, reducing the preset time.
Specifically, the anti-timeout method for backup log data disclosed in the embodiment of the present application further provides an automatic modification mechanism for a parameter of a preset duration. Because the preset time is the time reserved for processing the IO task in advance, if the preset time is too short, the IO task may not be processed and completed before timeout; if the preset time is too long, the resource occupancy rate of the backup device, such as the CPU occupancy rate, is increased.
Therefore, the preset duration is reasonably adjusted based on the timeout rate of the IO task in the delay queue and the CPU occupancy rate of the backup device respectively: when the timeout rate of the IO tasks in the delay processing queue is higher than a preset threshold value, the preset time is increased; and when the timeout rate of the IO tasks in the delay processing queue is not higher than the preset threshold value but the CPU occupancy rate of the backup device is higher than the preset ratio, the preset time can be adjusted to be shorter.
Referring to fig. 3, an embodiment of the present application discloses an anti-timeout device for backing up log data, which mainly includes:
a receiving module 301, configured to receive log data to be backed up sent by a source device;
the cache module 302 is configured to store an IO task corresponding to log data that cannot be currently processed to a delay processing queue;
the monitoring module 303 is configured to judge whether the remaining time of the distance timeout of each IO task in the delay processing queue is less than the preset time length through preset process polling, and execute the IO task whose remaining time is less than the preset time length.
Therefore, the overtime prevention device for backup log data disclosed by the embodiment of the application can automatically monitor the overtime condition of each IO task in the delay processing queue by using the preset process, and then process the IO task which is about to overtime in time before overtime, so that the backup overtime probability is effectively reduced, repeated data transmission between the source equipment and the backup equipment is avoided, the backup efficiency is further improved, and the service processing burden and the network communication burden of the source equipment are effectively relieved.
For specific contents of the above-mentioned timeout prevention device for backup log data, reference may be made to the foregoing detailed description of the timeout prevention method for backup log data, and details thereof are not repeated here.
Further, on the basis of the above content, the apparatus for preventing timeout of backup log data disclosed in the embodiment of the present application further includes, in a specific implementation manner, a correction module, configured to record a timeout rate of an IO task in a delay processing queue; judging whether the timeout rate is higher than a preset threshold value or not; if yes, increasing the preset time length.
Further, on the basis of the above content, in a specific implementation manner of the timeout prevention device for backup log data disclosed in the embodiment of the present application, the modification module is further configured to: after judging whether the timeout rate is higher than a preset threshold value, if not, judging whether the CPU occupancy rate of the backup equipment is higher than a preset ratio; and if the CPU occupancy rate is higher than the preset ratio, reducing the preset time.
Further, on the basis of the above content, in a specific implementation manner of the timeout prevention device for backup log data disclosed in the embodiment of the present application, the preset process is a preset daemon process.
Further, on the basis of the above content, in a specific implementation manner, the timeout prevention device for backup log data disclosed in the embodiment of the present application has a preset duration of 2 seconds.
Further, on the basis of the above content, in a specific implementation manner of the timeout prevention device for backup log data disclosed in the embodiment of the present application, the monitoring module 303 is specifically configured to: and calling the wake-up function of the IO task with the remaining time lower than the preset time.
Referring to fig. 4, an embodiment of the present application discloses an electronic device, including:
a memory 401 for storing a computer program;
a processor 402 for executing said computer program for implementing the steps of any of the above described anti-timeout methods for backing up log data.
Further, the present application also discloses a computer-readable storage medium, in which a computer program is stored, and the computer program is used for implementing the steps of any one of the above-mentioned anti-timeout methods for backing up log data when being executed by a processor.
For details of the electronic device and the computer-readable storage medium, reference may be made to the foregoing detailed description of the timeout prevention method for backing up log data, and details thereof are not repeated here.
The embodiments are described in a progressive manner, each embodiment focuses on differences from other embodiments, and the same and similar parts among the embodiments are referred to each other. For the equipment disclosed by the embodiment, the description is relatively simple because the equipment corresponds to the method disclosed by the embodiment, and the relevant parts can be referred to the method part for description.
It is further noted that, throughout this document, relational terms such as "first" and "second" are used solely to distinguish one entity or action from another entity or action without necessarily requiring or implying any actual such relationship or order between such entities or actions. Furthermore, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising an … …" does not exclude the presence of other identical elements in a process, method, article, or apparatus that comprises the element.
The technical solutions provided by the present application are described in detail above. The principles and embodiments of the present application are explained herein using specific examples, which are provided only to help understand the method and the core idea of the present application. It should be noted that, for those skilled in the art, without departing from the principle of the present application, several improvements and modifications can be made to the present application, and these improvements and modifications also fall into the protection scope of the present application.