CN114840526B - Data sampling processing method and device - Google Patents
Data sampling processing method and deviceInfo
- Publication number
- CN114840526B CN114840526B CN202210529441.5A CN202210529441A CN114840526B CN 114840526 B CN114840526 B CN 114840526B CN 202210529441 A CN202210529441 A CN 202210529441A CN 114840526 B CN114840526 B CN 114840526B
- Authority
- CN
- China
- Prior art keywords
- sampling
- data
- backup
- type
- backup strategy
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/22—Indexing; Data structures therefor; Storage structures
- G06F16/2282—Tablespace storage structures; Management thereof
-
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F11/00—Error detection; Error correction; Monitoring
- G06F11/07—Responding to the occurrence of a fault, e.g. fault tolerance
- G06F11/14—Error detection or correction of the data by redundancy in operation
- G06F11/1402—Saving, restoring, recovering or retrying
- G06F11/1446—Point-in-time backing up or restoration of persistent data
- G06F11/1458—Management of the backup or restore process
- G06F11/1469—Backup restoration techniques
-
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/23—Updating
-
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/24—Querying
- G06F16/245—Query processing
- G06F16/2455—Query execution
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Data Mining & Analysis (AREA)
- Databases & Information Systems (AREA)
- Quality & Reliability (AREA)
- Computational Linguistics (AREA)
- Software Systems (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
本发明提供一种数据抽检处理方法及装置，涉及数据处理技术领域，可用于金融领域或其他技术领域。所述方法包括：获取备份策略的抽检表信息；所述抽检表信息包括抽检状态和抽检类型；若确定所述抽检状态为待抽检状态，则获取处于所述待抽检状态的数据，对所述数据执行与所述抽检类型对应的命令；获取所述命令的执行结果，并根据所述执行结果更新所述抽检表信息。所述装置执行上述方法。本发明实施例提供的数据抽检处理方法及装置，能够自动完成数据抽检工作，避免由人工操作导致的误操作，还能够提高抽检任务处理效率。
The present invention provides a data sampling processing method and device, which relates to the field of data processing technology and can be used in the financial field or other technical fields. The method includes: obtaining sampling table information of a backup strategy; the sampling table information includes a sampling status and a sampling type; if it is determined that the sampling status is a pending sampling state, then obtaining data in the pending sampling state, executing a command corresponding to the sampling type on the data; obtaining the execution result of the command, and updating the sampling table information according to the execution result. The device executes the above method. The data sampling processing method and device provided in the embodiment of the present invention can automatically complete data sampling work, avoid erroneous operations caused by manual operations, and improve the efficiency of sampling task processing.
Description
Technical Field
The invention relates to the technical field of data processing, in particular to a data sampling detection processing method and device.
Background
With the development of internet technology, massive data are generated, part of the data need to be backed up, and in order to ensure the accuracy, the integrity and other requirements of the data, the backup data need to be subjected to spot check, so that the spot check work of the data becomes heavier.
The existing data sampling inspection work is mainly completed manually, misoperation is easy to occur, and efficiency is low.
Disclosure of Invention
Aiming at the problems in the prior art, the embodiment of the invention provides a data sampling detection processing method and device, which can at least partially solve the problems in the prior art.
In one aspect, the present invention provides a data sampling method, including:
The method comprises the steps of obtaining the information of a sampling inspection table of a backup strategy, wherein the information of the sampling inspection table comprises a sampling inspection state and a sampling inspection type;
If the sampling detection state is determined to be the sampling detection state, acquiring data in the sampling detection state, and executing a command corresponding to the sampling detection type on the data;
and acquiring an execution result of the command, and updating the spot check list information according to the execution result.
The method comprises the steps of executing a command corresponding to the sampling type on data, wherein the sampling type comprises medium availability verification, and correspondingly, the step of executing the command corresponding to the sampling type on the data comprises the following steps:
and executing a verification sampling inspection command on the data.
The method comprises the steps of checking the data, wherein the checking type comprises data availability verification, and correspondingly, executing a command corresponding to the checking type on the data comprises the following steps:
and acquiring the backup strategy type of the data, and executing a recovery command on the data corresponding to the backup strategy type respectively.
The backup strategy type comprises a file backup strategy type, and correspondingly, the executing of the recovery command on the data corresponding to the backup strategy type comprises the following steps:
and restoring the latest backup image corresponding to the file backup strategy type into a verification environment, wherein the verification environment completes the check in advance.
The backup strategy type comprises a database backup strategy type, and correspondingly, the executing of the recovery command on the data corresponding to the backup strategy type comprises the following steps:
and calling a database recovery script in a verification environment, and performing data interaction with backup software by using the database recovery script to realize database recovery, wherein the verification environment completes initialization setting in advance according to the database instance and the database parameters.
The step of obtaining the execution result of the command and updating the spot check list information according to the execution result comprises the following steps:
obtaining a restored file from the verification environment, and comparing the file with a file list in the latest backup image;
and if the comparison results of all the comparison items are the same, determining that the execution result is the completion of the execution, and updating the sampling inspection table information.
The step of obtaining the execution result of the command and updating the spot check list information according to the execution result comprises the following steps:
Logging in the database in the verification environment, if the database is confirmed to be recovered, determining that the execution result is the execution completion, and updating the spot check list information.
Before the step of obtaining the sampling table information of the backup strategy, the data sampling processing method further comprises the following steps:
and acquiring a backup strategy, and executing a corresponding backup strategy updating step according to a comparison result of the backup strategy and the pre-stored backup strategy in the spot check list information.
The data sampling inspection processing method comprises the steps of executing a corresponding backup strategy updating step, wherein the sampling inspection table information comprises application levels of the backup strategy, the application levels reflect the importance degree of the application, and the data sampling inspection processing method further comprises the following steps:
determining the data sampling frequency and the sampling type according to the application grade and a preset corresponding relation, wherein the preset corresponding relation comprises a corresponding relation among the preset application grade, the preset data sampling frequency and the preset sampling type;
Acquiring the latest spot check completion time of the backup strategy, and determining the time interval between the current time and the latest spot check completion time;
And if the time interval is greater than or equal to the data sampling frequency, updating the sampling detection state into the to-be-sampled detection state.
In one aspect, the present invention provides a data sampling processing device, including:
the acquisition unit is used for acquiring the information of the sampling inspection table of the backup strategy; the sampling test table information comprises sampling test states and sampling test types;
the execution unit is used for acquiring data in the sampling inspection state if the sampling inspection state is determined to be the sampling inspection state, and executing a command corresponding to the sampling inspection type on the data;
And the updating unit is used for acquiring an execution result of the command and updating the spot check list information according to the execution result.
In yet another aspect, an embodiment of the present invention provides an electronic device comprising a processor, a memory, and a bus, wherein,
The processor and the memory complete communication with each other through the bus;
The memory stores program instructions executable by the processor, the processor invoking the program instructions capable of performing the method of:
The method comprises the steps of obtaining the information of a sampling inspection table of a backup strategy, wherein the information of the sampling inspection table comprises a sampling inspection state and a sampling inspection type;
If the sampling detection state is determined to be the sampling detection state, acquiring data in the sampling detection state, and executing a command corresponding to the sampling detection type on the data;
and acquiring an execution result of the command, and updating the spot check list information according to the execution result.
Embodiments of the present invention provide a non-transitory computer readable storage medium comprising:
The non-transitory computer readable storage medium stores computer instructions that cause the computer to perform the method of:
The method comprises the steps of obtaining the information of a sampling inspection table of a backup strategy, wherein the information of the sampling inspection table comprises a sampling inspection state and a sampling inspection type;
If the sampling detection state is determined to be the sampling detection state, acquiring data in the sampling detection state, and executing a command corresponding to the sampling detection type on the data;
and acquiring an execution result of the command, and updating the spot check list information according to the execution result.
The data sampling inspection processing method and device provided by the embodiment of the invention acquire sampling inspection table information of a backup strategy, wherein the sampling inspection table information comprises a sampling inspection state and a sampling inspection type, if the sampling inspection state is determined to be the sampling inspection state, acquiring data in the sampling inspection state, executing a command corresponding to the sampling inspection type on the data, acquiring an execution result of the command, and updating the sampling inspection table information according to the execution result, so that the sampling inspection of the data can be automatically completed, misoperation caused by manual operation is avoided, and the processing efficiency of a sampling inspection task is improved.
Drawings
In order to more clearly illustrate the embodiments of the invention or the technical solutions in the prior art, the drawings that are required in the embodiments or the description of the prior art will be briefly described, it being obvious that the drawings in the following description are only some embodiments of the invention, and that other drawings may be obtained according to these drawings without inventive effort for a person skilled in the art. In the drawings:
Fig. 1 is a flowchart of a data sampling processing method according to an embodiment of the present invention.
Fig. 2 is a flowchart of a data sampling processing method according to another embodiment of the present invention.
Fig. 3 is a flowchart of a data sampling processing method according to another embodiment of the present invention.
Fig. 4 is a schematic structural diagram of a data sampling device according to an embodiment of the present invention.
Fig. 5 is a schematic diagram of an entity structure of an electronic device according to an embodiment of the present invention.
Detailed Description
For the purpose of making the objects, technical solutions and advantages of the embodiments of the present application more apparent, the embodiments of the present application will be described in further detail with reference to the accompanying drawings. The exemplary embodiments of the present application and their descriptions herein are for the purpose of explaining the present application, but are not to be construed as limiting the application. It should be noted that, without conflict, the embodiments of the present application and features of the embodiments may be arbitrarily combined with each other.
Fig. 1 is a flow chart of a data sampling inspection processing method according to an embodiment of the present invention, and as shown in fig. 1, the data sampling inspection processing method according to the embodiment of the present invention includes:
step S1, acquiring the information of a sampling inspection table of a backup strategy, wherein the information of the sampling inspection table comprises a sampling inspection state and a sampling inspection type.
And step S2, if the sampling detection state is determined to be the sampling detection state, acquiring data in the sampling detection state, and executing a command corresponding to the sampling detection type on the data.
And step S3, acquiring an execution result of the command, and updating the spot check list information according to the execution result.
In the step S1, the device acquires the information of the sampling inspection table of the backup strategy, wherein the information of the sampling inspection table comprises the sampling inspection state and the sampling inspection type. The apparatus may be a computer device performing the method and may include, for example, a server or a client, etc. It should be noted that, the data acquisition and analysis according to the embodiments of the present invention are authorized by the user.
Each record in the spot check list information may include the following information:
number for marking each record.
The backup strategy is an application needing backup, and specifically comprises information such as application names, application servers needing backup, application contents needing backup, backup starting time and the like.
And (3) applying the backup strategy, namely applying the backup strategy.
The application grade is that each application has default grade division, and the applications are sequenced from important to unimportant into the most important class A application, the second class B application, the second class C application and the least important class D application.
The sampling inspection state refers to the sampling inspection condition of the backup strategy, and the sampling inspection state is respectively that sampling inspection is not needed from sampling inspection, sampling inspection is already performed, sampling inspection is to be performed and sampling inspection is not needed.
The latest spot check completion time refers to the point in time when the last spot check of the backup strategy is completed from the current moment.
The sampling test table information can also comprise sampling test types, wherein the sampling test types comprise data availability verification and medium availability verification, and the sampling test types comprise:
And verifying the availability of the data, namely, in order to enable the backup data to be used normally, the backup data needs to be restored for verification.
The medium availability verification means that whether the medium where the backup image of the backup data is located is available only needs to be verified, the media command is only required to be verified through the API call of the backup software, and the backup software is used for reading the backup image and calculating whether the backup image is complete or not, so that the backup data does not need to be restored.
As shown in fig. 2, the following is explained:
3.1 starting the spot check task
And starting the sampling inspection task according to the sampling inspection table information. The polling mode can be adopted, and the sampling inspection starting task is executed regularly, so that the sampling inspection starting task is automated, and manual intervention is not needed.
3.2 Acquisition of the sampling test Meter
And 3.3, acquiring a sampling inspection table from the database, returning to 3.2, and judging according to the sampling inspection state by 3.2. If the sampling inspection state of the backup strategy is the sampling inspection state to be inspected, the sampling inspection task of the current time is required to implement the sampling inspection of the backup strategy, the sampling inspection state is updated to be in the sampling inspection, and the sampling inspection task is started to be implemented specifically.
In the step S2, if the device determines that the sampling test state is a sampling test state, the device acquires data in the sampling test state, and executes a command corresponding to the sampling test type on the data.
The method comprises the steps of executing a data, wherein the sampling type comprises medium availability verification, and correspondingly, executing a command corresponding to the sampling type on the data, wherein the command comprises the following steps:
and executing a verification sampling inspection command on the data.
The sampling type comprises data availability verification, and correspondingly, the executing of the command corresponding to the sampling type on the data comprises the following steps:
and acquiring the backup strategy type of the data, and executing a recovery command on the data corresponding to the backup strategy type respectively.
The backup strategy type comprises a file backup strategy type, and correspondingly, the executing of the recovery command on the data corresponding to the backup strategy type comprises the following steps:
and restoring the latest backup image corresponding to the file backup strategy type into a verification environment, wherein the verification environment completes the check in advance.
The backup strategy type comprises a database backup strategy type, and correspondingly, the executing of the recovery command on the data corresponding to the backup strategy type comprises the following steps:
and calling a database recovery script in a verification environment, and performing data interaction with backup software by using the database recovery script to realize database recovery, wherein the verification environment completes initialization setting in advance according to the database instance and the database parameters.
The following is described with reference to fig. 2:
3.4 determining the verification Environment according to the backup strategy type and the sampling inspection type
And if the sampling test type is media availability verification, calling a verification sampling test command of the API. The jump to 3.7 can be made directly without executing 3.5.
If the sampling type is data availability verification, a recovery command of calling an API is required, wherein the recovery command is divided into a file backup strategy type and a database backup strategy type, the backup data is required to be recovered to a verification environment, and the verification environment is required to be checked in advance.
3.5 Checking verification Environment
And confirming the size of the backup image according to the backup strategy and the backup image. It is confirmed through 3.6 whether the size of the verification environment can be restored. If the database backup strategy type is adopted, the database backup strategy type needs to be initialized according to the instance of the database and the parameters of the database. The checking and verifying environment can be selected from the existing mature technology, and will not be described in detail.
3.7 Executing the spot check Command
For the type of the spot check to be media availability verification, only the verification spot check command of the API needs to be started through 3.8, and the command execution result is waited.
And executing a 3.8API for the file backup strategy type with the data availability verification as the sampling inspection type, and controlling the backup software to restore the latest backup image corresponding to the file backup strategy type into the verification environment.
For the database backup policy type where the spot check type is data availability verification, a database recovery script needs to be invoked in a verification environment where initialization setup has been completed. The database recovery script can automatically communicate with the API of the backup software to perform data interaction, so as to complete the database recovery work. The database recovery script may be pre-compiled and stored in the database.
In the step S3, the device obtains the execution result of the command, and updates the spot check list information according to the execution result.
The step of obtaining the execution result of the command and updating the spot check list information according to the execution result comprises the following steps:
obtaining a restored file from the verification environment, and comparing the file with a file list in the latest backup image;
and if the comparison results of all the comparison items are the same, determining that the execution result is the completion of the execution, and updating the sampling inspection table information.
The step of obtaining the execution result of the command and updating the spot check list information according to the execution result comprises the following steps:
Logging in the database in the verification environment, if the database is confirmed to be recovered, determining that the execution result is the execution completion, and updating the spot check list information.
As shown in fig. 2, the following is explained:
3.9 execution of the inspection results
After the command is invoked, a command execution result is obtained.
And for medium availability verification, if the command execution result is that the execution is completed (namely successful), the sampling inspection work of the backup strategy is completed, the sampling inspection state is updated to be sampling inspection through 3.10, the sampling inspection time is updated to be the current time, and the sampling inspection time is written into a database.
And for the file backup strategy type with the sampling inspection type being data availability verification, if the command execution result is that the execution is completed (namely successful), acquiring the restored file from the verification environment, and comparing the file with a file list in the latest backup image.
If the comparison is the same with the data file name, file size, file time stamp and other information, the sampling inspection work of the backup strategy is completed, the sampling inspection state is updated to be sampling inspection through 3.10, the sampling inspection time is updated to be the current time, and the sampling inspection time is written into the database.
And (3) for the database backup strategy type with the sampling test type of data availability verification, if the command execution result is that the execution is completed (namely successful), logging in the database in the verification environment, and if the database recovery is confirmed to be completed, updating the sampling test state to be sampling test, updating the sampling test time to be the current time and writing the sampling test time into the database through 3.10.
It should be noted that if the execution result of the command is that the execution is not completed (i.e. fails), an alarm message may be generated, and a failure cause may be further generated, so that related personnel can check the processing as soon as possible.
3.11 Notification and reporting
And recording and informing the relevant system manager mainly of the verification result. If the spot check abnormality occurs, the relevant system manager can be informed in time, and the intervention is performed manually.
The data sampling inspection processing method provided by the embodiment of the invention acquires sampling inspection table information of a backup strategy, wherein the sampling inspection table information comprises a sampling inspection state and a sampling inspection type, acquires data in the sampling inspection state if the sampling inspection state is determined to be the sampling inspection state, executes a command corresponding to the sampling inspection type on the data, acquires an execution result of the command, and updates the sampling inspection table information according to the execution result, so that the sampling inspection of the data can be automatically completed, misoperation caused by manual operation is avoided, and the processing efficiency of a sampling inspection task is improved.
Further, the sampling test type comprises media availability verification, and correspondingly, the executing of the command corresponding to the sampling test type on the data comprises the following steps:
and executing a verification sampling inspection command on the data. Reference may be made to the above embodiments, and no further description is given.
The data sampling inspection processing method provided by the embodiment of the invention is suitable for the data sampling inspection processing of which the sampling inspection type is medium availability verification.
Further, the sampling type comprises data availability verification, and correspondingly, the executing of the command corresponding to the sampling type on the data comprises the following steps:
and acquiring the backup strategy type of the data, and executing a recovery command on the data corresponding to the backup strategy type respectively. Reference may be made to the above embodiments, and no further description is given.
The data sampling inspection processing method provided by the embodiment of the invention is suitable for the data sampling inspection processing of which the sampling inspection type is data availability verification.
Further, the backup policy type includes a file backup policy type, and correspondingly, the executing the restore command on the data corresponding to the backup policy type includes:
And restoring the latest backup image corresponding to the file backup strategy type into a verification environment, wherein the verification environment completes the check in advance. Reference may be made to the above embodiments, and no further description is given.
The data sampling inspection processing method provided by the embodiment of the invention is suitable for the data sampling inspection processing of which the backup strategy type is the file backup strategy type.
Further, the backup policy type includes a database backup policy type, and correspondingly, the executing the restore command on the data corresponding to the backup policy type includes:
And calling a database recovery script in a verification environment, and performing data interaction with backup software by using the database recovery script to realize database recovery, wherein the verification environment completes initialization setting in advance according to the database instance and the database parameters. Reference may be made to the above embodiments, and no further description is given.
The data sampling inspection processing method provided by the embodiment of the invention is suitable for the data sampling inspection processing of which the backup strategy type is the database backup strategy type.
Further, the obtaining the execution result of the command and updating the spot check list information according to the execution result includes:
And obtaining the restored file from the verification environment, and comparing the file with the file list in the latest backup image, wherein the description of the embodiment can be referred to and is not repeated.
And if the comparison results of all the comparison items are the same, determining that the execution result is the completion of the execution, and updating the sampling inspection table information. Reference may be made to the above embodiments, and no further description is given.
The data sampling inspection processing method provided by the embodiment of the invention can be suitable for the data sampling inspection processing of the file backup strategy type.
Further, the obtaining the execution result of the command and updating the spot check list information according to the execution result includes:
Logging in the database in the verification environment, if the database is confirmed to be recovered, determining that the execution result is the execution completion, and updating the spot check list information. Reference may be made to the above embodiments, and no further description is given.
The data sampling inspection processing method provided by the embodiment of the invention can be suitable for the data sampling inspection processing of the database backup strategy type.
Further, before the step of obtaining the sampling table information of the backup policy, the data sampling processing method further includes:
and acquiring a backup strategy, and executing a corresponding backup strategy updating step according to a comparison result of the backup strategy and the pre-stored backup strategy in the spot check list information.
As shown in fig. 3, the following is explained:
And for the sampling inspection table for acquiring the backup strategy, acquiring the backup strategy from the database, and executing a corresponding backup strategy updating step according to a comparison result of the backup strategy and the pre-stored backup strategy in the sampling inspection table information, wherein the method specifically comprises the following steps:
No update is needed for the backup policy that already exists in the spot check table.
And for the backup strategy which is not in the sampling inspection table, newly adding the backup strategy into the sampling inspection table.
For backup policies that do not exist in the backup policy table, but exist in the spot check table, it is indicated that spot checks are not needed, and the spot check state needs to be updated to be spot check-free.
The data sampling inspection processing method provided by the embodiment of the invention can ensure the synchronous consistency of the acquired backup strategy and the pre-stored backup strategy in the sampling inspection table information.
Further, the data sampling test processing method further comprises the following steps after the step of executing the corresponding backup strategy updating step:
determining the data sampling frequency and the sampling type according to the application grade and a preset corresponding relation, wherein the preset corresponding relation comprises a corresponding relation among the preset application grade, the preset data sampling frequency and the preset sampling type;
Acquiring the latest spot check completion time of the backup strategy, and determining the time interval between the current time and the latest spot check completion time;
And if the time interval is greater than or equal to the data sampling frequency, updating the sampling detection state into the to-be-sampled detection state.
As shown in fig. 3, the following is explained:
for confirming spot check requirements according to type:
The AB type application with higher importance comprises higher data sampling frequency, which can be selected to be 1 time per month, and sampling type which is data availability verification.
CD applications with lower importance degree are that the data sampling inspection frequency is lower, the data sampling inspection can be selected to be once in 3 months, and the sampling inspection type is medium availability verification. The corresponding relation among the data sampling frequency, the sampling type and the application level can be stored in a database and can be modified according to actual conditions.
And judging whether the time interval exceeds the data sampling frequency corresponding to each backup strategy by comparing the time interval between the latest sampling completion time and the current time of each backup strategy, wherein if the time interval is smaller than the data sampling frequency, the fact that new sampling work is not needed is indicated.
If the time interval is greater than or equal to the data sampling frequency, the backup strategy is indicated to need to carry out new sampling operation, and the sampling state is updated from the sampling state of the non-sampling inspection and the sampling state of the sampling inspection to the sampling state to be detected.
The data sampling inspection processing method provided by the embodiment of the invention can ensure the timeliness of sampling inspection state updating.
For forming the final spot checklist:
And updating the information of the sampling inspection table again according to the change, updating the sampling inspection state, the sampling inspection type and the like in the sampling inspection table, and writing the sampling inspection state, the sampling inspection type and the like into a database.
Before the step of obtaining the backup policy, the data sampling method further includes:
the automatic information collection method specifically comprises the following steps:
and acquiring the backup strategy and the backup data through the backup software API.
The backup strategy may be referred to above and will not be described again.
The backup data is that after the backup operation is initiated according to the preset starting time of the backup strategy, the backup of the data is successfully completed, and the related information of the data comprises the application of the data, the backup strategy, the backup time and the backup mirror image and the reserved medium information of the backup mirror image.
The backup image refers to the minimum unit of backup data.
When the backup software starts backup, the data to be backed up is compressed into a mirror image and stored in a designated backup medium.
The backup medium refers to the target environment in which the backup data is stored last, and can be on a tape or a disk.
A piece of backup data may comprise a backup image, and sometimes, in order to protect the data, the backup data needs to be stored in a redundant manner, so that the backup image is copied to other backup media. At this time, more than one backup image of the backup data exists.
The related information can be obtained from the backup software at regular time and updated to the database of the system, so as to provide basic data support for subsequent operation.
The method of the embodiment of the invention has the following beneficial effects:
1. The full-automatic backup data sampling inspection process is realized.
2. The method effectively reduces manual operation, thereby shortening the transfer operation time, reducing manual misoperation, effectively shortening the sampling inspection working time, simultaneously, independently and flexibly configuring the backup strategy, improving the sampling inspection coverage and enhancing the sampling inspection working effect.
It should be noted that, the data sampling inspection processing method provided by the embodiment of the invention can be used in the financial field and also can be used in any technical field except the financial field, and the application field of the data sampling inspection processing method is not limited by the embodiment of the invention.
Fig. 4 is a schematic structural diagram of a data sampling processing apparatus according to an embodiment of the present invention, and as shown in fig. 4, the data sampling processing apparatus according to an embodiment of the present invention includes an obtaining unit 401, an executing unit 402, and an updating unit 403, where:
The system comprises an acquisition unit 401, an execution unit 402 and an updating unit 403, wherein the acquisition unit 401 is used for acquiring the sampling test table information of the backup strategy, the sampling test table information comprises a sampling test state and a sampling test type, the execution unit 402 is used for acquiring data in the sampling test state and executing a command corresponding to the sampling test type on the data if the sampling test state is determined to be the sampling test state, and the updating unit 403 is used for acquiring an execution result of the command and updating the sampling test table information according to the execution result.
Specifically, the acquiring unit 401 in the device is configured to acquire the information of the sampling test table of the backup policy, where the sampling test table information includes a sampling test state and a sampling test type, the executing unit 402 is configured to acquire data in the sampling test state if the sampling test state is determined to be the sampling test state, execute a command corresponding to the sampling test type on the data, and the updating unit 403 is configured to acquire an execution result of the command and update the sampling test table information according to the execution result.
The data sampling inspection processing device provided by the embodiment of the invention acquires sampling inspection table information of a backup strategy, wherein the sampling inspection table information comprises a sampling inspection state and a sampling inspection type, acquires data in the sampling inspection state if the sampling inspection state is determined to be the sampling inspection state, executes a command corresponding to the sampling inspection type on the data, acquires an execution result of the command, and updates the sampling inspection table information according to the execution result, so that the sampling inspection of the data can be automatically completed, misoperation caused by manual operation is avoided, and the processing efficiency of a sampling inspection task is improved.
Further, the sampling test type includes media availability verification, and correspondingly, the execution unit 402 is specifically configured to:
and executing a verification sampling inspection command on the data.
The data sampling inspection processing device provided by the embodiment of the invention is suitable for the data sampling inspection processing of which the sampling inspection type is the medium availability verification.
Further, the sampling type includes data availability verification, and correspondingly, the execution unit 402 is specifically configured to:
and acquiring the backup strategy type of the data, and executing a recovery command on the data corresponding to the backup strategy type respectively.
The data sampling inspection processing device provided by the embodiment of the invention is suitable for data sampling inspection processing with sampling inspection type of data availability verification.
Further, the backup policy type includes a file backup policy type, and correspondingly, the executing unit 402 is further specifically configured to:
and restoring the latest backup image corresponding to the file backup strategy type into a verification environment, wherein the verification environment completes the check in advance.
The data sampling inspection processing device provided by the embodiment of the invention is suitable for data sampling inspection processing with the backup strategy type being the file backup strategy type.
Further, the backup policy type includes a database backup policy type, and correspondingly, the executing unit 402 is further specifically configured to:
and calling a database recovery script in a verification environment, and performing data interaction with backup software by using the database recovery script to realize database recovery, wherein the verification environment completes initialization setting in advance according to the database instance and the database parameters.
The data sampling inspection processing device provided by the embodiment of the invention is suitable for data sampling inspection processing with the backup strategy type being the database backup strategy type.
Further, the updating unit 403 is specifically configured to:
obtaining a restored file from the verification environment, and comparing the file with a file list in the latest backup image;
and if the comparison results of all the comparison items are the same, determining that the execution result is the completion of the execution, and updating the sampling inspection table information.
The data sampling inspection processing device provided by the embodiment of the invention can adapt to the data sampling inspection processing of the file backup strategy type.
Further, the updating unit 403 is specifically configured to:
Logging in the database in the verification environment, if the database is confirmed to be recovered, determining that the execution result is the execution completion, and updating the spot check list information.
The data sampling inspection processing device provided by the embodiment of the invention can be suitable for the data sampling inspection processing of the database backup strategy type.
Further, before the step of obtaining the sampling table information of the backup policy, the data sampling processing device is further configured to:
and acquiring a backup strategy, and executing a corresponding backup strategy updating step according to a comparison result of the backup strategy and the pre-stored backup strategy in the spot check list information.
The data sampling inspection processing device provided by the embodiment of the invention can ensure the synchronous consistency of the acquired backup strategy and the pre-stored backup strategy in the sampling inspection table information.
Further, the data sampling test processing device is further used for, after the step of executing the corresponding backup strategy updating step,:
determining the data sampling frequency and the sampling type according to the application grade and a preset corresponding relation, wherein the preset corresponding relation comprises a corresponding relation among the preset application grade, the preset data sampling frequency and the preset sampling type;
Acquiring the latest spot check completion time of the backup strategy, and determining the time interval between the current time and the latest spot check completion time;
And if the time interval is greater than or equal to the data sampling frequency, updating the sampling detection state into the to-be-sampled detection state.
The data sampling detection processing device provided by the embodiment of the invention can ensure the timeliness of sampling detection state update.
The embodiment of the data sampling processing device provided in the embodiment of the present invention may be specifically used to execute the processing flow of each method embodiment, and the functions thereof are not described herein again, and may refer to the detailed description of the method embodiments.
Fig. 5 is a schematic diagram of an entity structure of an electronic device according to an embodiment of the present invention, as shown in fig. 5, where the electronic device includes a processor (processor) 501, a memory (memory) 502, and a bus 503;
wherein, the processor 501 and the memory 502 complete communication with each other through a bus 503;
the processor 501 is configured to invoke the program instructions in the memory 502 to perform the methods provided in the above method embodiments, for example, including:
The method comprises the steps of obtaining the information of a sampling inspection table of a backup strategy, wherein the information of the sampling inspection table comprises a sampling inspection state and a sampling inspection type;
If the sampling detection state is determined to be the sampling detection state, acquiring data in the sampling detection state, and executing a command corresponding to the sampling detection type on the data;
and acquiring an execution result of the command, and updating the spot check list information according to the execution result.
The present embodiment discloses a computer program product comprising a computer program stored on a non-transitory computer readable storage medium, the computer program comprising program instructions which, when executed by a computer, are capable of performing the methods provided by the above-described method embodiments, for example comprising:
The method comprises the steps of obtaining the information of a sampling inspection table of a backup strategy, wherein the information of the sampling inspection table comprises a sampling inspection state and a sampling inspection type;
If the sampling detection state is determined to be the sampling detection state, acquiring data in the sampling detection state, and executing a command corresponding to the sampling detection type on the data;
and acquiring an execution result of the command, and updating the spot check list information according to the execution result.
The present embodiment provides a computer-readable storage medium storing a computer program that causes the computer to execute the methods provided by the above-described method embodiments, for example, including:
The method comprises the steps of obtaining the information of a sampling inspection table of a backup strategy, wherein the information of the sampling inspection table comprises a sampling inspection state and a sampling inspection type;
If the sampling detection state is determined to be the sampling detection state, acquiring data in the sampling detection state, and executing a command corresponding to the sampling detection type on the data;
and acquiring an execution result of the command, and updating the spot check list information according to the execution result.
It will be appreciated by those skilled in the art that embodiments of the present invention may be provided as a method, system, or computer program product. Accordingly, the present invention may take the form of an entirely hardware embodiment, an entirely software embodiment or an embodiment combining software and hardware aspects. Furthermore, the present invention may take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, and the like) having computer-usable program code embodied therein.
The present invention is described with reference to flowchart illustrations and/or block diagrams of methods, apparatus (systems) and computer program products according to embodiments of the invention. It will be understood that each flow and/or block of the flowchart illustrations and/or block diagrams, and combinations of flows and/or blocks in the flowchart illustrations and/or block diagrams, can be implemented by computer program instructions. These computer program instructions may be provided to a processor of a general purpose computer, special purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including instruction means which implement the function specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
In the description of the present specification, reference to the terms "one embodiment," "one particular embodiment," "some embodiments," "for example," "an example," "a particular example," or "some examples," etc., means that a particular feature, structure, material, or characteristic described in connection with the embodiment or example is included in at least one embodiment or example of the invention. In this specification, schematic representations of the above terms do not necessarily refer to the same embodiments or examples. Furthermore, the particular features, structures, materials, or characteristics described may be combined in any suitable manner in any one or more embodiments or examples.
The foregoing description of the embodiments has been provided for the purpose of illustrating the general principles of the invention, and is not meant to limit the scope of the invention, but to limit the invention to the particular embodiments, and any modifications, equivalents, improvements, etc. that fall within the spirit and principles of the invention are intended to be included within the scope of the invention.
Claims (10)
1. The data sampling inspection processing method is characterized by comprising the following steps of:
The method comprises the steps of obtaining the information of a sampling inspection table of a backup strategy, wherein the information of the sampling inspection table comprises a sampling inspection state and a sampling inspection type;
If the sampling detection state is determined to be the sampling detection state, acquiring data in the sampling detection state, and executing a command corresponding to the sampling detection type on the data;
Acquiring an execution result of the command, and updating the spot check list information according to the execution result;
Before the step of obtaining the sampling table information of the backup strategy, the method further comprises the following steps:
Acquiring a backup strategy, and executing a corresponding backup strategy updating step according to a comparison result of the backup strategy and a pre-stored backup strategy in the spot check list information, wherein the spot check list information comprises application grades of the backup strategy, and the application grades reflect the importance degree of the application;
determining the data sampling frequency and the sampling type according to the application grade and a preset corresponding relation, wherein the preset corresponding relation comprises a corresponding relation among the preset application grade, the preset data sampling frequency and the preset sampling type;
Acquiring the latest spot check completion time of the backup strategy, and determining the time interval between the current time and the latest spot check completion time;
And if the time interval is greater than or equal to the data sampling frequency, updating the sampling detection state into the to-be-sampled detection state.
2. The method of claim 1, wherein the type of spot check includes a media availability verification, and wherein the executing the command corresponding to the type of spot check on the data comprises:
and executing a verification sampling inspection command on the data.
3. The method of claim 1, wherein the type of sample includes a data availability verification, and wherein the executing the command corresponding to the type of sample on the data comprises:
and acquiring the backup strategy type of the data, and executing a recovery command on the data corresponding to the backup strategy type respectively.
4. The data sampling test processing method according to claim 3, wherein the backup policy type comprises a file backup policy type, and the executing the restore command on the data corresponding to the backup policy type respectively comprises:
and restoring the latest backup image corresponding to the file backup strategy type into a verification environment, wherein the verification environment completes the check in advance.
5. The data sampling test processing method according to claim 3, wherein the backup policy type comprises a database backup policy type, and the executing the restore command on the data corresponding to the backup policy type respectively comprises:
and calling a database recovery script in a verification environment, and performing data interaction with backup software by using the database recovery script to realize database recovery, wherein the verification environment completes initialization setting in advance according to the database instance and the database parameters.
6. The method of claim 4, wherein the obtaining the execution result of the command and updating the spot check list information according to the execution result comprises:
obtaining a restored file from the verification environment, and comparing the file with a file list in the latest backup image;
and if the comparison results of all the comparison items are the same, determining that the execution result is the completion of the execution, and updating the sampling inspection table information.
7. The method of claim 5, wherein the obtaining the execution result of the command and updating the spot check list information according to the execution result comprises:
Logging in the database in the verification environment, if the database is confirmed to be recovered, determining that the execution result is the execution completion, and updating the spot check list information.
8. A data spot check processing apparatus, comprising:
the acquisition unit is used for acquiring the information of the sampling inspection table of the backup strategy; the sampling test table information comprises sampling test states and sampling test types;
the execution unit is used for acquiring data in the sampling inspection state if the sampling inspection state is determined to be the sampling inspection state, and executing a command corresponding to the sampling inspection type on the data;
The updating unit is used for acquiring an execution result of the command and updating the spot check list information according to the execution result;
before the step of obtaining the sampling table information of the backup strategy, the data sampling processing device is further used for:
Acquiring a backup strategy, and executing a corresponding backup strategy updating step according to a comparison result of the backup strategy and a pre-stored backup strategy in the spot check list information, wherein the spot check list information comprises application grades of the backup strategy, and the application grades reflect the importance degree of the application;
determining the data sampling frequency and the sampling type according to the application grade and a preset corresponding relation, wherein the preset corresponding relation comprises a corresponding relation among the preset application grade, the preset data sampling frequency and the preset sampling type;
Acquiring the latest spot check completion time of the backup strategy, and determining the time interval between the current time and the latest spot check completion time;
And if the time interval is greater than or equal to the data sampling frequency, updating the sampling detection state into the to-be-sampled detection state.
9. An electronic device comprising a memory, a processor and a computer program stored on the memory and executable on the processor, characterized in that the processor implements the steps of the method of any one of claims 1 to 7 when the computer program is executed by the processor.
10. A computer readable storage medium, on which a computer program is stored, characterized in that the computer program, when being executed by a processor, implements the steps of the method according to any one of claims 1 to 7.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210529441.5A CN114840526B (en) | 2022-05-16 | 2022-05-16 | Data sampling processing method and device |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210529441.5A CN114840526B (en) | 2022-05-16 | 2022-05-16 | Data sampling processing method and device |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN114840526A CN114840526A (en) | 2022-08-02 |
| CN114840526B true CN114840526B (en) | 2025-09-12 |
Family
ID=82569948
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202210529441.5A Active CN114840526B (en) | 2022-05-16 | 2022-05-16 | Data sampling processing method and device |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN114840526B (en) |
Families Citing this family (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN115408180A (en) * | 2022-09-23 | 2022-11-29 | 中国农业银行股份有限公司 | Data processing method, device and medium |
Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN105930226A (en) * | 2015-12-30 | 2016-09-07 | 中国银联股份有限公司 | Data processing method and apparatus |
| CN112348521A (en) * | 2020-10-21 | 2021-02-09 | 上海淇玥信息技术有限公司 | Intelligent risk quality inspection method and system based on business audit and electronic equipment |
Family Cites Families (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN105573859A (en) * | 2014-10-15 | 2016-05-11 | 中兴通讯股份有限公司 | Data recovery method and device of database |
| CN111737244A (en) * | 2020-06-22 | 2020-10-02 | 平安医疗健康管理股份有限公司 | Data quality inspection method, device, computer system and storage medium |
| CN113987226A (en) * | 2021-07-21 | 2022-01-28 | 远光软件股份有限公司 | Hybrid storage method and device |
-
2022
- 2022-05-16 CN CN202210529441.5A patent/CN114840526B/en active Active
Patent Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN105930226A (en) * | 2015-12-30 | 2016-09-07 | 中国银联股份有限公司 | Data processing method and apparatus |
| CN112348521A (en) * | 2020-10-21 | 2021-02-09 | 上海淇玥信息技术有限公司 | Intelligent risk quality inspection method and system based on business audit and electronic equipment |
Also Published As
| Publication number | Publication date |
|---|---|
| CN114840526A (en) | 2022-08-02 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US7386752B1 (en) | Using asset dependencies to identify the recovery set and optionally automate and/or optimize the recovery | |
| CN108776643B (en) | Target code merging control method and system based on version control process | |
| EP2474910A1 (en) | Setting program, workflow creating method, and work flow creating apparatus | |
| CN111475401A (en) | Test method and related equipment | |
| CN109032838B (en) | Automatic verification method for consistency of backup and recovery data of virtual machine | |
| JP2017201470A (en) | Setting support program, setting support method, and setting support device | |
| CN113836001A (en) | Code detection method, device and storage medium | |
| CN113094362A (en) | Method and device for reliably delivering and processing asynchronous message | |
| CN114116493A (en) | Test data processing method, device, computer equipment and storage medium | |
| CN114840526B (en) | Data sampling processing method and device | |
| CN110780904A (en) | Application updating method and device | |
| CN118245282A (en) | Snapshot backup method and device | |
| CN110471681A (en) | Automatic variation, system, device and the storage medium of application program | |
| US20230236953A1 (en) | Dynamic debug log enabler for any protection failure jobs | |
| CN112148714B (en) | Data monitoring method, system, storage medium and electronic equipment | |
| CN114338363A (en) | Continuous integration method, device, equipment and storage medium | |
| CN111090491B (en) | Virtual machine task state recovery method and device and electronic equipment | |
| CN114817167B (en) | A method and device for processing merged data | |
| CN112487081A (en) | Data synchronization method, device, storage medium and equipment | |
| CN114371870B (en) | Code scanning and submitting method, code scanning server, client and server | |
| CN116185484A (en) | Method and device for realizing consistency of version library and version product data | |
| CN113157528B (en) | Operation fluctuation monitoring method and device based on big data service cloud | |
| JP2015172865A (en) | How to test the database | |
| CN112487010B (en) | Block chain user data table updating method, equipment and storage medium | |
| CN115393082A (en) | A blockchain-based transaction data reconciliation processing method and device |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |