CN110362534B - Snapshot verification method and system, electronic equipment and storage medium - Google Patents
Snapshot verification method and system, electronic equipment and storage medium Download PDFInfo
- Publication number
- CN110362534B CN110362534B CN201910663003.6A CN201910663003A CN110362534B CN 110362534 B CN110362534 B CN 110362534B CN 201910663003 A CN201910663003 A CN 201910663003A CN 110362534 B CN110362534 B CN 110362534B
- Authority
- CN
- China
- Prior art keywords
- file
- snapshot
- abstract
- content
- information
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/10—File systems; File servers
- G06F16/11—File system administration, e.g. details of archiving or snapshots
- G06F16/128—Details of file system snapshots on the file-level, e.g. snapshot creation, administration, deletion
-
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/10—File systems; File servers
- G06F16/18—File system types
- G06F16/182—Distributed file systems
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Data Mining & Analysis (AREA)
- Databases & Information Systems (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
- Debugging And Monitoring (AREA)
Abstract
The application discloses a snapshot verification method, a snapshot verification system, an electronic device and a computer readable storage medium, wherein the snapshot verification method comprises the following steps: when the verification time is reached, obtaining a summary file; each abstract file is a file generated according to each piece of snapshot information in a memory, each piece of snapshot information is information generated according to each writing request, and the snapshot information comprises a writing position in the writing request, a data size and a time stamp when the writing request is received; extracting first file contents according to writing positions in each abstract file, and calculating second file contents according to each abstract file; and verifying a snapshot by comparing the first file content with the second file content. According to the snapshot verification method, the storage space is saved while the snapshot is verified.
Description
Technical Field
The present application relates to the field of storage technologies, and in particular, to a snapshot verification method and system, an electronic device, and a computer readable storage medium.
Background
In the snapshot feature of a distributed storage system, it is necessary to continuously record changes in file content. In the prior art, the snapshot is triggered manually for the written file content, and snapshot verification is performed by comparing the md5 value of the original file content with the md5 value of the snapshot file content.
In order to improve the snapshot verification granularity, the writing data is divided into data blocks with preset sizes, the writing data is stored in a blocking mode, and the snapshot verification granularity can be reduced to the data blocks. Due to the large number of data blocks, a large number of md5 values need to be stored by adopting the scheme, so that memory overflow is caused.
Therefore, how to save storage space while performing snapshot verification is a technical problem that needs to be solved by those skilled in the art.
Disclosure of Invention
The object of the present application is to provide a snapshot verification method, a snapshot verification system, an electronic device and a computer readable storage medium, which save storage space while performing snapshot verification.
In order to achieve the above object, the present application provides a snapshot verification method, including:
when the verification time is reached, obtaining a summary file; each abstract file is a file generated according to each piece of snapshot information in a memory, each piece of snapshot information is information generated according to each writing request, and the snapshot information comprises a writing position in the writing request, a data size and a time stamp when the writing request is received;
extracting first file contents according to writing positions in each abstract file, and calculating second file contents according to each abstract file;
and verifying a snapshot by comparing the first file content with the second file content.
The snapshot information further comprises a snapshot version when the writing request is received, and the snapshot version is updated according to a preset rule when the snapshot operation is performed every time.
Wherein, still include:
when a target writing request is received, generating target snapshot information according to the target writing request and storing the target snapshot information into the memory;
and calculating the writing content according to the target snapshot information, dividing the writing content into data blocks with preset sizes, and starting filling from the target writing position.
Wherein after receiving the target write request, the method further comprises:
acquiring data in the target writing position, calculating first snapshot information corresponding to the data, and taking a snapshot version in the first snapshot information as a first snapshot version;
taking the latest snapshot version in the second snapshot information corresponding to the data in the memory as a second snapshot version;
judging whether the first snapshot version is consistent with the second snapshot version or not;
if yes, executing the step of generating snapshot information according to the writing request and storing the snapshot information into the memory.
The obtaining the abstract file comprises the following steps:
determining the abstract file according to the current abstract file or the comprehensive abstract file;
the method comprises the steps that a current abstract file is generated according to current snapshot information, the current snapshot information is snapshot information generated from last verification time to current verification time, the comprehensive abstract file comprises the current abstract file and a non-current abstract file, the non-current abstract file is obtained by obtaining the abstract file corresponding to non-current snapshot information, and the non-current snapshot information is snapshot information generated before the last verification time.
Wherein said verifying the snapshot by comparing the first file content with the second file content comprises:
judging whether the first file content is consistent with the second file content or not;
if yes, judging that the snapshot verification is passed;
if not, generating a summary file corresponding to the first file content as a reference summary file, and determining an error type by comparing the reference summary file with the summary file.
Wherein said determining an error type by comparing said reference summary file with said summary file comprises:
judging whether the content of the reference abstract file except the time stamp and the snapshot version in the abstract file is consistent with the content of the reference abstract file;
if so, judging whether the reference abstract file is consistent with the snapshot version in the abstract file; if yes, judging that the error type is IO disordered, and if not, judging that the error type is snapshot missing;
if the error types are inconsistent, the error types are judged to be data errors.
To achieve the above object, the present application provides a snapshot verification system, including:
the acquisition module is used for acquiring the abstract file when the verification time is reached; each abstract file is a file generated according to each piece of snapshot information in a memory, each piece of snapshot information is information generated according to each writing request, and the snapshot information comprises a writing position in the writing request, a data size and a time stamp when the writing request is received;
the first calculation module is used for extracting first file contents according to the writing position in each abstract file and calculating second file contents according to each abstract file;
and the comparison module is used for verifying the snapshot by comparing the first file content with the second file content.
To achieve the above object, the present application provides an electronic device, including:
a memory for storing a computer program;
and a processor for implementing the steps of the snapshot verification method as described above when executing the computer program.
To achieve the above object, the present application provides a computer readable storage medium having stored thereon a computer program which, when executed by a processor, implements the steps of the snapshot verification method as described above.
According to the scheme, the snapshot verification method provided by the application comprises the following steps: when the verification time is reached, obtaining a summary file; each abstract file is a file generated according to each piece of snapshot information in a memory, each piece of snapshot information is information generated according to each writing request, and the snapshot information comprises a writing position in the writing request, a data size and a time stamp when the writing request is received; extracting first file contents according to writing positions in each abstract file, and calculating second file contents according to each abstract file; and verifying a snapshot by comparing the first file content with the second file content.
According to the snapshot verification method, firstly, the abstract file generated according to each piece of snapshot information is obtained, and each piece of snapshot information corresponds to a file writing request. For multiple write operations of the same data file, each write operation corresponds to one snapshot information, so that one data file can correspond to multiple summary files, and each summary file can be distinguished by a timestamp. By comparing the file content directly extracted from the writing position with the file content verification snapshot generated according to the abstract file, the abstract file occupies less memory, and a large number of md5 values are not required to be stored in the memory, so that the memory space is saved. The application also discloses a snapshot verification system, electronic equipment and a computer readable storage medium, and the technical effects can be achieved.
It is to be understood that both the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the application.
Drawings
In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings that are required in the embodiments or the description of the prior art will be briefly described below, it being obvious that the drawings in the following description are only some embodiments of the present application, and that other drawings may be obtained according to these drawings without inventive effort for a person skilled in the art. The accompanying drawings are included to provide a further understanding of the disclosure, and are incorporated in and constitute a part of this specification, illustrate the disclosure and together with the description serve to explain, but do not limit the disclosure. In the drawings:
FIG. 1 is a flowchart illustrating a snapshot verification method according to an exemplary embodiment;
FIG. 2 is a flowchart illustrating another snapshot verification method in accordance with an exemplary embodiment;
FIG. 3 is a flowchart illustrating yet another snapshot verification method in accordance with an exemplary embodiment;
FIG. 4 is a block diagram of a snapshot verification system shown in accordance with an exemplary embodiment;
fig. 5 is a block diagram of an electronic device, according to an example embodiment.
Detailed Description
The following description of the embodiments of the present application will be made clearly and fully with reference to the accompanying drawings, in which it is evident that the embodiments described are only some, but not all, of the embodiments of the present application. All other embodiments, which can be made by one of ordinary skill in the art based on the embodiments herein without making any inventive effort, are intended to be within the scope of the present application.
The embodiment of the application discloses a snapshot verification method, which saves storage space while carrying out snapshot verification.
Referring to fig. 1, a flowchart of a snapshot verification method according to an exemplary embodiment is shown, as shown in fig. 1, including:
s101: when the verification time is reached, obtaining a summary file;
each abstract file is a file generated according to each piece of snapshot information in a memory, each piece of snapshot information is information generated according to each writing request, and the snapshot information comprises a writing position in the writing request, a data size and a time stamp when the writing request is received;
the embodiment can be applied to a distributed storage system, and aims to verify a snapshot in the system, and an execution subject can be a processor in the distributed storage system, and more particularly can be a snapshot verification object. The verification time is not specifically limited, that is, the embodiment may be a timing trigger, and the snapshot verification may be triggered every preset time period, or may be a manual trigger, where the verification time may also include other specific moments, for example, when the writing operation is abnormal, which will be described in detail in the following embodiments.
The snapshot file records the writing operation of all data files in the system between the last verification time and the current verification time, and can be understood as the collection of all snapshot information in the time period. Snapshot information is generated according to a write request, and includes a write location in the write request, a data size, a timestamp when the write request is received, and may also include a random seed (key), and so on. The writing position is the starting position, and since the writing data is divided into data blocks during writing, the number of the data blocks occupied by the writing data can be determined according to the size of the data blocks and the data size of the writing data. Preferably, the snapshot information may further include a snapshot version at the time of receiving the write request. In a specific implementation, a snapshot version (snapID) is set for each snapshot file, and the snapshot version is updated according to a preset rule at each snapshot operation. Preferably, the snapshot files are sequentially incremented according to a preset increment gradient, for example, snapID of the snapshot file in the last verification time to the current verification time is 1, then snapID of the snapshot file in the current verification time to the next verification time is 2, and the increment gradient of snapID is 1. Of course, those skilled in the art may choose other incremental gradients and incremental manners, as long as the continuity of the snapshot version increment is ensured.
It can be understood that all writing operations in the storage system need to be suspended when the snapshot is verified, the snapshot center (snappeer) sends a broadcast to all file writing objects (fileband for creating files and writing operations to files), the file writing objects stop the writing operations when receiving the broadcast and reply to the snapshot center, and the snapshot center executes the step of obtaining the summary file after receiving reply messages of all file writing objects. And after the snapshot is verified successfully, recovering the writing operation of the file writing object.
When the snapshot verification is carried out, firstly, a summary file is obtained, and the summary file is generated according to the snapshot information in the memory. Each snapshot information corresponds to one abstract file, and for a plurality of write operations of the same data file, each write operation corresponds to one snapshot information, so that one data file corresponds to a plurality of abstract files, each abstract file can be distinguished according to different snapshot versions during the write operation, and if the abstract files are write operations generated in the same snapshot period, the abstract files can be distinguished according to time stamps.
Preferably, the step includes: determining the abstract file according to the current abstract file or the comprehensive abstract file; the method comprises the steps that a current abstract file is generated according to current snapshot information, the current snapshot information is snapshot information generated from last verification time to current verification time, the comprehensive abstract file comprises the current abstract file and a non-current abstract file, the non-current abstract file is obtained by obtaining the abstract file corresponding to non-current snapshot information, and the non-current snapshot information is snapshot information generated before the last verification time.
That is, the summary file in this step may include only the current summary file generated from the snapshot information generated from the last verification time to the current verification time, and this embodiment is only used for verifying the snapshot in the above-described period (the current verification period). The summary file may further include a non-current summary file generated according to snapshot information generated before the last verification time, and this embodiment may verify all snapshots in the system, which may be understood that, since the non-current summary file is already generated in other verification periods, the verification period may be directly obtained from the memory. Naturally, in order to save the memory, the regeneration may be performed in the present verification period, and the present verification period is not particularly limited.
In a specific implementation, the types of write operations of the file write object include write, create, delete, and the like, and for the write process, the following is specific: when a target writing request is received, generating target snapshot information according to the target writing request and storing the target snapshot information into the memory; and calculating the writing content according to the target snapshot information, dividing the writing content into data blocks with preset sizes, and starting filling from the target writing position. It can be seen that the written content is calculated by using an md5 algorithm according to the snapshot information, and the snapshot of the written content is verified, and the summary file generated by the snapshot information is correspondingly adopted. In addition, when the file is written, the written content is subjected to blocking operation, and the error data block can be specified during snapshot verification. The predetermined size is preferably 512 bytes, i.e. a verification granularity of 512 bytes can be achieved.
S102: extracting first file contents according to writing positions in each abstract file, and calculating second file contents according to each abstract file;
in a specific implementation, each summary file includes a writing position corresponding to a writing operation, and file content of the writing position is extracted as first file content. It will be appreciated that since the writing location included in the writing request is the start address, which corresponds to one data block, the first file content in this step is the file content in one data block. And simultaneously, calculating the corresponding second file content of each abstract file obtained in the previous step by using an md5 algorithm, wherein the second file content is also the file content in a data block corresponding to the writing position in the abstract file.
S103: and verifying a snapshot by comparing the first file content with the second file content.
In this step, the first file content is compared with the second file content, if the first file content is consistent with the second file content, the snapshot information corresponding to the summary file is verified, and if the first file content is inconsistent with the second file content, the summary file obtained in the step S101 and the summary file reversely analyzed according to the first file content can be displayed. The reverse resolution here can be understood as the inverse of the md5 algorithm described above. Because the first file content and the second file content both represent the file content of one data block, when the snapshot generates errors, the specific error data block can be determined, and the error reporting granularity is the data block.
In addition, when the snapshot information includes a snapshot version and the summary file in step S101 includes all the summary files in the system, it may also be continuously determined whether there is a snapshot loss according to whether the snapshot version in all the summary files is lost.
According to the snapshot verification method provided by the embodiment of the application, firstly, the abstract file generated according to each piece of snapshot information is obtained, and each piece of snapshot information corresponds to a file writing request. For multiple write operations of the same data file, each write operation corresponds to one snapshot information, so that one data file can correspond to multiple summary files, and each summary file can be distinguished by a timestamp. By comparing the file content directly extracted from the writing position with the file content verification snapshot generated according to the abstract file, the abstract file occupies less memory, and a large number of md5 values are not required to be stored in the memory, so that the memory space is saved.
The following describes the data writing process of the distributed storage system, specifically:
referring to fig. 2, a flowchart of another snapshot verification method, as shown in fig. 2, is shown, including:
s201: when a target writing request is received, acquiring data in a target writing position, calculating first snapshot information corresponding to the data, and taking a snapshot version in the first snapshot information as a first snapshot version;
before data is written, in order to ensure continuity of snapshot versions, firstly, data in a target writing position is extracted, snapshot information corresponding to the data is calculated by using the inverse process of an md5 algorithm, namely first snapshot information in the step, and the snapshot version in the first snapshot information is used as the first snapshot version.
S202: taking the latest snapshot version in the second snapshot information corresponding to the data in the memory as a second snapshot version;
s203: judging whether the first snapshot version is consistent with the second snapshot version or not; if yes, go to S204;
in this step, whether the first snapshot version is consistent with the second snapshot version is compared, if so, it is indicated that the snapshot versions are continuous, S203 is entered, if not, an error is reported, or step S101 in the previous embodiment may be performed, and the lost snapshot version may be determined according to the continuity of the snapshot versions in all the summary files.
S204: generating target snapshot information according to the target writing request and storing the target snapshot information into the memory;
in this step, a plurality of file write objects may be invoked to write to each file in the distributed storage system. And generating snapshot information and storing the snapshot information into a memory for each writing operation so as to carry out snapshot verification when verification time arrives.
S205: and calculating the writing content according to the target snapshot information, dividing the writing content into data blocks with preset sizes, and starting filling from the target writing position.
In this step, the writing content of the data block is calculated according to the snapshot information, and correspondingly, the digest file corresponding to the snapshot information is also used for calculation and comparison during snapshot verification.
Therefore, the embodiment compares whether the snapshot version corresponding to the data stored in the writing position is consistent with the snapshot version in the snapshot information stored in the memory before writing operation, and ensures that the snapshot versions are continuous. The data is written in the unit of data block, and the specific error data block can be determined during the snapshot verification, so that the error reporting granularity of the snapshot verification is reduced.
The embodiment of the application discloses a snapshot verification method, and compared with the first embodiment, the embodiment further describes and optimizes a technical scheme. Specific:
referring to fig. 3, a flowchart of yet another snapshot verification method, as shown in fig. 3, is shown, including:
s301: when the verification time is reached, obtaining a summary file;
s302: extracting first file contents according to writing positions in each abstract file, and calculating second file contents according to each abstract file;
s303: judging whether the first file content is consistent with the second file content or not; if yes; then S304 is entered; if not, then enter S305;
s304: judging that the snapshot verification is passed;
s305: and generating a summary file corresponding to the first file content as a reference summary file, and determining an error type by comparing the reference summary file with the summary file.
In this step, when the snapshot is in error, the reference digest file reversely parsed according to the first file content is compared with the digest file obtained in step S301 to determine the error type. The error types may include data errors, i.e. the first file content is completely inconsistent with the second file content, IO disorder, i.e. the time of the system is disordered, snapshot loss, i.e. the snapshot version in the system is discontinuous.
Specifically, the step of determining the error type by comparing the reference digest file with the digest file may include:
s51: judging whether the content of the reference abstract file except the time stamp and the snapshot version in the abstract file is consistent with the content of the reference abstract file; if so, the process proceeds to S52: if not, entering S55;
s52: judging whether the reference abstract file is consistent with the snapshot version in the abstract file or not; if yes, go to S53; if not, entering S54;
s53: judging the error type as IO disorder;
s54; determining the error type as snapshot missing;
s55: the type of error is determined to be a data error.
In a specific implementation, if the two summary files are inconsistent with the time stamp, the error type may be determined to be IO out of order, and if the two summary files have inconsistent snapshot versions, the error type may be determined to be snapshot loss.
Therefore, the embodiment can determine the error type by comparing the reference abstract file reversely analyzed according to the first file content with the obtained abstract file, and realizes the accurate identification of the snapshot error.
The following describes a snapshot verification system provided in an embodiment of the present application, and a snapshot verification system described below and a snapshot verification method described above may be referred to mutually.
Referring to FIG. 4, a block diagram of a snapshot verification system is shown according to an exemplary embodiment, as shown in FIG. 4, comprising:
an obtaining module 401, configured to obtain a summary file when the verification time arrives; each abstract file is a file generated according to each piece of snapshot information in a memory, each piece of snapshot information is information generated according to each writing request, and the snapshot information comprises a writing position in the writing request, a data size and a time stamp when the writing request is received;
a first calculation module 402, configured to extract first file contents according to a writing location in each summary file, and calculate second file contents according to each summary file;
a comparison module 403, configured to verify the snapshot by comparing the first file content with the second file content.
The snapshot verification system provided by the embodiment of the application firstly obtains the abstract file generated according to each piece of snapshot information, and each piece of snapshot information corresponds to a file writing request. For multiple write operations of the same data file, each write operation corresponds to one snapshot information, so that one data file can correspond to multiple summary files, and each summary file can be distinguished by a timestamp. By comparing the file content directly extracted from the writing position with the file content verification snapshot generated according to the abstract file, the abstract file occupies less memory, and a large number of md5 values are not required to be stored in the memory, so that the memory space is saved.
On the basis of the foregoing embodiment, as a preferred implementation manner, the snapshot information further includes a snapshot version when the write request is received, where the snapshot version is updated according to a preset rule at each snapshot operation.
On the basis of the above embodiment, as a preferred implementation manner, the method further includes:
the generating module is used for generating target snapshot information according to the target writing request and storing the target snapshot information into the memory when the target writing request is received;
and the writing module is used for calculating writing content according to the target snapshot information, dividing the writing content into data blocks with preset sizes and starting filling from the target writing position.
On the basis of the above embodiment, as a preferred implementation manner, the method further includes:
the second calculation module is used for acquiring data in the target writing position, calculating first snapshot information corresponding to the data, and taking a snapshot version in the first snapshot information as a first snapshot version;
the determining module is used for taking the latest snapshot version in the second snapshot information corresponding to the data in the memory as a second snapshot version;
the judging module is used for judging whether the first snapshot version is consistent with the second snapshot version or not; if yes, starting the workflow of the generating module.
On the basis of the above embodiment, as a preferred implementation manner, the obtaining module 401 is specifically a module that determines the summary file according to the current summary file or the comprehensive summary file when the verification time is reached; the method comprises the steps that a current abstract file is generated according to current snapshot information, the current snapshot information is snapshot information generated from last verification time to current verification time, the comprehensive abstract file comprises the current abstract file and a non-current abstract file, the non-current abstract file is obtained by obtaining the abstract file corresponding to non-current snapshot information, and the non-current snapshot information is snapshot information generated before the last verification time.
On the basis of the above embodiment, as a preferred implementation manner, the comparing module 403 includes:
the judging unit is used for judging whether the first file content is consistent with the second file content; if yes, starting the workflow of the judging unit; if not, starting the workflow of the determining unit;
the judging unit is used for judging that the snapshot verification passes;
the determining unit is configured to generate a summary file corresponding to the first file content as a reference summary file, and determine an error type by comparing the reference summary file with the summary file.
On the basis of the above embodiment, as a preferred implementation manner, the determining unit includes:
a generating subunit, configured to generate a summary file corresponding to the first file content as a reference summary file;
a first judging subunit, configured to judge whether the content of the reference summary file except the timestamp and the snapshot version in the summary file is consistent with the content of the reference summary file; if the two judging sub-units are consistent, starting the working flow of the second judging sub-unit; if the first judging subunit is inconsistent, starting the workflow of the third judging subunit;
the second judging subunit is configured to judge whether the reference summary file is consistent with a snapshot version in the summary file; if yes, starting the workflow of the first judging subunit; if not, starting the workflow of the second judging subunit;
the first judging subunit is used for judging that the error type is IO disordered;
the second judging subunit is used for judging that the error type is snapshot missing;
and the third judging subunit is used for judging the error type as the data error.
The specific manner in which the various modules perform the operations in relation to the systems of the above embodiments have been described in detail in relation to the embodiments of the method and will not be described in detail herein.
The present application also provides an electronic device, referring to fig. 5, and a block diagram of an electronic device 500 provided in an embodiment of the present application, as shown in fig. 5, may include a processor 11 and a memory 12. The electronic device 500 may also include one or more of a multimedia component 13, an input/output (I/O) interface 14, and a communication component 15.
Wherein the processor 11 is configured to control the overall operation of the electronic device 500 to perform all or part of the steps in the snapshot verification method described above. The memory 12 is used to store various types of data to support operation at the electronic device 500, which may include, for example, instructions for any application or method operating on the electronic device 500, as well as application-related data, such as contact data, messages sent and received, pictures, audio, video, and so forth. The Memory 12 may be implemented by any type or combination of volatile or non-volatile Memory devices, such as static random access Memory (Static Random Access Memory, SRAM for short), electrically erasable programmable Read-Only Memory (Electrically Erasable Programmable Read-Only Memory, EEPROM for short), erasable programmable Read-Only Memory (Erasable Programmable Read-Only Memory, EPROM for short), programmable Read-Only Memory (Programmable Read-Only Memory, PROM for short), read-Only Memory (ROM for short), magnetic Memory, flash Memory, magnetic disk, or optical disk. The multimedia component 13 may include a screen and an audio component. Wherein the screen may be, for example, a touch screen, the audio component being for outputting and/or inputting audio signals. For example, the audio component may include a microphone for receiving external audio signals. The received audio signals may be further stored in the memory 12 or transmitted through the communication component 15. The audio assembly further comprises at least one speaker for outputting audio signals. The I/O interface 14 provides an interface between the processor 11 and other interface modules, which may be a keyboard, mouse, buttons, etc. These buttons may be virtual buttons or physical buttons. The communication component 15 is used for wired or wireless communication between the electronic device 500 and other devices. Wireless communication, such as Wi-Fi, bluetooth, near field communication (Near Field Communication, NFC for short), 2G, 3G or 4G, or a combination of one or more thereof, the corresponding communication component 15 may thus comprise: wi-Fi module, bluetooth module, NFC module.
In an exemplary embodiment, the electronic device 500 may be implemented by one or more application specific integrated circuits (Application Specific Integrated Circuit, abbreviated ASIC), digital signal processor (Digital Signal Processor, abbreviated DSP), digital signal processing device (Digital Signal Processing Device, abbreviated DSPD), programmable logic device (Programmable Logic Device, abbreviated PLD), field programmable gate array (Field Programmable Gate Array, abbreviated FPGA), controller, microcontroller, microprocessor, or other electronic components for performing the snapshot verification method described above.
In another exemplary embodiment, a computer readable storage medium is also provided, comprising program instructions which, when executed by a processor, implement the steps of the snapshot verification method described above. For example, the computer readable storage medium may be the memory 12 described above including program instructions executable by the processor 11 of the electronic device 500 to perform the snapshot verification method described above.
In the description, each embodiment is described in a progressive manner, and each embodiment is mainly described by the differences from other embodiments, so that the same similar parts among the embodiments are mutually referred. For the system disclosed in the embodiment, since it corresponds to the method disclosed in the embodiment, the description is relatively simple, and the relevant points refer to the description of the method section. It should be noted that it would be obvious to those skilled in the art that various improvements and modifications can be made to the present application without departing from the principles of the present application, and such improvements and modifications fall within the scope of the claims of the present application.
It should also be noted that in this specification, relational terms such as first and second, and the like are used solely to distinguish one entity or action from another entity or action without necessarily requiring or implying any actual such relationship or order between such entities or actions. Moreover, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising one … …" does not exclude the presence of other like elements in a process, method, article, or apparatus that comprises the element.
Claims (10)
1. A snapshot verification method, comprising:
when the verification time is reached, obtaining a summary file; each abstract file is a file generated according to each piece of snapshot information in a memory, each piece of snapshot information is information generated according to each writing request, and the snapshot information comprises a writing position in the writing request, a data size and a time stamp when the writing request is received;
extracting first file contents according to writing positions in each abstract file, and calculating second file contents according to each abstract file; the first file content is file content in a data block, the second file content is file content in a data block, and the data block is obtained by performing block division operation on the written content;
and verifying a snapshot by comparing the first file content with the second file content.
2. The snapshot verification method according to claim 1, wherein the snapshot information further includes a snapshot version upon receipt of the write request, the snapshot version being updated according to a preset rule at each snapshot operation.
3. The snapshot verification method according to claim 2, further comprising:
when a target writing request is received, generating target snapshot information according to the target writing request and storing the target snapshot information into the memory;
and calculating the writing content according to the target snapshot information, dividing the writing content into data blocks with preset sizes, and starting filling from the target writing position.
4. The snapshot verification method of claim 3, further comprising, after receiving the target write request:
acquiring data in the target writing position, calculating first snapshot information corresponding to the data, and taking a snapshot version in the first snapshot information as a first snapshot version;
taking the latest snapshot version in the second snapshot information corresponding to the data in the memory as a second snapshot version;
judging whether the first snapshot version is consistent with the second snapshot version or not;
if yes, executing the step of generating snapshot information according to the writing request and storing the snapshot information into the memory.
5. The snapshot verification method according to claim 1, wherein said obtaining a digest file includes:
determining the abstract file according to the current abstract file or the comprehensive abstract file;
the method comprises the steps that a current abstract file is generated according to current snapshot information, the current snapshot information is snapshot information generated from last verification time to current verification time, the comprehensive abstract file comprises the current abstract file and a non-current abstract file, the non-current abstract file is obtained by obtaining the abstract file corresponding to non-current snapshot information, and the non-current snapshot information is snapshot information generated before the last verification time.
6. The snapshot verification method according to any one of claims 2 to 5, wherein said verifying a snapshot by comparing said first file content with said second file content comprises:
judging whether the first file content is consistent with the second file content or not;
if yes, judging that the snapshot verification is passed;
if not, generating a summary file corresponding to the first file content as a reference summary file, and determining an error type by comparing the reference summary file with the summary file.
7. The snapshot verification method of claim 6, wherein said determining an error type by comparing said reference digest file to said digest file comprises:
judging whether the content of the reference abstract file except the time stamp and the snapshot version in the abstract file is consistent with the content of the reference abstract file;
if so, judging whether the reference abstract file is consistent with the snapshot version in the abstract file; if yes, judging that the error type is IO disordered, and if not, judging that the error type is snapshot missing;
if the error types are inconsistent, the error types are judged to be data errors.
8. A snapshot verification system, comprising:
the acquisition module is used for acquiring the abstract file when the verification time is reached; each abstract file is a file generated according to each piece of snapshot information in a memory, each piece of snapshot information is information generated according to each writing request, and the snapshot information comprises a writing position in the writing request, a data size and a time stamp when the writing request is received;
the first calculation module is used for extracting first file contents according to the writing position in each abstract file and calculating second file contents according to each abstract file; the first file content is file content in a data block, the second file content is file content in a data block, and the data block is obtained by performing block division operation on the written content;
and the comparison module is used for verifying the snapshot by comparing the first file content with the second file content.
9. An electronic device, comprising:
a memory for storing a computer program;
a processor for implementing the steps of the snapshot verification method according to any one of claims 1 to 7 when executing the computer program.
10. A computer readable storage medium, characterized in that the computer readable storage medium has stored thereon a computer program which, when executed by a processor, implements the steps of the snapshot verification method according to any of claims 1 to 7.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910663003.6A CN110362534B (en) | 2019-07-22 | 2019-07-22 | Snapshot verification method and system, electronic equipment and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910663003.6A CN110362534B (en) | 2019-07-22 | 2019-07-22 | Snapshot verification method and system, electronic equipment and storage medium |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN110362534A CN110362534A (en) | 2019-10-22 |
| CN110362534B true CN110362534B (en) | 2023-05-12 |
Family
ID=68220708
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201910663003.6A Active CN110362534B (en) | 2019-07-22 | 2019-07-22 | Snapshot verification method and system, electronic equipment and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN110362534B (en) |
Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US8954383B1 (en) * | 2012-06-29 | 2015-02-10 | Emc Corporation | Analyzing mapping objects of file systems |
| US9405483B1 (en) * | 2014-03-11 | 2016-08-02 | Amazon Technologies, Inc. | Optimized write performance at block-based storage during volume snapshot operations |
Family Cites Families (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US6993539B2 (en) * | 2002-03-19 | 2006-01-31 | Network Appliance, Inc. | System and method for determining changes in two snapshots and for transmitting changes to destination snapshot |
| US7440966B2 (en) * | 2004-02-12 | 2008-10-21 | International Business Machines Corporation | Method and apparatus for file system snapshot persistence |
| US7457832B2 (en) * | 2004-08-31 | 2008-11-25 | Microsoft Corporation | Verifying dynamically generated operations on a data store |
| US20100114832A1 (en) * | 2008-10-31 | 2010-05-06 | Lillibridge Mark D | Forensic snapshot |
| US9696940B1 (en) * | 2013-12-09 | 2017-07-04 | Forcepoint Federal Llc | Technique for verifying virtual machine integrity using hypervisor-based memory snapshots |
| US10769024B2 (en) * | 2015-07-31 | 2020-09-08 | Netapp Inc. | Incremental transfer with unused data block reclamation |
| US9703643B2 (en) * | 2015-11-25 | 2017-07-11 | International Business Machines Corporation | Calculation of representative values for similarity units in deduplication of snapshots data |
| CN109597794A (en) * | 2018-12-05 | 2019-04-09 | 郑州云海信息技术有限公司 | A kind of file management method, system and electronic equipment and storage medium |
| CN110018999B (en) * | 2019-04-15 | 2023-07-11 | 深信服科技股份有限公司 | File management method and system, electronic equipment and storage medium |
-
2019
- 2019-07-22 CN CN201910663003.6A patent/CN110362534B/en active Active
Patent Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US8954383B1 (en) * | 2012-06-29 | 2015-02-10 | Emc Corporation | Analyzing mapping objects of file systems |
| US9405483B1 (en) * | 2014-03-11 | 2016-08-02 | Amazon Technologies, Inc. | Optimized write performance at block-based storage during volume snapshot operations |
Also Published As
| Publication number | Publication date |
|---|---|
| CN110362534A (en) | 2019-10-22 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN110275916B (en) | Data operation record storage method, system, device and equipment | |
| CN107491383A (en) | Catching method, device and the mobile terminal of application crashes Operation Log | |
| US20120296878A1 (en) | File set consistency verification system, file set consistency verification method, and file set consistency verification program | |
| CN110442473B (en) | Nonvolatile data storage method and device, electronic equipment and medium | |
| US20150067316A1 (en) | Electronic device and testing method | |
| CN111475401A (en) | Test method and related equipment | |
| CN107391357A (en) | Application crashes replay method, device and mobile terminal | |
| US20190227710A1 (en) | Incremental data restoration method and apparatus | |
| CN109284331B (en) | Certificate making information acquisition method based on service data resources, terminal equipment and medium | |
| CN105095074B (en) | Upgrade test method and device for configuration file | |
| CN111625402A (en) | Data recovery method and device, electronic equipment and computer readable storage medium | |
| CN107861832B (en) | Data verification method and device and readable storage medium | |
| CN117979118A (en) | Data stream recording method, device, recorder and recording system | |
| CN107729541A (en) | A kind of data processing method, device and computer-readable recording medium | |
| CN111625853B (en) | Snapshot processing method, device and equipment and readable storage medium | |
| CN114036002B (en) | Blockchain data backup method, device, storage medium and electronic device | |
| CN108196975B (en) | Data verification method and device based on multiple checksums and storage medium | |
| CN110362534B (en) | Snapshot verification method and system, electronic equipment and storage medium | |
| CN112988776B (en) | Method, device and equipment for updating text parsing rule and readable storage medium | |
| CN111078418B (en) | Operation synchronization method, device, electronic equipment and computer readable storage medium | |
| WO2021168654A1 (en) | Method and apparatus for detecting synchronized data, server, and data synchronization system | |
| CN113613063B (en) | Application anomaly reduction method, device and storage medium | |
| CN108052570B (en) | Conference system configuration information storage method, reading method and device | |
| CN107092671B (en) | Method and equipment for managing meta information | |
| CN112948020B (en) | Configuration file processing method and device |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |