[go: up one dir, main page]

CN117076200A - A metadata-based remote sensing data recovery method, device and storage medium - Google Patents

A metadata-based remote sensing data recovery method, device and storage medium Download PDF

Info

Publication number
CN117076200A
CN117076200A CN202311045885.2A CN202311045885A CN117076200A CN 117076200 A CN117076200 A CN 117076200A CN 202311045885 A CN202311045885 A CN 202311045885A CN 117076200 A CN117076200 A CN 117076200A
Authority
CN
China
Prior art keywords
data
target
remote sensing
refers
frame
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Granted
Application number
CN202311045885.2A
Other languages
Chinese (zh)
Other versions
CN117076200B (en
Inventor
岳国军
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Beijing Tianhua Xinghang Technology Co ltd
Original Assignee
Beijing Tianhua Xinghang Technology Co ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Beijing Tianhua Xinghang Technology Co ltd filed Critical Beijing Tianhua Xinghang Technology Co ltd
Priority to CN202311045885.2A priority Critical patent/CN117076200B/en
Publication of CN117076200A publication Critical patent/CN117076200A/en
Application granted granted Critical
Publication of CN117076200B publication Critical patent/CN117076200B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Classifications

    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F11/00—Error detection; Error correction; Monitoring
    • G06F11/07—Responding to the occurrence of a fault, e.g. fault tolerance
    • G06F11/14—Error detection or correction of the data by redundancy in operation
    • G06F11/1402—Saving, restoring, recovering or retrying
    • G06F11/1415—Saving, restoring, recovering or retrying at system level
    • G06F11/1435—Saving, restoring, recovering or retrying at system level using file system or storage system metadata
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F11/00—Error detection; Error correction; Monitoring
    • G06F11/07—Responding to the occurrence of a fault, e.g. fault tolerance
    • G06F11/14—Error detection or correction of the data by redundancy in operation
    • G06F11/1402—Saving, restoring, recovering or retrying
    • G06F11/1446—Point-in-time backing up or restoration of persistent data
    • Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
    • Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
    • Y02D—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN INFORMATION AND COMMUNICATION TECHNOLOGIES [ICT], I.E. INFORMATION AND COMMUNICATION TECHNOLOGIES AIMING AT THE REDUCTION OF THEIR OWN ENERGY USE
    • Y02D10/00—Energy efficient computing, e.g. low power processors, power management or thermal management

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Quality & Reliability (AREA)
  • Physics & Mathematics (AREA)
  • General Engineering & Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Library & Information Science (AREA)
  • Information Retrieval, Db Structures And Fs Structures Therefor (AREA)

Abstract

The invention provides a remote sensing data recovery method, equipment and storage medium based on metadata, which comprises the following steps: the method comprises the steps of obtaining a target remote sensing data list, obtaining a first intermediate file list according to the target remote sensing data list, obtaining a key filing information list, and recovering all target remote sensing data by reading one piece of information when a request for recovering the target remote sensing data is received, so that the working efficiency is improved.

Description

Remote sensing data recovery method, device and storage medium based on metadata
Technical Field
The present invention relates to the field of data processing, and in particular, to a remote sensing data recovery method, apparatus and storage medium based on metadata.
Background
With the development of information technology, the amount of information stored in a computer is continuously increased, and data backup and disaster recovery become hot topics. The data is very valuable, and to ensure the continuous operation and success of the information system, the information based on the computer is protected, and human errors, hard disk damage, computer viruses, natural disasters and the like can cause the loss of the data, so that no appreciable loss is caused. The system data loss can cause the loss of system files, transaction data, user data, technical files and financial accounts, and the business is difficult to normally process, so that important data needs to be backed up at any time in actual work.
In the field of mass data storage, according to the importance of data, entity data files and database information are required to be backed up simultaneously, so that rapid data reconstruction under the condition of disaster is prevented, and the processing requirement of a production system is met.
In the prior art, database backup software is generally adopted to carry out full-quantity backup or incremental backup on database information, entity data files are backed up through data archiving software, two different software are required to be adopted for backup, and steps are complicated when data are restored, so that the working efficiency is low.
Disclosure of Invention
Aiming at the technical problems, the invention adopts the following technical scheme:
a remote sensing data recovery method based on metadata comprises the following steps:
s1, acquiring a target remote sensing data list H= { H 1 ，H 2 ，……，H x ，……，H p }，H x The x-th target remote sensing data is referred to, x= … … p, p is the total number of the target remote sensing data;
s2, according to H, obtaining a first intermediate file list H0= { H0 corresponding to H 1 ，H0 2 ，……，H0 x ，……，H0 p }，H0 x Refers to H x A corresponding first intermediate file;
s21, obtaining H x Corresponding target archive information list C x ＝{C x 1 ，C x 2 ，……，C x i ，……C x n }， Refers to H x The starting byte position in the metadata corresponding to the ith target frame data,/for the metadata>Refers to H x Metadata mid-frame length corresponding to the ith target frame data,/and the like>Refers to H x Filing an identification in metadata corresponding to the ith target frame data;
wherein the method comprises the steps ofWhere a is the frame header length and b is H x A corresponding start frame position;
s22, will (C) 1 ，C 2 ……C x ) Write into key file and regard key file as H0 x It can be understood that: the key file is a file for storing metadata corresponding to target remote sensing data;
s3, acquiring a key archive information list G= { G corresponding to H according to H and H0 1 ，G 2 ，……，G x ，……，G p },G x ＝{H x ，H0 x ，G0 x }，G0 x Is H x Identification of the corresponding target tape;
s4, G x Sequentially writing into the storage area of the target tape to generate U= { H 1 ，H0 1 ，G0 1 ，……，H x ，H0 x ，G0 x ，……，H p ，H0 p ，G0 p }；
S5, when a request for recovering target remote sensing data is received, H0 in U is recovered p And analyzing the corresponding SQL sentences to restore the target remote sensing data.
The invention has at least the following beneficial effects:
H0 x including all the first intermediate files in the first x files, when the target remote sensing data needs to be recovered later, only H0 in U is needed p And reading the metadata to obtain the initial byte position, the frame length and the archiving identification of the corresponding metadata of all the target remote sensing data in the H, and recovering the metadata according to the information, so that the workload of recovering the data is reduced, and the working efficiency is improved.
Drawings
In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings required for the description of the embodiments will be briefly described below, and it is apparent that the drawings in the following description are only some embodiments of the present invention, and other drawings may be obtained according to these drawings without inventive effort for a person skilled in the art.
FIG. 1 is a flowchart of a remote sensing data recovery method based on metadata according to an embodiment of the present invention;
fig. 2 is another flowchart provided in an embodiment of the present invention.
Detailed Description
The following description of the embodiments of the present invention will be made clearly and completely with reference to the accompanying drawings, in which it is apparent that the embodiments described are only some embodiments of the present invention, but not all embodiments. All other embodiments, which can be made by those skilled in the art based on the embodiments of the invention without making any inventive effort, are intended to fall within the scope of the invention.
The embodiment provides a remote sensing data recovery method based on metadata, which comprises the following steps, as shown in fig. 1:
and S100, when receiving an archiving instruction of the target remote sensing data, archiving the target remote sensing data in the target tape.
Specifically, the target remote sensing data refers to strip data stored in an online storage medium; it can be understood that: the strip data are data acquired by remote sensing satellites during remote sensing.
Specifically, the archiving instruction refers to an instruction for archiving target telemetry data from an online storage medium to a magnetic tape.
Specifically, the step S100 further includes the following steps:
s101, acquiring a target archive information list A= { A corresponding to the target remote sensing data 1 ，A 2 ，……，A i ，……A n }，Ai＝{A i 1 ，A i 2 ，A i 3 }，A i 1 Refers to the ith destination in the target remote sensing dataStarting byte position A in metadata corresponding to standard frame data i 2 Refers to the frame length A in the metadata corresponding to the ith target frame data in the target remote sensing data i 3 The method is that an archive identifier is in metadata corresponding to the ith target frame data in the target remote sensing data, and i= … … n, wherein n is the total frame number of the target frame data in the target remote sensing data.
Further, A i 1 Meets the following conditions:
A i 1 ＝a+(b i -1)*A i 2 wherein a refers to the frame header length, b i The position of the initial frame of the ith target frame data in the target remote sensing data in the strip data is referred to.
Specifically, the A i 2 Are all equal.
When the target remote sensing data is archived from the online storage medium to the magnetic tape, the target archiving information list A, A corresponding to the target remote sensing data is obtained simultaneously i The starting byte position of the metadata corresponding to the ith target frame of the target remote sensing data, the frame length of the metadata and the corresponding archiving identification. When the target remote sensing data in the magnetic tape is required to be recovered, the starting byte position and the frame length of the metadata corresponding to each target frame data can be obtained by reading the target filing information list, so that the metadata corresponding to the target frame data is convenient to recover.
Furthermore, the type of the archiving identifier is a hash value, and any method for acquiring the data hash value in the prior art is known by a person skilled in the art, and falls into the protection scope of the present invention, and is not described herein again; the method for obtaining the data hash value can adopt an MD5 method.
S102, A is i The first intermediate file is stored as follows: the first intermediate file refers to a file for storing metadata corresponding to the target frame data.
Further, the format of the first intermediate file is xml format.
S103, when A i Corresponding target frame data as frame data to be processedIn this case, the intermediate archive information list a0= { a corresponding to the frame data to be acquired from a 1 ，A 2 ，……，A r ，……，A i-1 }，A r Refers to the r-th intermediate archive information, which refers to the archive information corresponding to the frame data to be transmitted.
S104, when A r 3 ＝A i 3 When A is r Corresponding pointer information is archived in the target tape.
S105, when A r 3 ≠A i 3 At the time, A i The corresponding first intermediate file is archived in the target tape.
The hash value can be used for converting the data with longer length into the character string with shorter length, fixed and not easy to repeat through the algorithm, and when A is used for i When the first intermediate file is stored in the first intermediate file, firstly judging A i Whether or not the archive identification of metadata corresponding to the previous target frame is identical to the archive identification of metadata corresponding to the previous target frame. If the archiving identification is the same as the archiving identification of any metadata, only the pointer information corresponding to the metadata is stored in the target tape, and all information is not required to be stored in the target tape, so that the occupation of the space of the target tape is reduced. When the metadata corresponding to the target frame data needs to be recovered, the target frame data can be found out from the over-read pointer information i Identical A r Will not be to A i Has an influence on the recovery of (a).
S200, when receiving an extraction instruction of target remote sensing data, extracting target data from the target tape.
Specifically, the extraction instruction refers to an instruction that will extract the telemetry data required by the user from the magnetic tape.
Specifically, the step S200 further includes the following steps:
s201, obtaining the data ID to be extracted and extraction information B= { B corresponding to the data ID to be extracted 1 ，B 2 ，B 3 }, wherein B is 1 Refers to the position of the initial frame corresponding to the data ID to be extracted, B 2 Refers to the position of the end frame corresponding to the data ID to be extracted, B 3 Refers to data I to be extractedD corresponds to the frame length.
Further, the data ID to be extracted refers to a unique identifier of the data to be extracted, where the data to be extracted refers to remote sensing data to be extracted by a user, for example, the data ID to be extracted is a name of the data to be extracted.
S202, when the storage state of the target remote sensing data is offline storage, acquiring the storage capacity D of each storage block in the target tape 0 。
S203 according to D 0 Obtaining intermediate extraction information D= { D corresponding to the data ID to be extracted 1 ，D 2 ，D 3 }，D 1 Refers to the initial storage block position corresponding to the data ID to be extracted, D 2 Refers to the offset of the storage block corresponding to the data ID to be extracted, D 3 Refers to the extracted data length corresponding to the data ID to be extracted.
Further, D 1 Meets the following conditions:
c is B 1 At the storage block location in the target tape.
Further, D2 meets the following conditions:
mod((((B 1 -1)*B 3 +a0+1)+c*D 0 +D 0 )，D 0 ) Mod () is a remainder function.
Further, D 3 Meets the following conditions:
D 3 ＝(B 2 -B 1 +1)*B 3 。
s204, extracting the coincidence D from the target tape 1 、D 2 And D 3 As target data.
In the above, the magnetic tape is a storage medium in which storage blocks are sequentially stored and read, and thus the size of the storage blocks needs to be taken into consideration when extracting target data from the target magnetic tape. The storage block size of the target tape in this embodiment is D 0 Refers to the total number of bytes capable of being stored in one storage block as D 0 . When the target magnet is neededWhen the tape is cut, the position D of the storage block of the initial frame of the data corresponding to the data ID to be extracted in the target tape is calculated first 1 Then calculating to obtain the position of the initial frame of the data corresponding to the data ID to be extracted in the initial storage block, and then calculating to obtain the data length D of the data corresponding to the data ID to be extracted 3 . Finally according to D 1 、D 2 、D 3 The data on the target magnetic tape is extracted to obtain the target data, so that the accurate extraction of the target data is realized, the target data is not required to be extracted after the data on the target magnetic tape are completely recovered, the working steps are reduced, and the working efficiency is improved.
S205, when the storage state of the target remote sensing data is online storage, according to B 1 And B 3 Determining a starting byte position B corresponding to the data ID to be extracted 0 Data length B corresponding to data ID to be extracted 4 。
Further, B 0 Meets the following conditions:
B 0 ＝a0+1+(B 1 -1)*B 3 wherein a0 refers to a frame header length corresponding to the data ID to be extracted.
Further, B 4 Meets the following conditions:
B 4 ＝(B 2 -B 1 +1)*B 3 。
s206, extracting the coincidence B0 and B from the target tape 4 As target data.
When the storage state of the target remote sensing data is online storage, the method comprises the following steps of 0 Cut B at byte position of (2) 4 The data of the length is taken as target data.
Specifically, any method for determining the storage state of remote sensing data in the prior art is known to those skilled in the art, which is not described herein in detail, and for example, the storage state is determined by using the remote sensing data extension attribute.
In a specific embodiment, the step S200 further includes the following steps:
and S300, storing the target data in a preset storage area, wherein the storage mode of the target data is one of a distributed storage mode, an object storage mode or a local storage mode.
S400, when the target data is stored in the preset storage area, the first metadata (B) 0 、B 1 、B 2 、B 3 And B 4 ) Or second metadata (D of the target data 0 ，D 1 ，D 2 ，D 3 ) Stored in the second intermediate file, it can be understood that: the second intermediate file refers to a file for storing metadata corresponding to the target data.
Specifically, the format of the second intermediate file is xml format.
S500, storing the second intermediate file in the preset storage area.
And when the target data needs to be recovered later, the second intermediate file is read to obtain the first metadata or the second metadata of the target data, so that the target data can be recovered conveniently.
In another specific embodiment, the computer program, when executed by a processor, further performs the following steps, as shown in fig. 2:
s1, acquiring a target remote sensing data list H= { H 1 ，H 2 ，……，H x ，……，H p }，H x The x-th target remote sensing data is referred to, and x= … … p, p is the total number of the target remote sensing data.
S2, according to H, obtaining a first intermediate file list H0= { H0 corresponding to H 1 ，H0 2 ，……，H0 x ，……，H0 p }，H0 x Refers to H x A corresponding first intermediate file.
Specifically, the format of the first intermediate file is xml format.
Specifically, the step S2 further includes the following steps of obtaining H0 x ：
S21, obtainTaking H x Corresponding target archive information list C x ＝{C x 1 ，C x 2 ，……，C x i ，……C x n }， Refers to H x The starting byte position in the metadata corresponding to the ith target frame data,/for the metadata>Refers to H x Metadata mid-frame length corresponding to the ith target frame data,/and the like>Refers to H x The metadata corresponding to the ith target frame data is filed with an identification.
Further, the method comprises the steps of,meets the following conditions:
where a is the frame header length and b is H x Corresponding start frame position.
When the target remote sensing data list H is obtained, the first intermediate file list H0 is obtained at the same time, according to H x Obtaining C x Obtaining H x The initial byte position, the frame length and the archiving identification of the metadata corresponding to the ith target frame data are convenient for recovering H through H0 when the target remote sensing data are recovered later.
Furthermore, the type of the archiving identifier is a hash value, and any method for acquiring the data hash value in the prior art is known by a person skilled in the art, and falls into the protection scope of the present invention, and is not described herein again; the method for obtaining the data hash value can adopt an MD5 method.
S22, will (C) 1 ，C 2 ……C x ) Write into key file and regard key file as H0 x It can be understood that: the key file is a file for storing metadata corresponding to the target remote sensing data.
Further, the format of the first intermediate file is xml format.
Further, C 1 To C x SQL statements are adopted in writing the key file.
S3, acquiring a key archive information list G= { G corresponding to H according to H and H0 1 ，G 2 ，……，G x ，……，G p },G x ＝{H x ，H0 x ，G0 x }，G0 x Is H x Identification of the corresponding target tape.
S4, G x Sequentially writing into the storage area of the target tape to generate U= { H 1 ，H0 1 ，G0 1 ，……，H x ，H0 x ，G0 x ，……，H p ，H0 p ，G0 p }。
S5, when a request for recovering target remote sensing data is received, H0 in U is recovered p Analyzing the corresponding SQL sentences to recover target remote sensing data; those skilled in the art will know that any method for parsing SQL statement belongs to the protection scope of the present invention, and will not be described in detail herein.
Above, H0 x Including all the first intermediate files in the first x files, when the target remote sensing data needs to be recovered later, only H0 in U is needed p And reading the metadata to obtain the initial byte position, the frame length and the archiving identification of the corresponding metadata of all the target remote sensing data in the H, and recovering the metadata according to the information, so that the workload of recovering the data is reduced, and the working efficiency is improved.
Embodiments of the present invention also provide a non-transitory computer readable storage medium that may be disposed in an electronic device to store at least one instruction or at least one program for implementing one of the methods embodiments, the at least one instruction or the at least one program being loaded and executed by the processor to implement the methods provided by the embodiments described above.
Embodiments of the present invention also provide an electronic device comprising a processor and the aforementioned non-transitory computer-readable storage medium.
While certain specific embodiments of the invention have been described in detail by way of example, it will be appreciated by those skilled in the art that the above examples are for illustration only and are not intended to limit the scope of the invention. Those skilled in the art will also appreciate that many modifications may be made to the embodiments without departing from the scope and spirit of the invention. The scope of the invention is defined by the appended claims.

Claims (9)

1.一种基于元数据的遥感数据恢复方法，其特征在于，所述方法包括如下步骤：1. A method for remote sensing data recovery based on metadata, characterized in that the method includes the following steps: S1，获取目标遥感数据列表H＝{H1，H2，……，Hx，……，Hp}，Hx是指第x个目标遥感数据，x＝1……p，p为目标遥感数据的总个数；S1, obtain the target remote sensing data list H = { H1 , H2 , ..., Hx , ..., Hp }, where Hx refers to the x-th target remote sensing data, x = 1 ... p, and p is the total number of target remote sensing data; S2，根据H，获取H对应的第一中间文件列表H0＝{H01，H02，……，H0x，……，H0p}，H0x是指Hx对应的第一中间文件；S2, Based on H, obtain the list of first intermediate files H0 = { H01 , H02 , ..., H0x , ..., H0p } corresponding to H, where H0x refers to the first intermediate file corresponding to Hx ; S21，获取Hx对应的目标归档信息列表Cx＝{Cx 1，Cx 2，……，Cx i，……Cx n}，Cx i＝{Cx i 1，Cx i 2，Cx i 3}，Cx i 1是指Hx中第i个目标帧数据对应的元数据中起始字节位置，Cx i 2是指Hx中第i个目标帧数据对应的元数据中帧长度，Cx i 3是指Hx中第i个目标帧数据对应的元数据中归档标识；S21, obtain the target archive information list Cx = { Cx1 , Cx2, ..., Cxi, ..., Cxn } corresponding to Hx , Cxi = {Cxi1, Cxi2, Cxi3}, where Cxi1 refers to the starting byte position in the metadata corresponding to the i-th target frame data in Hx, Cxi2 refers to the frame length in the metadata corresponding to the i - th target frame data in Hx, and Cxi3 refers to the archive identifier in the metadata corresponding to the i - th target frame data in Hx ; 其中Cx i 1＝a+1+(bx-1)*Cx i 2；其中，a是帧头长度，b是指Hx对应的起始帧位置；Where C x i 1 ＝a+1+(b x -1)*C x i 2 ; where a is the frame header length and b refers to the starting frame position corresponding to H x ; S22，将(C1，C2……Cx)写入关键文件中且将关键文件作为H0x，可以理解为：所述关键文件是指用于存储目标遥感数据对应的元数据的文件；S22, writing ( C1 , C2, ..., Cx ) into the key file and using the key file as H0x can be understood as: the key file refers to the file used to store the metadata corresponding to the target remote sensing data; S3，根据H和H0，获取到H对应的关键归档信息列表G＝{G1，G2，……，Gx，……，Gp},Gx＝{Hx，H0x，G0x}，G0x为Hx对应的目标磁带的标识；S3, based on H and H0, obtain the key archive information list G = { G1 , G2 , ..., Gx , ..., Gp } corresponding to H, Gx = { Hx , H0x , G0x }, where G0x is the identifier of the target tape corresponding to Hx ; S4，将Gx按照顺序写入至目标磁带的存储区域，生成U＝{H1，H01，G01，……，Hx，H0x，G0x，……，Hp，H0p，G0p}；S4, write Gx sequentially to the storage area of the target tape to generate U = { H1 , H01 , G01 , ..., Hx , H0x , G0x , ..., Hp , H0p , G0p }; S5，当接收到恢复目标遥感数据的请求时，对U中H0p对应的语句进行解析，恢复目标遥感数据。S5, when a request to restore the target remote sensing data is received, the statement corresponding to H0 p in U is parsed and the target remote sensing data is restored. 2.根据权利要求1所述的方法，其特征在于，所述方法在S1前还包括：2. The method according to claim 1, characterized in that, the method further includes, before S1: S100，当接收到目标遥感数据的归档指令时，将目标遥感数据归档于目标磁带中；S100: When an archiving instruction for the target remote sensing data is received, the target remote sensing data is archived in the target magnetic tape. S200，当接收到目标遥感数据的提取指令时，从所述目标磁带中提取目标数据；S200: When a command to extract target remote sensing data is received, the target data is extracted from the target magnetic tape; S200步骤中还包括如下步骤：The S200 procedure also includes the following steps: S201，获取待提取数据ID和待提取数据ID对应的提取信息B＝{B1，B2，B3}，其中，B1是指待提取数据ID对应的起始帧的位置，B2是指待提取数据ID对应的结束帧的位置，B3是指待提取数据ID对应的帧长度；S201, obtain the ID of the data to be extracted and the extraction information B = { B1 , B2 , B3 } corresponding to the ID of the data to be extracted, where B1 refers to the position of the starting frame corresponding to the ID of the data to be extracted, B2 refers to the position of the ending frame corresponding to the ID of the data to be extracted, and B3 refers to the frame length corresponding to the ID of the data to be extracted. S202，当目标遥感数据的存储状态为离线存储时，获取目标磁带中每一存储块的存储容量D0；S202, when the target remote sensing data is stored offline, obtain the storage capacity D0 of each storage block in the target magnetic tape; S203，根据D0，获取所述待提取数据ID对应的中间提取信息D＝{D1，D2，D3}，D1是指待提取数据ID对应的起始存储块位置，D2是指待提取数据ID对应的存储块偏移量，D3是指待提取数据ID对应的提取数据长度；S203, based on D0 , obtain the intermediate extraction information D = { D1 , D2 , D3 } corresponding to the data ID to be extracted, where D1 refers to the starting storage block position corresponding to the data ID to be extracted, D2 refers to the storage block offset corresponding to the data ID to be extracted, and D3 refers to the length of the extracted data corresponding to the data ID to be extracted. 其中，D1符合如下条件：Among them, D1 meets the following conditions: c为B1处于目标磁带中的存储块位置； c represents the location of the storage block B1 in the target magnetic tape; 进一步的，D2符合如下条件：Furthermore, D2 meets the following conditions: mod((((B1-1)*B3+a0+1)+c*D0+D0)，D0)，mod()为取余函数；mod((((B 1 -1)*B 3 +a0+1)+c*D 0 +D 0 ), D 0 ), mod() is the modulo function; 进一步的，D3符合如下条件：Furthermore, D3 meets the following conditions: D3＝(B2-B1+1)*B3； D3 = ( B2 - B1 + 1) * B3 ; S204，从所述目标磁带中提取符合D1、D2和D3的遥感数据作为目标数据。S204, extract remote sensing data conforming to D1 , D2 and D3 from the target magnetic tape as target data. 3.根据权利要求2所述的方法，其特征在于，所述方法还包括以下步骤：3. The method according to claim 2, characterized in that the method further comprises the following steps: S205，当目标遥感数据的存储状态为在线存储时，根据B1和B3，确定出所述待提取数据ID对应的起始字节位置B0以及待提取数据ID对应的数据长度B4；S205, when the storage status of the target remote sensing data is online storage, determine the starting byte position B0 corresponding to the data ID to be extracted and the data length B4 corresponding to the data ID to be extracted based on B1 and B3 ; 进一步的，B0符合如下条件：Furthermore, B0 meets the following conditions: B0＝a0+1+(B1-1)*B3，其中，a0是指待提取数据ID对应的帧头长度； B0 = a0 + 1 + ( B1 - 1) * B3 , where a0 refers to the length of the frame header corresponding to the data ID to be extracted; 进一步的，B4符合如下条件：Furthermore, B4 meets the following conditions: B4＝(B2-B1+1)*B3； B4 = ( B2 - B1 + 1) * B3 ; S206，从所述目标磁带中提取符合B0和B4的遥感数据作为目标数据。S206, extract remote sensing data conforming to B0 and B4 from the target magnetic tape as target data. 4.根据权利要求2所述的方法，其特征在于，所述方法还包括如下步骤：4. The method according to claim 2, characterized in that the method further comprises the following steps: S300，将所述目标数据存储于预先设置的存储区域上，其中，目标数据的存储方式为分布式存储方式、对象存储方式或者本地存储方式中一种。S300, the target data is stored in a pre-set storage area, wherein the target data is stored in one of the following ways: distributed storage, object storage, or local storage. 5.根据权利要求4所述的方法，其特征在于，所述方法还包括以下步骤：5. The method according to claim 4, characterized in that the method further comprises the following steps: S400，当所述目标数据存储于预先设置的存储区域时，将目标数据的第一元数据(B0、B1、B2、B3和B4)或者目标数据的第二元数据(D0，D1，D2，D3)存储于第二中间文件中，所述第二中间文件是指用于存储目标数据对应的元数据的文件；S400, when the target data is stored in a pre-set storage area, the first metadata ( B0 , B1 , B2 , B3 and B4 ) or the second metadata ( D0 , D1 , D2 , D3 ) of the target data is stored in a second intermediate file, whereby the second intermediate file is a file used to store the metadata corresponding to the target data. S500，将所述第二中间文件存储于所述预先设置的存储区域。S500, the second intermediate file is stored in the pre-set storage area. 6.根据权利要求1所述的方法，其特征在于，所述第一中间文件的格式为xml格式。6. The method according to claim 1, wherein the format of the first intermediate file is XML format. 7.根据权利要求2所述的方法，其特征在于，所述S100步骤还包括以下步骤：7. The method according to claim 2, wherein step S100 further comprises the following step: S101，获取所述目标遥感数据对应的目标归档信息列表A＝{A1，A2，……，Ai，……An}，Ai＝{Ai 1，Ai 2，Ai 3}，Ai 1是指所述目标遥感数据中第i个目标帧数据对应的元数据中起始字节位置，Ai 2是指所述目标遥感数据中第i个目标帧数据对应的元数据中帧长度，Ai 3是指所述目标遥感数据中第i个目标帧数据对应的元数据中归档标识，i＝1……n，n为目标遥感数据中目标帧数据的总帧数。S101, obtain the target archive information list A = { A1 , A2 , ..., Ai , ... , An } corresponding to the target remote sensing data, Ai = { Ai1 , Ai2 , Ai3 }, where Ai1 refers to the starting byte position in the metadata corresponding to the i-th target frame data in the target remote sensing data, Ai2 refers to the frame length in the metadata corresponding to the i-th target frame data in the target remote sensing data, and Ai3 refers to the archive identifier in the metadata corresponding to the i - th target frame data in the target remote sensing data, i = 1...n, where n is the total number of target frame data in the target remote sensing data. 进一步的，Ai 1符合如下条件：Furthermore, A<sub> i </sub> 1 satisfies the following conditions: Ai 1＝a+(bi-1)*Ai 2，其中，a是指帧头长度，bi是指目标遥感数据中第i个目标帧数据的起始帧在长条带数据中的位置；A <sub>i1 </sub> = a + (b <sub>i -1</sub>) * A <sub>i2 </sub> , where a refers to the frame header length and b<sub> i </sub> refers to the position of the starting frame of the i-th target frame in the long strip data. S102，将Ai存入至第一中间文件中，可以理解为：所述第一中间文件是指用于存储目标帧数据对应的元数据的文件；S102, store Ai into the first intermediate file, which can be understood as: the first intermediate file is a file used to store the metadata corresponding to the target frame data; S103，当Ai对应的目标帧数据作为待帧数据时，从A中获取待帧数据对应的中间归档信息列表A0＝{A1，A2，……，Ar，……，Ai-1}，Ar是指第r个中间归档信息，所述中间归档信息是指待帧数据对应的归档信息；S103, when the target frame data corresponding to Ai is used as the frame data to be framed, the intermediate archive information list A0 = { A1 , A2 , ..., Ar , ..., Ai -1 } corresponding to the frame data to be framed is obtained from A, where Ar refers to the r-th intermediate archive information, and the intermediate archive information refers to the archive information corresponding to the frame data to be framed. S104，当Ar 3＝Ai 3时，Ar对应的指针信息归档于目标磁带中；S104, when Ar3 = Ai3 , the pointer information corresponding to Ar is archived in the target tape; S105，当Ar 3≠Ai 3时，将Ai对应的第一中间文件归档于目标磁带中。S105, when Ar3 ≠ Ai3 , archive the first intermediate file corresponding to Ai into the target tape. 8.一种非瞬时性计算机可读存储介质，所述存储介质中存储有至少一条指令或至少一段程序，所述至少一条指令或所述至少一段程序由处理器加载并执行以实现如权利要求1-8中任意一项的所述方法。8. A non-transitory computer-readable storage medium storing at least one instruction or at least one program segment, said at least one instruction or said at least one program segment being loaded and executed by a processor to implement the method of any one of claims 1-8. 9.一种电子设备，其特征在于，包括处理器和权利要求9中所述的非瞬时性计算机可读存储介质。9. An electronic device, characterized in that it comprises a processor and a non-transitory computer-readable storage medium as described in claim 9.
CN202311045885.2A 2023-08-18 2023-08-18 A remote sensing data recovery method, device and storage medium based on metadata Active CN117076200B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202311045885.2A CN117076200B (en) 2023-08-18 2023-08-18 A remote sensing data recovery method, device and storage medium based on metadata

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202311045885.2A CN117076200B (en) 2023-08-18 2023-08-18 A remote sensing data recovery method, device and storage medium based on metadata

Publications (2)

Publication Number Publication Date
CN117076200A true CN117076200A (en) 2023-11-17
CN117076200B CN117076200B (en) 2024-04-12

Family

ID=88703702

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202311045885.2A Active CN117076200B (en) 2023-08-18 2023-08-18 A remote sensing data recovery method, device and storage medium based on metadata

Country Status (1)

Country Link
CN (1) CN117076200B (en)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN117093541A (en) * 2023-08-18 2023-11-21 北京天华星航科技有限公司 A data extraction and archiving system for remote sensing data

Citations (12)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2004003912A1 (en) * 2002-06-28 2004-01-08 Koninklijke Philips Electronics N.V. Method and arrangement for the generation of an identification data block
US20060010227A1 (en) * 2004-06-01 2006-01-12 Rajeev Atluri Methods and apparatus for accessing data from a primary data storage system for secondary storage
CN106133717A (en) * 2014-06-25 2016-11-16 深圳市大疆创新科技有限公司 Multimedia file restorative procedure and equipment
CN107046647A (en) * 2017-05-14 2017-08-15 华中科技大学 A kind of control method of multispectral remote sensing image real-time decompression device
US20190251402A1 (en) * 2018-02-13 2019-08-15 Slingshot Aerospace, Inc. Adaptive neural network selection to extract particular results
CN113761012A (en) * 2020-06-05 2021-12-07 义乌市时空人工智能有限公司 Analysis visualization method of remote sensing data, server and storage medium
CN114244833A (en) * 2022-02-24 2022-03-25 中国科学院空天信息创新研究院 A method for real-time transmission of remote sensing satellite raw data using FTP protocol
CN114860558A (en) * 2022-04-21 2022-08-05 中国资源卫星应用中心 A method to facilitate remote sensing satellite data archiving and recovery performance testing
CN115525602A (en) * 2021-06-25 2022-12-27 华为技术有限公司 Data processing method and related device
CN115630028A (en) * 2022-12-12 2023-01-20 中国科学院空天信息创新研究院 Remote sensing image filing management method, device, equipment and storage medium
CN115964224A (en) * 2022-12-29 2023-04-14 北京易华录信息技术股份有限公司 Index metadata backup method and device and electronic equipment
CN116452461A (en) * 2023-05-08 2023-07-18 中国长江三峡集团有限公司 A remote sensing data processing method, device, electronic equipment and storage medium

Patent Citations (12)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2004003912A1 (en) * 2002-06-28 2004-01-08 Koninklijke Philips Electronics N.V. Method and arrangement for the generation of an identification data block
US20060010227A1 (en) * 2004-06-01 2006-01-12 Rajeev Atluri Methods and apparatus for accessing data from a primary data storage system for secondary storage
CN106133717A (en) * 2014-06-25 2016-11-16 深圳市大疆创新科技有限公司 Multimedia file restorative procedure and equipment
CN107046647A (en) * 2017-05-14 2017-08-15 华中科技大学 A kind of control method of multispectral remote sensing image real-time decompression device
US20190251402A1 (en) * 2018-02-13 2019-08-15 Slingshot Aerospace, Inc. Adaptive neural network selection to extract particular results
CN113761012A (en) * 2020-06-05 2021-12-07 义乌市时空人工智能有限公司 Analysis visualization method of remote sensing data, server and storage medium
CN115525602A (en) * 2021-06-25 2022-12-27 华为技术有限公司 Data processing method and related device
CN114244833A (en) * 2022-02-24 2022-03-25 中国科学院空天信息创新研究院 A method for real-time transmission of remote sensing satellite raw data using FTP protocol
CN114860558A (en) * 2022-04-21 2022-08-05 中国资源卫星应用中心 A method to facilitate remote sensing satellite data archiving and recovery performance testing
CN115630028A (en) * 2022-12-12 2023-01-20 中国科学院空天信息创新研究院 Remote sensing image filing management method, device, equipment and storage medium
CN115964224A (en) * 2022-12-29 2023-04-14 北京易华录信息技术股份有限公司 Index metadata backup method and device and electronic equipment
CN116452461A (en) * 2023-05-08 2023-07-18 中国长江三峡集团有限公司 A remote sensing data processing method, device, electronic equipment and storage medium

Non-Patent Citations (4)

* Cited by examiner, † Cited by third party
Title
XINNI LIU.ET.AL.: "A review of Convolutional Neural Networks in Remote Sensing Image", 《ICSCA \'19》, pages 263 - 267 *
刘庆利;潘成胜;田野;窦雪荔;: "基于CCSDS建议的遥感图像传输系统性能研究", 计算机应用, no. 10 *
李斌斌;冯钟葵;唐梦辉;: "遥感数据异地归档方法研究与实现", 遥感信息, no. 02 *
缪嘉嘉;付印金;毛捍东;: "KingCloud:智能对象归档系统", 计算机科学, no. 2 *

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN117093541A (en) * 2023-08-18 2023-11-21 北京天华星航科技有限公司 A data extraction and archiving system for remote sensing data
CN117093541B (en) * 2023-08-18 2024-02-23 北京天华星航科技有限公司 A data extraction and archiving system for remote sensing data

Also Published As

Publication number Publication date
CN117076200B (en) 2024-04-12

Similar Documents

Publication Publication Date Title
US7366859B2 (en) Fast incremental backup method and system
CN105009067B (en) Manage operations on storage data units
CN105027071B (en) Manage the operation to data storage unit
US9904601B2 (en) Synchronization of storage using comparisons of fingerprints of blocks
US7831789B1 (en) Method and system for fast incremental backup using comparison of descriptors
US7680998B1 (en) Journaled data backup during server quiescence or unavailability
CN110543446B (en) A Snapshot-Based Direct Archiving Method for Blockchain
CN107506260B (en) A kind of dynamic division database incremental backup method
CN111241062B (en) Method and device for migrating database backup metadata
CN103744936A (en) Multi-version concurrency control method in database and database system
JP6632380B2 (en) Managing operations on stored data units
CN111858164B (en) Continuous data protection method, device and equipment and readable storage medium
CN114090332A (en) Data processing method and device
CN103838645B (en) Remote difference synthesis backup method based on Hash
WO2020119143A1 (en) Database deleted record recovery method and system
CN117076200B (en) A remote sensing data recovery method, device and storage medium based on metadata
CN109388617B (en) Method and device for judging reliability of file timestamp
CN113971002A (en) Block chain based data storage method, device, equipment and readable storage medium
CN113761059A (en) Data processing method and device
CN117093541B (en) A data extraction and archiving system for remote sensing data
CN111221801A (en) A database migration method, system and related device
CN112328433B (en) Processing method, device, electronic device and storage medium for archival data recovery
CN113742097B (en) A data differential identification coding synchronization method, device, equipment and medium
CN116795296A (en) A data storage method, storage device and computer-readable storage medium
CN116185711A (en) Data backup and recovery method and device

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant
PE01 Entry into force of the registration of the contract for pledge of patent right

Denomination of invention: A metadata based remote sensing data recovery method, device, and storage medium

Granted publication date: 20240412

Pledgee: Pudong Development Bank of Shanghai Limited by Share Ltd. Beijing branch

Pledgor: BEIJING TIANHUA XINGHANG TECHNOLOGY Co.,Ltd.

Registration number: Y2024110000246

PE01 Entry into force of the registration of the contract for pledge of patent right
PC01 Cancellation of the registration of the contract for pledge of patent right

Granted publication date: 20240412

Pledgee: Pudong Development Bank of Shanghai Limited by Share Ltd. Beijing branch

Pledgor: BEIJING TIANHUA XINGHANG TECHNOLOGY Co.,Ltd.

Registration number: Y2024110000246

PC01 Cancellation of the registration of the contract for pledge of patent right
PE01 Entry into force of the registration of the contract for pledge of patent right

Denomination of invention: A metadata based remote sensing data recovery method, device, and storage medium

Granted publication date: 20240412

Pledgee: Pudong Development Bank of Shanghai Limited by Share Ltd. Beijing branch

Pledgor: BEIJING TIANHUA XINGHANG TECHNOLOGY Co.,Ltd.

Registration number: Y2026990000021