JP6671708B2 - Backup restore system and backup restore method - Google Patents

Description

  The present invention generally relates to backup and / or restore, for example, both.

  In order to improve disaster resistance, a system for backing up and restoring through a network has been constructed. As shown in Non-Patent Document 1, a primary site backs up data to a remote data center. When a disaster occurs at the primary site and the primary site is restored, the restored primary site restores the backed up data from the data center.

  Further, there is known a technology for improving efficiency by processing backup and restore based on update difference data of a file. For example, as disclosed in Patent Document 1, an updated file data block is backed up as difference data for managing update differences.

WO2007 / 131190 US6604118

  According to the technology disclosed in Non-Patent Document 1, it takes time to restore data. Specifically, after the primary site damaged by the disaster is physically restored, data is transmitted from the remote data center to the restored primary site. At this time, it takes time to transfer the data due to the limitation of the network bandwidth.

  Even with the technique disclosed in Patent Literature 1, when the difference data is large, the same problem as in Non Patent Literature 1 occurs. Further, for restoration, it is necessary to first send the base snapshot that is the source of the difference to the restored primary site, which increases the data transfer time.

  Further, if the base snapshot is lost, restoration using the difference becomes impossible, so that it is necessary for the remote data site to store a plurality of base snapshots to improve disaster resistance. However, doing so increases the backup data capacity.

  A first object server for backing up one or more backup objects each of which is a file or a directory to at least one of a plurality of backup servers, and a first object server of one or more backup objects each of which is a file or a directory for a plurality of backup servers At least one of a second object server to be restored from at least one is provided. The one or more objects to be restored may be one or more objects to be backed up. Each of the one or more objects to be backed up and each of the one or more objects to be restored are composed of object data and metadata about the object data.

The first object server, for each of the object data and metadata in each of the one or more objects to be backed up,
Determine whether the target data, which is the data and the object data or the metadata, satisfies the first condition or the second condition,
When the target data satisfies the first condition, one or more replicated data of the target data is backed up to two or more backup servers,
When the target data satisfies the second condition, two or more pieces of data that are fragment data (V is a value greater than 1) of the redundancy V according to EC (Erasure Coding) of the target data are backed up to two or more backup servers. . Note that the fragment data having the redundancy V of the target data may not be fragment data complying with EC. For example, if the duplicate data of the fragment data is included in two or more pieces of data, even if the fragment data is lost, it can be replaced with the duplicate data, so that the redundancy of the fragment data is realized.

  The first object server, for each of the one or more backup target objects, stores information on the position of the backup target object in the first file system space and the backup destination of the backup target object in at least one of the plurality of backup servers. Registration in the management information in one. The first file system space is a file system space (for example, a name space) including one or more backup target objects. The management information may be stored in a shared storage area other than the plurality of backup servers (a storage area accessible by both the first and second object servers).

  The second object server specifies, for each of the one or more objects to be restored, the position of the object to be restored in the second file system space and the backup destination of the object to be restored by referring to the management information described above. I do. The second file system space is a file system space including one or more objects to be restored, and may be, for example, the first file system space to be restored.

The second object server, for each of the one or more objects to be restored,
As metadata of a target object that is the object, at least one copy of the metadata or fragment data of the metadata is received from at least one of the plurality of backup servers,
Receiving, as object data of the target object, at least one copy of the object data or fragment data of the object data from at least one of the plurality of backup servers;
The target object is restored to the position in the second file system space of the target object based on the received metadata and the received object data.

  The second object server may be an object server after replacement (after restoration) of the first object server, or may be an object server added for the purpose of scaling out the first object server.

  High restore performance and backup data capacity reduction can both be achieved. More specifically, a copy of data that satisfies the first condition (for example, generally small-sized data such as metadata) is backed up, and data that satisfies the second condition (for example, object data such as object data). Data that can be large in size) is divided into a plurality of fragmented data and backed up, so that the time required to transfer one or more objects to be restored from a plurality of backup servers can be reduced. In addition, since the data that satisfies the second condition is encoded according to EC (Erasure Coding), the data capacity can be reduced while maintaining redundancy.

1 shows a configuration of a backup / restore system according to a first embodiment. 1 shows a configuration of a file server. 2 shows the configuration of a backup server. 4 shows a configuration of a backup setting table. 4 shows an example of the configuration of a backup management table. It is a schematic diagram of a restoration process. 4 shows a flow of a backup process. 10 shows a flow of a restore process. 9 illustrates a configuration of a backup mode table according to the second embodiment. 9 illustrates a flow of a backup method determination process according to the second embodiment. 13 illustrates a configuration of a backup mode table according to the third embodiment. 13 illustrates a flow of a backup method determination process according to the third embodiment.

  Hereinafter, some embodiments will be described.

  In the following description, information may be described in an expression such as “xxx table” in the following description, but the information may be expressed in any data structure. That is, "xxx table" can be referred to as "xxx information" to indicate that the information does not depend on the data structure. Further, in the following description, the configuration of each table is an example, and one table may be divided into two or more tables, or even if all or some of the two or more tables are one table. Good.

  Further, in the following description, processing may be described with “program” as the subject, but the program is stored in a processor (for example, a CPU (Central Processing Unit)) so that the determined processing is appropriately stored. The processing may be performed using a resource (for example, a memory) and / or a communication interface device (for example, a communication port), or the like, and the subject of the processing may be a processor. The processor may include a hardware circuit that performs part or all of the processing.The program may be installed in each controller from a program source. Program distribution computer or computer readable憶 may be media. In the following description, to 2 or more programs may be implemented as a single program, a single program may be implemented as two or more programs.

  In the following description, reference numerals (or common parts in reference numerals) are used when describing the same kind of elements without distinguishing them, and when discriminating and describing the same kind of elements, the element IDs (or Element reference number). For example, when the file server is described without particular distinction, it is described as “file server 105”, and when the individual file server is described separately, it is represented as “file server 105A” or “file server 105B”. May be described.

  FIG. 1 illustrates a configuration of the backup / restore system according to the first embodiment.

  The backup / restore system 100 is located at a client 101, a primary site 103, a management server 104 at the primary site 103, a file server 105A, a backup site 106, a backup server 107 at the backup site 106, a recovery site 108, and a recovery site 108. It has a file server 105B. There are a plurality of backup servers 107. The backup / restore system 100 is an example of a computer system and includes the file servers 105A and 105B, and is therefore called a backup / restore system. If there is no file server 105B, the computer system including the file server 105A may be called a backup system. On the other hand, if there is no file server 105A, the computer system including the file server 105B may be called a restore system.

  The client 101, the primary site 103, the backup site 106, and the recovery site 108 are mutually connected by the communication path 102. The communication path 102 may be, for example, a communication network such as a LAN (Local Area Network) or the Internet.

  The client 101 is a terminal (computer) that uses the file server 105A or 105B. An end user using the client 101 connects to the file server 105A or the file server 105B using the file access program 110 of the client 101, and reads and writes a file stored in the file server 105A or the file server 105B.

  The primary site 103 is a site that provides a file access service to the client 101 before a disaster occurs.

  The management server 104 of the primary site 103 is a terminal (computer) that manages the entire backup / restore system 100. The administrator of the backup / restore system 100 sets the backup / restore system 100 using the management server 104. For example, the administrator uses the management server 104 to set the IP address of the backup site 106A in the file server 105A. Note that the management server 104 may be arranged at an arbitrary position such as the backup site 106 or the recovery site 108.

  The file server 105A in the primary site 103 is a computer that provides a file access service to the client 101. The file server 105A is an example of a first object server. The file access service may be of any type, for example, NFS (Network File System) or CIFS (Common Internet File System). The file server 105A of the main site 103 is, for example, a server operated before a disaster occurs.

  The backup site 106 is a site that stores backup data. The backup server 107 of the backup site 106 stores at least a part of the object data (for example, file data) of the object (file or directory) held by the file server 105 and its metadata (specifically, So that at least one of the duplicate data, fragment data, and parity data) is backed up. The backup server 107 receives data from the file server 105A regularly or irregularly, and stores the received data in a disk device (an example of a nonvolatile storage device) of the backup server 107. The operations of the file server 105 and the backup server 107 will be described later in detail. Further, since one backup server 107 exists at one backup site 106, the backup site 106 and the backup server 107 are substantially synonymous in this embodiment.

  The recovery site 108 is a site that provides a file access service to the client 101 after a disaster occurs.

  The file server 105B of the recovery site 108 is a computer that provides a file access service to the client 101 after a disaster occurs. The file server 105B is an example of a second object server. When the primary site 103 is physically damaged due to a disaster, the file server 105B of the recovery site 108 takes over the service. Therefore, the file server 105A of the primary site 103 regularly (or irregularly) backs up (saves) the data held by the file server 105A to the backup server 107 of the backup site 106. After the disaster, the file server 105B of the recovery site 108 acquires data (backed up data) from the backup server 107 and restarts the file access service. The client 101 uses the resumed file service.

  In the backup / restore system 100, the file server 105A of the primary site 103 determines a backup method for each of the object data and the metadata in each of the one or more backup target objects. It is determined which of the first condition and the second condition is satisfied.

  Generally, small-sized data such as metadata satisfies the first condition. In this case, the file server 105A multiplexes the data to a plurality of backup servers 107.

  On the other hand, data that can be generally large in size, such as object data, corresponds to the second condition. In this case, the file server 105A backs up a plurality of fragment data obtained by dividing the data to a plurality of backup servers 107. As data division, encoding according to EC (Erasure Coding) is adopted in order to improve disaster resistance.

  When data is restored, at least one backup server 107 transfers duplicate data to the file server 105B for data that satisfies the first condition. Restoration is performed using the duplicated data. In the present embodiment, the first condition is that the data is metadata. Metadata is restored prior to file data. In other words, the metadata can be restored immediately after the disaster. In addition, since metadata is generally small-sized data (for example, data of about several kilobytes), the capacity overhead is small.

  For data that satisfies the second condition, fragment data is transferred from the plurality of backup servers 107 to the file server 105B in parallel. Therefore, the data transfer time is reduced as compared with the case where data (typically, file data) is transferred from one backup server 107. In addition, since EC is adopted for data division, a wide band is realized in backup and restoration, and capacity overhead can be suppressed. Data is restored by decoding using a plurality of fragment data.

  Hereinafter, Example 1 will be described in detail.

  FIG. 2 shows the configuration of the file server 105.

  The file server 105 has a network I / F (interface) 201, a CPU 202, a disk device 203, and a memory 205. They are interconnected by an internal communication path 204.

  The network I / F 201 is a device that is mutually connected to the communication path 102 and is used when receiving a file access request from the client 101. The CPU 202 is a device that executes a program stored in the memory 205, and is an example of a processor. The disk device 203 is a storage device that stores program files and files created by end users. The memory 205 is a device that holds programs and data structures. The program is stored in the disk device 203 and is read into the memory 205 when the CPU 202 executes the program. Hereinafter, the program is executed by the CPU unless otherwise specified. The program is read from the disk device into the memory and executed.

  The file sharing server program 206 is a program that processes a file access request of the file access program 110 of the client 101. The file access request is, for example, reading of a file (READ) or writing of data to a file (WRITE). The file data received by the file access request is stored in the disk device 203. The file data is backed up to the backup server 107.

  The file system program 207 is a program for managing file data and management information (metadata) stored in the disk device 203. The file sharing server program 206 transfers the file data sent from the client 101 to the file system program 207, and the file system program 207 stores and manages the data in the disk device 203.

  The backup determination program 208 is a program for determining the backup server 107 as a backup destination and the backup method. The processing operation of the backup determination program 208 will be described later in detail.

  The backup data transmission / reception program 209 is a program for transmitting backup data to the backup server 107 via the communication path 102. The protocol used for transmitting and receiving the backup data is, for example, a REST (Representational State Transfer) protocol or an SMB (Server Message Block) protocol.

  The copy data creation program 210 is a program for creating backup data of a file to be backed up. The duplicate data creation program 210 creates backup data in which metadata and data of a file to be backed up are separated. Further, the duplicate data creation program 210 performs redundant encoding of data according to EC. EC (Erasure Coding) is an encoding method for dividing data into fragment data of a certain size and creating parity data of the fragment data. Even if fragment data of the number of parity data is lost, the original data can be restored. For example, it is assumed that the copy data creation program 210 has created 10 copy data (simple copy metadata), 10 fragment data, and 2 parity data as backup data of a certain file. The duplicate data creation program 210 backs up (distributes) them to the plurality of backup servers 107. Thereby, disaster resistance of the backup / restore system 100 is improved. There are various schemes for EC such as Reed Solomon coding, and any scheme may be adopted as the EC scheme.

  The copy data restoration program 211 is a program for restoring the original file using the backup data created by the copy data creation program 210. If the backup data is made redundant by the EC, decoding processing is performed to obtain the original data by calculation.

  The management program 212 is a program for creating or managing management information used by a program such as the duplicate data creation program 210. For example, the management program 212 sets or manages the EC system, the number of simple copies, and the like. The management program 212 is executed by the administrator through the management server 104. The administrator connects to the file server 105 from the management server 104 using an arbitrary protocol (for example, SSH (Secure Shell) protocol) and operates the management program 212.

  The backup setting table 213 is a table that defines a backup method and the like generated by the management program 212. The backup setting table 213 will be described later in detail.

  FIG. 3 shows the configuration of the backup server 107.

  The backup server 107 has a network I / F 301, a CPU 302, a memory 303, and a disk device 310. They are interconnected by an internal communication path 307.

  The network I / F 301 is a device connected to the external communication path 102. The CPU 302 is a device that executes a program stored in the memory 303. The disk device 310 is a device that holds a program, a backup management table 308, fragment data 309 (309A, 309B,...), Parity data 311 and duplicate data 312. The fragment data 309 is a part of the data, and is partial data obtained by encoding according to the data EC. The parity data 311 is data used to generate one of the one or more pieces of fragment data 309. The copy data is a copy of data (for example, metadata). The file system program 304 is a program for managing backup data as a file.

  The backup data transmission / reception program 305 stores the backup data sent from the file server 105 (105A) as a file in the disk device 310, and also stores the backup data in response to a backup data acquisition request from the file server 105 (105B). This is a program to be transmitted to the server 105 (105B). As described above, a protocol such as REST may be used for transmitting and receiving the backup data.

  The management program 306 is a program for setting the backup server 107. For example, the management program 306 sets the IP address of the backup server 107 and the like.

  The backup management table 308 stored in the disk device 310 is a table for managing the backup server 107 as the storage destination of the backup data, the EC method, and the like. The contents and usage of the backup management table 308 will be described later in detail. The backup management table 308 may be stored in at least one backup server 107 or another shared storage area accessible by the file servers 105A and 105B.

  FIG. 4 shows the configuration of the backup setting table 213.

  The backup setting table 213 has a record for each of the plurality of backup servers 107 (backup site 106). Each record stores a site ID 401, a site name 402, and a URL 403. The backup setting table 213 stores an EC setting 410 and a replication setting 411.

  The site ID 401 is the ID (for example, serial number) of the backup site 106. The site name 402 is the name of the backup site 106. The URL 403 is an access path from the file server 105 to the backup server 107 (storage in the backup server 107). The file server 105 generates an access path based on the URL 403 for accessing the fragment data 309 and the parity data 311 and acquires or transmits the data. Instead of the URL 403, another type of access path may be employed. The EC setting 410 includes information on an encoding policy according to the EC, for example, information indicating the number of fragment data, the number of parity data, and the EC method. The replication setting 411 includes information indicating the number of replications (the number of replicated data) when replicating by the replication method. For example, the number of replicated data of metadata (the number of replications) is determined according to the replication setting 411.

  FIG. 5 shows the configuration of the backup management table 308.

  The backup management table 308 has a record for each of one or more backup target objects (restore target objects). Each record stores a path 501, an ID 502, a type 503, a Meta 504, a Data 505, a MetaBUP 506, a DataBUP 507, and a ParityBUP 508.

  A path 501 represents a path (bus name) to an object in the file system space. The path 501 is an example of position information of an object in the file system space. At the time of restoration, the file or directory is restored to the same path (position) as the path 501. The ID 502 is the ID of the object. The type 503 is information indicating the type (file or directory) of the object.

  Meta 504 is information indicating a backup method of metadata in the object. When Meta 504 is “Rep.”, Duplicate data of the metadata is backed up. When metadata is backed up in accordance with EC, “(p, q)” is set as described later. Data 505 is information indicating a backup method of the object data in the object. When the object data is backed up according to EC, “(p, q)” is set as Data 505. p is the number of fragment data, and q is the number of parity data.

  MetaBUP 506 indicates a site name of a metadata backup destination (backup site 106). DataBUP 507 indicates the site name of the backup destination (backup site 106) of the object data. When one or more parity data 311 is backed up for at least one of the metadata and the object data, the ParityBUP 508 indicates a site name of a backup destination (backup site 106) of the one or more parity data 311. The ParityBUP 508 may distinguish and store two types of site names such as a backup destination site name of the parity data 311 for metadata and a backup destination site name of the parity data 311 for object data.

  The backup management table 308 may be divided into a plurality of record groups (one or more records), and the plurality of record groups may be distributed to the plurality of backup servers 107. Further, at least one record of the backup management table 308 may be stored in the above-mentioned shared storage area.

  Hereinafter, processing performed in the present embodiment will be described.

  FIG. 6 is a schematic diagram of the restore processing.

  Using the fragment data 309, the duplicate data 312, and the parity data 311 stored in the two disk devices 310 of the two backup servers 107A and 107B (backup sites 106A and 106B), three files are stored in the file server 105 of the recovery site 108. Restoring objects (directory "HOME", file X and file Y) is taken as an example. When the number of data of two or more data (one or more fragment data and one or more parity data) of one object data is equal to or less than the number of backup servers, the two or more data are transferred to different two or more backup servers 107 respectively. Backed up. When the number of data is larger than the number of backup servers, one backup server 107 backs up a plurality of data (at least one of fragment data and parity data) for one object data.

  The file server 105B generates a temporary directory “TMP” in the second file system space (the file system space where the object is restored). The temporary directory “TMP” is generated, for example, as a child directory of the root directory “Root”.

  By referring to the backup management table 308, the file server 105B specifies, for each of the three objects, the position of the object in the second file system space and the name of the backup destination site of the object.

  After that, first, the file server 105B restores the metadata with priority over the object data. Specifically, the file server 105B obtains three pieces of duplicate data 312 (metadata copies) corresponding to the three objects from the backup servers 107A and 107B in parallel, and stores the three pieces of data in the temporary directory “TMP”. Temporarily recover each of the two replicated data. Temporary restoration of metadata means generating a temporary object which is an object having only the metadata and no object data. As a result, three temporary objects are generated in the temporary directory “TMP”. Since the directory generally does not have object data, the directory “HOME” is temporarily restored by the temporary restoration of the metadata.

  Next, the file server 105B acquires a plurality of fragment data and a plurality of parity data corresponding to the files X and Y from the backup servers 107A and 107B in parallel. The file server 105B calculates two file data respectively corresponding to the files X and Y by performing decoding using the obtained data. The file server 105B temporarily recovers each of the two file data. Temporary recovery of file data means storing the file data in a temporary object corresponding to the file data. As a result, each of the files X and Y is restored to the temporary directory “TMP”.

  Finally, the file server 105B moves each of the three objects from the temporary directory “TMP” to a corresponding position in the second file system space. Specifically, for example, the directory “HOME” is arranged as a child directory of the root directory “Root” (moved to a position according to the path “/ Root / HOME”). Then, the file X and the file Y are moved into the directory “HOME” (the file X is moved to a position according to the path “/ Root / HOME / FileX”, and the file Y is moved to the path “/ Root / HOME / FileY”). Will be moved to a position according to). After the three objects have been moved (that is, after the restoration process is completed), the temporary directory “TMP” may be deleted by the file server 105B.

  With the temporary directory “TMP”, the lower-level restoration target object can be temporarily restored first without waiting for the higher-level restoration target object of the plurality of restoration target objects to be restored. As a result, restore processing can be executed at high speed.

  FIG. 7 shows a flow of the backup process.

  Before the disaster, the file server 105A of the primary site 103 performs a backup process. In the backup process, the program of the file server 105A and the program of the backup server 107 operate in cooperation with each other. The file received by the file system program 304 from the file access program 110 of the client 101 is temporarily stored in the disk device 203. After a certain period of time, the file data of the file is converted into two or more pieces of fragment data 309 and one or more pieces of parity data 311 in accordance with EC, and those data and metadata (replicated data) are transferred to a plurality of backup servers 107. Are sent to two or more backup servers 107.

  First, in the backup process, the backup decision program 208 executes a backup method decision process (Step 701). Specifically, in the first embodiment, the backup determination program 208 determines the number of fragment data and the number of parity data for the object data of the backup target object according to the EC setting 410 of the backup setting table 213. Further, the backup determination program 208 determines the number of replicated data for the metadata of the backup target data according to the replication setting 411 of the backup setting table 213. If the backup target is a directory, there is no object data, so only the replication setting 411 is used for backing up the directory. Hereinafter, in the description of FIG. 7, the backup target object is a file.

  Next, the backup determination program 208 selects, from the backup setting table 213, the backup site 106 of the total of the number of fragment data and the number of parity data for the file data of the file to be backed up (Step 702). The backup destination (backup site 106) may be determined on a round robin basis based on the site ID number. In this step 702, the backup site 106 of the backup destination of the replicated data may be selected. The backup site 106 may be a backup site 106 (backup server 107) different from the backup destination of the fragment data and the parity data.

  Next, the copy data creation program 210 encodes the file data of the backup target file according to the EC (Step 703). At this time, the duplicate data creation program 210 receives the number of fragment data and the number of parity data from the backup determination program 208, and generates fragment data 309 for the number of fragment data and parity data 311 for the number of parity data. If the object to be backed up is a directory, step 703 is skipped.

  The backup data transmission / reception program 209 backs up (transmits) the generated fragment data 309 and parity data 311 to the backup server 107 selected in Step 702 (Step 704). Specifically, the backup data transmission / reception program 209 receives from the backup determination program 208 a list of the backup server 107 as a backup destination of the fragment data 309 and the parity data 311 and transmits the data according to the list. If the object to be backed up is a directory, step 704 is skipped.

  Next, the backup data transmission / reception program 209 transmits the metadata of the backup target file to the backup server 107 (Step 705). At this time, the backup data transmission / reception program 209 receives from the backup determination program 208 a list of the backup servers 107 to which the metadata is to be transmitted, and transmits the metadata according to the list.

Next, the backup data transmission / reception program 209 adds a record to the backup management table 308 (Step 706). The backup data transmission / reception program 209 registers the following in the added record (step 707).
A path 501 representing the path of the file to be backed up.
An ID (serial number) 502 of the file to be backed up;
-Type 503 "FILE".
Meta 504 “Rep.” (Because duplicate data of metadata has been backed up).
Data 505 “(p, q)” (p = number of fragment data, q = number of parity data).
MetaBUP 506 (the site name of the backup site 106 to which duplicate data (metadata duplication) is transmitted).
DataBUP 507 (site name of backup site 106 to which fragment data is transmitted).
ParityBUP 508 (site name of backup site 106 to which parity data is transmitted).

  FIG. 8 shows a flow of the restore processing.

  The restoration process is executed by the file server 105B of the recovery site 108 after a disaster occurs.

  The backup data transmission / reception program 209 of the file server 105B connects to the backup server 107 described in the backup setting table 213 and acquires the backup management table 308 (Step 801). When the backup management tables 308 are not stored in all the backup servers 107, the backup servers 107 may be accessed sequentially until the backup management tables 308 can be obtained from all the backup servers 107. Further, the URL of the backup server 107 in which the backup management table 308 is stored may be described in the backup setting table 213, and the backup management table 308 may be obtained (referenced) using the URL. .

  Next, the copy data restoration program 211 selects one record 520 of the backup management table 308 (Step 802), and specifies a backup method from the Meta 504 and Data 505 of the record (Step 803). As a result, the copy data restoration program 211 knows the restoration method of each restoration target object. Then, the copy data restoration program 211 specifies the backup server 107 as a restoration source (Step 804). Here, for example, the network connection may be confirmed by a ping command, and the backup server 107 may be determined based on the result. At this time, the backup server 107 is specified based on DataBUP 507 in order to obtain the object data. When the backup server 107 described in the DataBUP 507 is stopped, the duplicate data restoration program 211 selects one alternative site (the backup server 107) from the ParityBUP 508.

  Next, the copy data restoration program 211 restores the metadata from the selected backup server 107 through the backup data transmission / reception program 209 (Step 805). At this time, as described above, the duplicate data restoration program 211 generates a temporary file in a temporary directory to temporarily recover the metadata (Step 806).

  Then, the copy data restoration program 211 receives fragment data or parity data from the selected backup server 107 through the backup data transmission / reception program 209 (Step 807). Then, the copy data restoration program 211 performs decoding according to the EC using the received data, and temporarily restores the file to the temporary directory (step 808). The replicated data restoration program 211 moves the temporarily restored file or directory (that is, the temporarily restored file or directory) to a position following the original path based on the path 501 of the backup management table 308 (step 809). ). Thereby, restoration of one file or directory is completed.

  Next, the copy data restoration program 211 checks whether there is an unrestored file or directory (step 810). If there is an unrestored file or directory (step 810: N), the processing is restarted from step 802. On the other hand, if not (step 810: Y), the process ends.

  Here, in the parallel processing of the restoration described with reference to FIG. 6, the section where the parallel processing is performed is from step 803 to step 809. This section can be processed in parallel because files or directories can be processed individually. In the case of parallelization, steps belonging to a corresponding section are executed by a plurality of threads or processes.

  As described above, according to the first embodiment, the backup method differs between metadata and object data. Specifically, the metadata performs a replication backup, and the object data performs a distributed backup according to EC. To restore object data, metadata restoration must be completed. Therefore, according to the present embodiment, it is possible to prepare the precondition for restoring the metadata as soon as the backup site storing the metadata is restored. Further, thereafter, as the plurality of backup sites are sequentially recovered, the object data, which takes longer time than the metadata, is executed, and a shorter time is efficiently used than in the case of using any one method. Recovery can be made possible.

  Hereinafter, a second embodiment will be described. At this time, differences from the first embodiment will be mainly described, and description of common points with the first embodiment will be omitted or simplified.

  In the second embodiment, depending on the attributes of the backup target object (specifically, the size of the data in the object, the path depth of the object (described later)), the backup method and the backup redundancy (the number of duplicate data, the number of fragment data, and the parity data) At least one of the numbers). As a result, backup and restoration optimal for the attributes of the object can be expected.

  For example, even if the object to be backed up is file data, if the data size of the file data is smaller than the threshold, replication backup is adopted instead of EC. Even if small data is divided and backed up according to EC, the parallelization effect at the time of restoration is low. Therefore, for the file data whose data size is less than the threshold, decoding according to EC (step 808) can be omitted by employing replication backup. This can be expected to further speed up the restore processing.

  FIG. 9 illustrates a configuration of a backup mode table according to the second embodiment.

  The backup method table 900 is stored in a storage unit (at least one of a memory and a disk device) of the file server 105. The backup method table 900 has a record in which a parameter 901 and a value 902 are set as one set. In this embodiment, the parameters 901 include “metadata”, “object data (large size)”, “object data (small size)”, and “size threshold”. Here, an appropriate value is set as an example by the administrator. Therefore, the present invention is not limited to this example.

  The value 902 of the parameter 901 “metadata”, the value 902 of the parameter 901 “object data (large size)”, and the value 902 of the parameter 901 “object data (small size)” are all (x, y, z). That is, the relationship between the path depth (x), the backup method (y), and the redundancy (z) is shown.

  The "path depth" of an object is the depth of the object in the file system space, in other words, the number of links that go from the root directory to the object. For example, when the path is “/ DIR1 / DIR2 / FileX”, three links (a link from the root directory to the directory “DIR1” and a link from the directory “DIR1” to the directory “DIR2”) from the root directory to the file X are provided. The path depth is 3 through the link and the directory “DIR2” to “FileX”. In the present embodiment, a range of 1 or more and 4 or less and a range of 5 or more are provided as ranges of the path depth. Accordingly, in FIG. 9, the path depth “1” means that the path depth is 1 or more and 4 or less, and the path depth “5” means that the path depth is 5 or more. The range of the path depth is not limited to the present embodiment.

  The “backup method” is either a copy or EC. The backup method “Rep” means duplication, and the backup method “EC” means EC.

  The meaning of “redundancy” differs depending on whether the backup method is “Rep” or “EC”. When the backup method is “Rep”, “redundancy” is the number of duplicate data. When the backup method is “EC”, “redundancy” is EC redundancy. Here, the EC redundancy V is defined as (p + q) / p. p is the number of fragment data generated as described above, and q is the number of parity data generated as described above. Specifically, for example, when one piece of parity data is generated for two pieces of fragment data, the EC redundancy V is (2 + 1) /2=1.5. The reason why the value of the EC redundancy V is changed depending on the path depth is that an object having a small path depth (that is, a shallow object) is considered to have a high importance. For example, below the directory "DIR1", other directories and other files may be stored in addition to the directories "DIR2" and "FileX" described above. That is, the directory “DIR1” depends on these objects. Unless restoration of the directory "DIR1" is completed, restoration of these objects cannot be completed. Therefore, it is preferable that a relatively shallow object has a higher EC redundancy value V than a relatively deep object.

  The parameter 901 “size threshold” is a threshold of the size of the object data. According to FIG. 9, the value 902 of the “size threshold” is “1 MB”. “Object data (large size)” is object data having a size equal to or larger than the size threshold, and “object data (small size)” is object data having a size smaller than the size threshold. Therefore, in this embodiment, “object data (large size)” is object data having a size of 1 MB or more, and “object data (small size)” is object data having a size of less than 1 MB. The “size threshold” may be one or more. Further, a size threshold for metadata may be used as the size threshold.

  FIG. 10 illustrates a flow of a backup method determination process according to the second embodiment. In FIG. 10, values according to the table of FIG. 9 are described.

  The backup mode determination process is a process executed by the backup determination program 208. In the first embodiment, in the backup method determination processing, a replication backup is adopted for metadata, and an EC division backup (a backup method for dividing and backing up data according to EC) is adopted for object data.

  In the second embodiment, instead of or in addition to the viewpoint of the first embodiment, in the backup method determination processing, the following is performed based on the backup method table 900.

  The backup determination program 208 determines whether the processing target data (target data) in the backup target object is metadata or object data (step 1000).

  If the target data is metadata, the backup determination program 208 determines the redundancy based on the path depth of the object including the metadata (“Object A” in this paragraph) (Step 1001). According to the table 900 of FIG. 9, when the path depth of the object A is 1 or more and 4 or less, the redundancy is set to 20 (20 pieces of duplicate data (metadata copy) are generated) (step 1002). On the other hand, if the path depth of the object A is 5 or more, the redundancy is set to 10 (10 pieces of duplicate data are generated) (step 1008).

  On the other hand, if the target data is object data, the backup determination program 208 determines whether the size of the object data is equal to or larger than the size threshold (1 MB) (step 1003).

  If the result of the determination in step 1003 is negative, the backup decision program 208 decides to back up the five replicated data (step 1004). The five pieces of copy data are an example of one or more pieces of copy data. The reason that the EC division backup is not adopted in this case is that if the data size is smaller than the threshold value, the effect of the EC division is low.

  If the result of the determination in step 1003 is positive, the backup determination program 208 determines the EC redundancy V based on the path depth of the object containing the target data (“object B” in this paragraph) (step 1005). According to the table 900 of FIG. 9, when the path depth of the object B is 1 or more and 4 or less, the EC redundancy V is set to 10 (step 1006). On the other hand, if the path depth of the object B is 5 or more, the EC redundancy V is set to 5 (step 1007).

  As described above, according to the second embodiment, it is possible to determine the backup method and the redundancy (at least one of the number of replicated data, the number of fragmented data, and the number of parity data) that are optimal for the attribute of the backup target object.

  Note that the second embodiment can be combined with the first embodiment as described above. For example, when the target data is object data, EC division backup is adopted, but the value of EC redundancy V is determined according to the path depth of the object including the target data.

  In the second embodiment, one of steps 1003 and 1005 may not be included in the backup method determination processing shown in FIG.

  Hereinafter, a third embodiment will be described. At this time, differences from the first and second embodiments will be mainly described, and descriptions of common points with the first and second embodiments will be omitted or simplified.

  In the third embodiment, the backup method and the backup redundancy (at least one of the number of replicated data, the number of fragmented data, and the number of parity data) are based on the recovery rate of the backup site after a disaster (for example, a large-scale disaster) occurs. Is determined. Thereby, it is possible to expect an optimal backup and restore according to the region where the backup / restore system according to the third embodiment is operated.

  FIG. 11 illustrates a configuration of a backup mode table according to the third embodiment.

  The backup method table 1100 is a table prepared instead of or in addition to the table 900 shown in FIG. 9, and is stored in the storage unit (at least one of the memory and the disk device) of the file server 105.

  The backup method table 1100 indicates the relationship between the elapsed time since the occurrence of the disaster and the recovery rate. Specifically, for example, the one-day recovery rate 1101 in the backup method table 1100 indicates the recovery rate on the first day after the occurrence of the disaster. The three-day recovery rate 1102 indicates the recovery rate on the third day after the occurrence of the disaster. The recovery rate is a ratio of the number of recovered backup servers (operable backup servers) 107 to the number of backup servers. For example, the recovery rate “30%” when 100 backup servers 107 are installed indicates that 30 backup servers 107 can operate. Generally, the recovery rate increases with the passage of time from the occurrence of a disaster.

  FIG. 12 illustrates a flow of a backup method determination process according to the third embodiment.

  First, the backup determination program 208 determines whether or not the value of the one-day recovery rate 1101 is equal to or less than the one-day recovery rate threshold (for example, 20%) (step 1201).

  If the result of the determination in step 1201 is affirmative, it is defined that a backup site exists in an area where recovery is relatively slow. For this reason, for each backup target object, replication backup is adopted for each of the metadata and the object data. Specifically, for example, the backup determination program 208 determines to generate 10 pieces of duplicate data for each of the metadata and the object data (Step 1205). The ten pieces of copy data are an example of one or more pieces of copy data. The greater the number of replicated data, the more the backup server 107 recovers even a little, thereby increasing the probability of restoration.

  If the determination result in step 1201 is negative, the backup determination program 208 determines whether the value of the three-day recovery rate 1102 is equal to or less than a three-day recovery rate threshold (for example, 50%) (step 1202).

  If the determination result in step 1202 is affirmative, the recovery rate is slightly poor, so the backup determination program 208 determines a replication backup (generates 10 pieces of replicated data) for the metadata and the EC redundancy V for the object data. = 3 (p (number of fragment data) = 2, q (number of parity data) = 4) is determined (step 1204).

  If the determination result in step 1202 is negative, it means that a backup site is defined in an area where recovery is relatively quick. The backup determination program 208 determines a replication backup (generates five pieces of replicated data) for metadata, and determines an EC split backup of EC redundancy V = 2 (p = 4, q = 4) for object data. (Step 1203).

  The backup method and the backup redundancy determined in steps 1203 to 1205 are registered in the backup management table 308 by the backup determination program 208. Restore processing is performed based on the backup method and the backup redundancy.

  As described above, according to the third embodiment, the backup capacity is suppressed by changing the backup method and the backup redundancy assuming the recovery status of the entire backup site (the relationship between the elapsed time after the occurrence of the disaster and the recovery rate). Efficient backup and restore, such as improving restore performance, can be expected.

  Note that the third embodiment can be combined with at least one of the first and second embodiments as described above. For example, when the value of the one-day recovery rate 1101 is equal to or less than 20% and the size of the target data is less than 1 MB, a replication backup may be employed. Further, for example, even if the value of the one-day recovery rate 1101 is equal to or less than 20%, if the size of the target data is equal to or greater than 1 MB, EC division backup may be employed.

  Although some embodiments have been described above, these are examples for describing the present invention, and are not intended to limit the scope of the present invention only to these embodiments. The present invention can be implemented in various other forms.

101: Client, 103: Primary Site, 105: File Server, 106: Backup Site, 107: Backup Server, 108: Recovery Site

Claims (9)

Multiple backup servers,
A first object server for backing up one or more objects, each of which is a file or directory, to at least one of the plurality of backup servers;
A second object server that restores the backed up one or more objects from at least one of the plurality of backup servers;
The file in the one or more objects includes file data that is object data and metadata about the object data, and the directory in the one or more objects includes metadata about the object. Yes,
The processor of the first object server, for each of the object data and metadata in the one or more objects,
(B1) a function of determining which of the first condition and the second condition the target data, which is the data and is object data or metadata,
(B2) as a result of (B1), when the target data satisfies the first condition, a function of backing up duplicate data of the target data to two or more backup servers;
(B3) As a result of (B1), when the target data satisfies the second condition, fragment data of redundancy V according to EC (Erasure Coding) of the target data (V is a value based on (p + q) / p , P is the number of fragment data, and q is the number of parity data), and has a function of backing up two or more data to two or more backup servers.
The processor of the first object server, for each of the one or more objects, information on a position of the object in a first file system space in the first object server and a backup destination of the object, the plurality of backup servers Has a function of registering in the management information stored in the storage device in at least one of
The processor of the second object server,
(R1) for each of the one or more objects, a function of specifying the position of the object in the second file system space of the second object server and the backup destination of the object by referring to the management information; ,
(R2) As a function for each of the one or more objects,
(R21) a function of receiving, from at least one of the plurality of backup servers, duplicate data of the metadata or fragment data of the metadata as metadata of the target object which is the object;
(R22) When the target object is a file, receiving, as object data of the target object, a copy of the object data or a fragment of the object data from at least one of the plurality of backup servers. Function and
(R23) the second metadata of the target object based on the received metadata when the target object is a directory, or the received metadata and the received object data when the target object is a file. Generating and arranging the target object at a position in a file system space,
The first condition is that the target data is metadata, and the second condition is that the target data is object data, or
The first condition is that the target data is metadata, the target data is object data, and the data size of the object data is smaller than a threshold, and the second condition is: The target data is object data, and the data size of the object data is equal to or larger than the threshold.
Backup and restore system. The processor of the second object server is configured to execute a process according to (R21) prior to a process according to (R22) for at least one of the one or more objects.
The backup / restore system according to claim 1. The processor of the second object server,
Before starting the processing by (R2), a temporary directory is provided in the second file system space,
The processing according to (R21) and the processing according to (R22) are executed in parallel for the one or more objects,
By the processing in (R23), a directory in the one or more objects is generated in the temporary directory based on the metadata acquired in the processing in (R21), and a file in the one or more objects is generated according to (R21). Based on the metadata obtained in the process and the object data obtained in the process of (R22), the temporary directory is generated in the temporary directory, and the directory and the file are transferred from the temporary directory to a corresponding position in the second file system space. 3. The backup / restore system according to claim 1, wherein the backup / restore system is configured to move. When the target data satisfies the second condition, the value of the redundancy V is a value according to a path depth of an object including the target data;
The path depth of the object including the target data is the number of links that pass from a root directory of the first file system space to the object.
The backup / restore system according to any one of claims 1 to 3. A replication backup in which each of the object data and metadata in one or more objects, each of which is a file or a directory, is to backup duplicate data to two or more backup servers of a plurality of backup servers; and EC (Erasure Coding). )), Two or more pieces of data that are fragment data (V is a value based on (p + q) / p, p is the number of fragment data, and q is the number of parity data) are backed up by two or more. backup accordance selected backup method of the distributed backup is to backup the server, a backup restoration method for generating a plurality of backup servers that are backed up to the one or more objects to restore the object to the server of the
The file in the one or more objects includes file data that is object data and metadata about the object data, and the directory in the one or more objects includes metadata about the object. Yes,
The object server is:
(R1) For each of the one or more objects, specify a position of the object in the file system space of the object server and a backup destination of the object;
(R2) for each of the one or more objects:
(R21) receiving, from at least one of the plurality of backup servers, duplicate data of the metadata or fragment data of the metadata as metadata of the target object which is the object;
(R22) receiving, as object data of the target object, duplicate data of the object data or fragment data of the object data from at least one of the plurality of backup servers;
(R23) the file system of the target object based on the received metadata when the target object is a directory or the received metadata and the received object data when the target object is a file; Generating and placing the target object at a position in space ,
If the backup source object server is
For each of the object data and metadata in the one or more objects,
(B1) determining whether the target data, which is the data and is object data or metadata, satisfies the first condition or the second condition;
(B2) As a result of (B1), if the target data satisfies the first condition, backup the duplicate data of the target data to two or more backup servers;
(B3) As a result of (B1), when the target data satisfies the second condition, two or more data that are fragment data of the redundancy V according to the EC of the target data are backed up to two or more backup servers,
The first condition is that the target data is metadata, and the second condition is that the target data is object data, or
The first condition is that the target data is metadata, the target data is object data, and the data size of the object data is smaller than a threshold, and the second condition is: A backup / restoration method, wherein the target data is object data, and the data size of the object data is equal to or larger than the threshold. An object server for backing up one or more backup target objects, each of which is a file or a directory, to at least one of a plurality of backup servers,
An interface connected to the plurality of backup servers;
A processor connected to the interface,
The file in the one or more objects to be backed up comprises file data as object data and metadata about the object data, and the directory in the one or more objects comprises metadata about the object. Has been
The processor, for each of the object data and metadata in one or more objects to be backed up, each being a file or a directory,
(B1) a function of determining which of the first condition and the second condition the target data, which is the data and is object data or metadata,
(B2) as a result of (B1), when the target data satisfies the first condition, a function of backing up duplicate data of the target data to two or more backup servers;
(B3) As a result of (B1), when the target data satisfies the second condition, fragment data of redundancy V according to EC (Erasure Coding) of the target data (V is a value based on (p + q) / p , P is the number of fragment data, and q is the number of parity data), and has a function of backing up two or more data to two or more backup servers.
The processor further includes, for each of the one or more backup objects, information on a position of the backup object in the first file system space and a backup destination of the backup object, among the plurality of backup servers. Registering in the management information stored in at least one storage device,
The first file system space is a file system space including the one or more backup target objects,
The processor comprises:
(R1) For each of one or more objects to be restored each of which is a file or a directory, the location of the object to be restored in the second file system space and the backup destination of the object are referred to by referring to the management information. Has a function to specify,
The second file system space is a file system space including the one or more objects to be restored,
(R2) As a function for each of the one or more objects to be restored,
(R21) a function of receiving, as metadata of the target object being the restore target object, duplicate data of the metadata or fragment data of the metadata from at least one of the plurality of backup servers;
(R22) When the target object is a file, receiving, as object data of the target object, a copy of the object data or a fragment of the object data from at least one of the plurality of backup servers. Function and
(R23) the second metadata of the target object based on the received metadata when the target object is a directory, or the received metadata and the received object data when the target object is a file. A function of generating and arranging the target object at a position in a file system space,
The first condition is that the target data is metadata, and the second condition is that the target data is object data, or
The first condition is that the target data is metadata, the target data is object data, and the data size of the object data is smaller than a threshold, and the second condition is: An object server, wherein the target data is object data, and a data size of the object data is equal to or larger than the threshold. The processor is configured to execute a process of (R21) prior to a process of (R22) for at least one of the one or more restore target objects.
The object server according to claim 6 . The processor comprises:
Before starting the processing by (R2), a temporary directory is provided in the second file system space,
The processing according to (R21) and the processing according to (R22) are executed in parallel with respect to the one or more objects to be restored,
By the processing in (R23), a directory in the one or more objects to be restored is generated in the temporary directory based on the metadata acquired in the processing in (R21), and a file in the one or more objects is written in (R21). ) Is generated in the temporary directory based on the metadata acquired in the process of (R22) and the object data acquired in the process of (R22), and the target object is stored from the temporary directory to a corresponding position in the second file system space. claim 6 or 7 object server according is configured to move to, so. When the target data satisfies the second condition, the value of the redundancy V is a value according to the path depth of the backup target object including the target data;
The path depth of the backup target object including the target data is the number of links from the root directory of the first file system space to the backup target object.
An object server according to any one of claims 6 to 8 .

Images