JP6055197B2 - Database system - Google Patents

Description

  The present invention relates to data backup technology handled by a distributed processing system including a plurality of servers.

  In recent years, with the rise of cloud computing, it has been required to efficiently process and retain a large amount of data. Thus, distributed processing technology has been developed that realizes efficient processing by operating a plurality of servers in a coordinated manner.

  When performing distributed processing, it is necessary to distribute processing target (management target) data to each server constituting the cluster (hereinafter also referred to as “cluster member” or “member”). At this time, in order to increase the processing capacity of the entire cluster, it is desirable that the number of data (data amount) handled by each cluster member is averaged.

  As a typical data distribution method, the remainder obtained by dividing the value obtained by multiplying the key of each data by a hash function (hereinafter referred to as “hash (key)”) by the number N of cluster members, that is, “hash (key) mod N There is a method of distributing data to cluster members having "" as a number. In this case, it is assumed that numbers “0” to “N−1” are assigned to each cluster member in advance. When such a distribution method is used, if a cluster member is added, the value of N changes, and the cluster member in charge of a lot of data is changed, so that the data in charge must be rearranged.

  Therefore, as a method for suppressing the number of data that the cluster member in charge changes with the addition of the cluster member to about 1 / N, there is a distribution method using a consistent hashing method (see Non-Patent Document 1). is there.

  In this data distribution method using the consistent hash method, an ID (IDentifier) is assigned to both the cluster member and the data, and when the ID space is traced clockwise from the data ID, the first cluster member encountered is Take charge of data.

  In addition, when managing a large amount of data in a cluster-structured distributed processing system, data redundancy is maintained by maintaining a copy of the data so that even if a failure occurs in one cluster member, processing can be continued on other cluster members. Needs to be realized. The same applies to a distributed processing system that uses a data management technique based on the consistent hash method.

  As shown in FIG. 4, in the consistent hash method, IDs are assigned to both cluster members (members 1 to 4) and data (data A to D, indicated by black circles), and the ID space is clockwise from the data ID. As a result, the first cluster member encountered is determined to be responsible for the data. Then, the cluster member that is further to the right of the cluster member in charge (next clockwise) is assigned the duplicate data.

  For example, in FIG. 4, data A is traced clockwise in the ID space and is assigned to the member 1 that first encounters, and the duplicate data is assigned to the member 2 on the right side of the member 1 in the ID space. By determining the cluster member in charge of the original data / replicated data in this way, even if the cluster member leaves, the cluster member that owns the replicated data becomes the new cluster member in charge of the data. There is an advantage that it can respond. When a plurality of pieces of duplicate data are taken, the second duplicate data can be assigned to the cluster member on the right side.

  As a database server that can be used in such a method, for example, there is a distributed database server described in Cassandra (Non-Patent Document 2) or the like, in which a plurality of servers can dynamically join and leave a cluster. In the technique of Non-Patent Document 2, uniform distribution of data using the consistent hash method and high-speed access of O (1) are realized. In addition, fault tolerance is enhanced by providing duplicate information to a plurality of servers.

  In addition, even when a catastrophic disaster or the like occurs and a plurality of servers are down (failed) at the same time, it is necessary to reduce the possibility that both the original data and the duplicated data will be lost. For example, if multiple servers are placed only in the data center area with the Kanto area (the area under the control of the data center that manages the server), if a catastrophic disaster occurs in the entire Kanto area, all servers will go down, Data may be lost. Therefore, it is first necessary to distribute a plurality of servers in a wide area.

  Here, as shown in FIG. 5, consider a case where a plurality of servers belong to one of K types (here, five types) of data center areas across the country. In the ID space of FIG. 5, it is assumed that servers having the same symbol (black circle, white circle, black triangle, white triangle, white square) belong to the same data center area.

  In this case, even if a catastrophic disaster or the like occurs in the entire Kanto region, servers belonging to the Kanto data center area go down, but servers belonging to other data center areas do not go down. In other words, in the consistent hash method ID space, servers in the Kanto data center area leave, but servers in other data center areas do not leave.

  Normally, in the ID space of the consistent hash method, the servers are distributed (arranged) without considering the physical positions of the servers. Therefore, in FIG. 5, when servers belonging to the same data center area are adjacent to each other, such as two servers surrounded by a broken line in the upper part of the ID space, when a catastrophic disaster or the like occurs in the entire data center area, There is a possibility that both original data and duplicated data will be lost at the same time. Therefore, for example, by preventing the servers in the same region from being adjacent to each other in the ID space of the consistent hash method, it is possible to reduce the possibility that both the original data and the duplicated data are lost simultaneously.

David Karger et al., “Consistent Hashing and Random Trees: Distributed Caching Protocols for Relieving Hot Spots on the World Wide Web”, [online], 1997, ACM, [searched February 20, 2012], Internet <URL: http://www.akamai.com/dl/technical_publications/ConsistenHashingandRandomTreesDistributedCachingprotocolsforrelievingHotSpotsontheworldwideweb.pdf> Avinash Lakshman et al., “Cassandra-A Decentralized Structured Storage System”, [online], [searched February 20, 2012], Internet <URL: http://www.cs.cornell.edu/projects/ladis2009 /papers/lakshman-ladis2009.pdf>

  However, for example, if a software failure occurs in a cluster and multiple servers in the cluster go down at the same time, both the original data and the replicated data are lost at the same time, and the data cannot be recovered easily. There is a possibility that.

  Therefore, the present invention has been made in view of the circumstances described above, and even in a distributed processing system, even when a plurality of servers in a cluster are down at the same time, both original data and duplicated data are lost at the same time. The task is to recover data easily.

In order to solve the above problems, the present invention includes a plurality of servers that process requests from client machines, and a management device that distributes requests from the client machines to any of the plurality of servers. the ID space, all of the plurality of data managed by the plurality of servers to handle, and said plurality of servers, are allocated, each of the server, itself from the next in a predetermined direction around in the ID space A distributed processing system for managing the data located between the server and storing a copy of the data located between the next server and the next server around the predetermined direction; A backup system comprising a storage device for storing the management target data handled by the distributed processing system as a backup. When the data to be managed is updated in the distributed processing system, either the management device or the plurality of servers sends the updated data to the backup system. The storage device in the backup system stores the received data, and when the plurality of servers in the distributed processing system fail and the management target data is lost, the management device using the data stored in the storage device of the backup system, the data have a row data recovery of the server which has lost, the backup system has the memory device into each of the two or more regions different, each of said The storage device stores the backup data, and a plurality of the data in the distributed processing system When the management target data is lost due to a server failure, the management device uses the data stored in any one of the storage devices in the backup system, and the data of the server that has lost the data. It is characterized by performing recovery .

According to this, in order to back up data with the storage device of the backup system provided outside the distributed processing system, a plurality of servers in the distributed processing system (cluster) go down at the same time, and both the original data and the replicated data are Even if they are lost at the same time, the data stored in the storage device of the backup system can be used to easily recover the data of the server that lost the data.
In addition, according to this, storage devices arranged in two or more different regions in the backup system store backup data, so that a plurality of servers can be simultaneously used in a distributed processing system (cluster) due to a software failure or the like. Even if all servers in one region go down due to a catastrophic disaster, etc., some of the storage devices in the backup system will not go down, and the management device will store the data stored in the storage device that is not down Can be used to easily recover data from a server that has lost data.

  According to the present invention, in a distributed processing system, even when a plurality of servers in a cluster are down at the same time and both original data and duplicated data are lost at the same time, data can be easily recovered.

It is explanatory drawing of the outline | summary of this embodiment. It is a block diagram of the database system etc. of this embodiment. It is a flowchart which shows the flow of the process by the management apparatus of this embodiment. It is explanatory drawing of the conventional consistent hash method. It is explanatory drawing of ID space and the physical position of a cluster member in the conventional consistent hash method.

  Hereinafter, modes for carrying out the present invention (hereinafter referred to as embodiments) will be described with reference to the drawings (refer to drawings other than the referenced drawings as appropriate). In order to facilitate understanding, the outline of this embodiment will be described first with reference to FIG. 1, and then this embodiment will be described.

(Outline of this embodiment)
Consider a case where there are K types (here, 5 types) of data center areas (hereinafter also referred to as “regions”) as shown in FIG. First, as a countermeasure against a catastrophic disaster, servers are arranged so that servers in the same region are not adjacent to each other in the ID space of the consistent hash method.

  In addition, when adding a server, after determining the insertion destination of a new cluster member in the ID space, investigate the area of the server on both sides of the insertion destination, and select a server in an area to which neither of the servers on both sides belongs. You can add more servers (when there are 3 or more regions). Then, it is possible to maintain a state where servers in the same region are not adjacent to each other in the ID space of the consistent hash method.

  As a result, servers in the same area in the ID space will not be next to each other. Therefore, even if all servers in one of the three or more areas go down due to a catastrophic disaster, the original data is copied. A situation where both data are lost can be avoided.

  Here, for example, if a software failure occurs in a cluster and multiple servers in the cluster go down at the same time, and both the original data and the replicated data are lost at the same time, backup can be performed easily. Install the equipment. Specifically, as an external device of the distributed processing system, a first backup device 5 (storage device) and a second backup device 6 (storage device) are respectively provided at two geographically distant locations in the K types of data center areas. ). The first backup device 5 and the second backup device 6 each store a backup of data handled by a plurality of servers in the cluster.

  As a result, in addition to a software failure in the cluster, even if a catastrophic disaster or the like occurs in one region and all servers in that region go down, at least one of the first backup device 5 and the second backup device 6 Since data does not go down, lost data can be easily recovered by using the data stored on the side that is not down.

  Specifically, the first backup device 5 includes a plurality of backup servers (storage devices). The second backup device 6 is composed of a plurality of storages (large capacity storage devices). The plurality of storages can be realized as a cluster configuration such as a Key Value cluster.

  For example, the management device 4 performs data backup for the first backup device 5 at the same time when updating data for a database cluster (cluster consisting of a plurality of servers). Here, the backup server of the first backup device 5 may perform data backup using, for example, a randomly accessible medium. While performing data backup using the first backup device 5 as described above, for example, the first backup device 5 to the second backup device 6 asynchronously and periodically (for example, once a day) backup (snap) Shot). The storage of the second backup device 6 may perform data backup using a tape drive, for example.

  Then, after the software failure of the database cluster occurs, the management device 4 may recover lost data using the first backup device 5 or the second backup device 6. For example, the data is restored by overwriting the data from the first backup device 5 or the second backup device 6 after restarting the database cluster again. Which of the first backup device 5 and the second backup device 6 is used for data recovery may be appropriately determined depending on the scale and type of data loss, whether or not the lost data can be specified, and the like. In addition, when a catastrophic disaster or the like occurs at the same time and one of the first backup device 5 and the second backup device 6 is down, the one that is not down may be used for data recovery.

(Embodiment)
Next, this embodiment will be described. As shown in FIG. 2, the database system S of the present embodiment includes a distributed processing system 1000 and a backup system 10.

  The distributed processing system 1000 includes a management device 4 and a plurality of servers 3 that constitute the cluster 100. The management device 4 is connected to a plurality of client machines 1 via a network 2 such as the Internet.

  An overview of the overall operation (when no failure occurs) will be described. The management device 4 receives a request (data processing request) from the client machine 1 via the network 2. Based on the server allocation table (ID space management information 411) on the data ID space, the management device 4 distributes the request to one of the plurality of servers 3 that perform data processing. The distributed server 3 processes the request.

  Next, the configuration of the management device 4 will be described. The management device 4 is a computer device that determines to which of a plurality of servers 3 (cluster members) a request from the client machine 1 is distributed based on a consistent hash method. As described above, in the consistent hash method, a plurality of data to be managed and a plurality of servers 3 that manage the data and constitute the cluster 100 are allocated to the circular ID space, The server 3 manages (responsible for) data positioned between itself and the next server 3 in the clockwise direction (predetermined direction) in the ID space, and from the next server 3 to the next server 3 in the clockwise direction. It is assumed that a copy of the data located in between is stored.

The management device 4 includes a storage unit 41, a processing unit 42, an input unit 43, a display unit 44, and a communication unit 45.
The storage unit 41 is a means for storing information, and includes a memory such as a random access memory (RAM) or a read only memory (ROM), a hard disk drive (HDD), or the like. The storage unit 41 stores ID space management information 411, area name information 412, server management information 413, first backup device management information 414, and second backup device management information 415. The storage unit 41 also stores an operation program for the processing unit 42 (not shown).

  The ID space management information 411 is information for managing the server 3 in charge of the data using the ID calculated by performing predetermined hash value conversion on the data to be managed.

  The area name information 412 is information for managing the association between the area ID and the area name corresponding to the area ID.

  The server management information 413 is information for managing the association with all the servers 3 physically existing in the area for each area (area ID).

The first backup device management information 414 is information such as the ID and address of the backup server that constitutes the first backup device 5.
The second backup device management information 415 is information such as the ID and address of the storage that constitutes the second backup device 6.

  The processing unit 42 is a unit that performs arithmetic processing based on information stored in the storage unit 41, and is configured by a CPU, for example. When updating (writing) data to the cluster 100, the processing unit 42 transmits the data ID and value (data content) to the cluster 100. Further, when transmitting the data to the first backup device 5 as a backup, the processing unit 42 similarly transmits the data ID and the value (data content) value. An address for accessing each backup server of the first backup device 5 is stored in the first backup device management information 414. The backup data may be transmitted to the first backup device 5 not by the management device 4 but by the server 3 in the cluster 100.

The input unit 43 is a means for the user of the management device 4 to input information, and is realized by, for example, a keyboard or a mouse.
The display unit 44 is a means for displaying information, and is realized by an LCD (Liquid Crystal Display), for example.
The communication unit 45 is a communication interface used for communication with an external device.

The backup system 10 includes a first backup device 5 and a second backup device 6.
The first backup device 5 includes a plurality of backup servers that store management target data handled by the distributed processing system 1000 as backups.
The second backup device 6 is composed of a plurality of storages that store management target data handled by the distributed processing system 1000 as backups.

Next, processing by the management device 4 will be described.
As illustrated in FIG. 3, in step S <b> 1, the processing unit 42 of the management device 4 executes distributed processing (normal (when no failure occurs) request processing) by the cluster 100.

  Next, in step S2, the processing unit 42 determines whether or not the backup timing by the first backup device 5 has come. If Yes, the process proceeds to step S3, and if No, the process proceeds to step S4. The backup timing may be, for example, every time data update occurs in the request processing, or every predetermined time (for example, 10 minutes).

  In step S <b> 3, the processing unit 42 performs backup by the first backup device 5. That is, the processing unit 42 transmits the update data ID and value (data content) to the first backup device 5, and the backup server of the first backup device 5 stores the received data. The backup data may be transmitted to the first backup device 5 not by the management device 4 but by the server 3 in the cluster 100.

  Next, in step S4, the processing unit 42 determines whether or not the backup timing by the second backup device 6 has arrived. If Yes, the process proceeds to step S5, and if No, the process proceeds to step S6. The backup timing may be, for example, every predetermined time (for example, one day).

  In step S <b> 5, the processing unit 42 performs backup by the second backup device 6. In other words, the processing unit 42 instructs the first backup device 5 and the second backup device 6 to transmit the ID and value (data content) of the stored data of the first backup device 5 to the second backup device 6. The storage of the second backup device 6 stores the received data.

  In step S6, the processing unit 42 determines whether or not a software failure has occurred in the cluster 100. If Yes, the process proceeds to step S7. If No, the process returns to step S1.

  In step S7, the processing unit 42 determines whether the device to be used for backup is the first backup device 5 or the second backup device 6, and if it determines that the device is the first backup device 5, the process proceeds to step S8. If it is determined that it is the second backup device 6, the process proceeds to step S9. As described above, this determination may be appropriately determined according to the scale and type of data loss, whether or not loss data can be specified.

  In step S <b> 8, the processing unit 42 performs data recovery of the cluster 100 by the first backup device 5. That is, the data lost in the cluster 100 is recovered using the stored data of the first backup device 5.

In step S <b> 9, the processing unit 42 executes data recovery of the cluster 100 by the second backup device 6. That is, the data lost in the cluster 100 is recovered using the stored data of the second backup device 6.
The processing unit 42 returns to Step S1 after Step S8 or Step S9.

  As described above, according to the database system S of the present embodiment, the storage device (the backup server of the first backup device 5 and the storage of the second backup device 6) provided outside the distributed processing system 1000. Similarly, even if a plurality of servers 3 in the distributed processing system 1000 (cluster 100) goes down at the same time and both the original data and the duplicated data are lost at the same time, the backup system 10 Using the data stored in the storage device, it is possible to easily recover the data of the server 3 that has lost the data.

  In addition, the storage devices arranged in two or more different regions in the backup system 10 store the backup data, so that a plurality of servers 3 can be down at the same time due to software failure or the like in the distributed processing system 1000 (cluster 100). In addition, even if all the servers 3 in one area are down due to a catastrophic disaster or the like, some of the storage devices in the backup system 10 are not down, and the management device 4 is stored in the storage device that is not down. Data recovery of the server 3 that has lost the data can be easily performed using the stored data.

Although description of this embodiment is finished above, the aspect of the present invention is not limited to these.
For example, in the present embodiment, the first backup device 5 and the second backup device 6 arranged in different regions as backup devices in order to take measures against both the occurrence of a software failure in the cluster 100 and the occurrence of a catastrophic disaster or the like. However, only one backup device may be used as a countermeasure for the occurrence of a software failure or the like in the cluster 100.

Further, the timing of data backup to the backup device may be either synchronous or asynchronous with respect to the update of the data to be managed.
Further, a load distribution device (load balancer) may be used in the distributed processing system 1000.

Moreover, although the case where the data center area is used as a unit as an area has been described as an example, another unit such as a further divided data center area or a prefecture may be adopted.
Moreover, although the storage device of the first backup device 5 is a backup server and the storage device of the second backup device 6 is a storage, the type of the storage device is not limited thereto, for example, both use storage, or Any other storage device may be used.

In the present embodiment, the consistent hash method is assumed, but another method may be assumed.
In addition, about a concrete structure, it can change suitably in the range which does not deviate from the main point of this invention.

DESCRIPTION OF SYMBOLS 1 Client machine 2 Network 3 Server 4 Management apparatus 5 1st backup apparatus 6 2nd backup apparatus 10 Backup system 41 Memory | storage part 42 Processing part 43 Input part 44 Display part 45 Communication part 100 Cluster 411 ID space management information 412 Area name information 413 Server management information 414 First backup device management information 415 Second backup device management information 1000 Distributed processing system S Database system

Claims (1)

A plurality of servers for processing requests from client machines, the request from the client machine and a management device for allocating to one of said plurality of servers, the annular ID (IDentifier) space, said plurality of servers All of the plurality of managed data to be processed by the server and the plurality of servers are allocated, and each of the servers is located between itself and the next server around the predetermined direction in the ID space. A distributed processing system for managing data and storing a copy of the data located between the next server and the next server around the predetermined direction;
A backup system comprising a storage device for storing the management target data handled by the distributed processing system as a backup;
A database system comprising:
When the management target data is updated in the distributed processing system, either the management device or the plurality of servers transmits the updated data to the backup system,
The storage device in the backup system stores the received data,
When a failure occurs in a plurality of the servers in the distributed processing system and the management target data is lost, the management device uses the data stored in the storage device of the backup system to lose the data. There line data recovery of the server,
The backup system has the storage device in each of two or more different areas, and each of the storage devices stores the backup data,
When a failure occurs in a plurality of the servers in the distributed processing system and the management target data is lost, the management device uses data stored in any of the storage devices in the backup system, A database system for performing data recovery of the server that has lost data .

Images