CN113868022A - Master-slave switching method and device for database - Google Patents
Master-slave switching method and device for database Download PDFInfo
- Publication number
- CN113868022A CN113868022A CN202111101760.8A CN202111101760A CN113868022A CN 113868022 A CN113868022 A CN 113868022A CN 202111101760 A CN202111101760 A CN 202111101760A CN 113868022 A CN113868022 A CN 113868022A
- Authority
- CN
- China
- Prior art keywords
- database
- binary log
- backup
- task
- incremental
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F11/00—Error detection; Error correction; Monitoring
- G06F11/07—Responding to the occurrence of a fault, e.g. fault tolerance
- G06F11/14—Error detection or correction of the data by redundancy in operation
- G06F11/1402—Saving, restoring, recovering or retrying
- G06F11/1446—Point-in-time backing up or restoration of persistent data
- G06F11/1458—Management of the backup or restore process
- G06F11/1469—Backup restoration techniques
-
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/25—Integrating or interfacing systems involving database management systems
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Databases & Information Systems (AREA)
- Quality & Reliability (AREA)
- Data Mining & Analysis (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The application discloses a master-slave switching method and device for a database, which are used for solving the problems of single point of management nodes and data loss during master-slave switching. The method provided in the present application comprises: when a first database serving as a current master database fails, a management node switches a second database in a slave database of the first database into a master database; and acquiring an initial position of data compensation from the second database, and sending a data compensation task to a backup service node, wherein the data compensation task is used for indicating the backup service node to acquire an incremental binary log from the backup data file according to the initial position and write the incremental binary log into the second database to complete the data compensation.
Description
Technical Field
The present application relates to the field of computer technologies, and in particular, to a method and an apparatus for master-slave switching of a database.
Background
MySQL is a database management system. Generally, in order to enhance reliability and stability of a system, a master/slave mode is often used to deploy a database used in the system, and currently, MySQL generally includes a master database and at least one slave database, the master database is used to provide functions such as data management and data query, and the slave database is used to back up data in the master database, so as to provide corresponding functions for the master database when the master database fails. The scheme for realizing the master-slave mode in the MySQL database comprises the following steps: and acquiring a binary log file from the main database from the database, converting the binary log (binlog) file into a relay log file (relaylog), updating data in the backup database according to the relay log file, and generating the binary log file of the backup database.
In some cases, when the master database fails, some binary logs on the master database are not synchronized to the slave database due to network transmission and the like, and in this case, switching the slave database to the master database may cause some binary logs to be lost, which may cause some service interruption.
Disclosure of Invention
The embodiment of the application provides a master-slave switching method and device for a database, which are used for solving the problem of data loss in master-slave switching.
In a first aspect, an embodiment of the present application provides a backup service node, including:
the task scheduling unit is used for determining the initial position of data compensation indicated by the management node and a second database needing to be switched to be used as a main database when a first database used as the current main database fails; the starting position is the position of a binary log which is contained in the second database and is most recently synchronized from the first database;
the first execution unit is used for acquiring the incremental binary log in the backup data file according to the initial position;
the backup data file comprises a binary log file copied from a current main database by the first execution unit in real time, and the incremental binary log is a binary log positioned after the starting position in the binary log file copied from the first database and included in the backup data file;
the first execution unit is further configured to write the incremental binary log into the second database.
Based on the scheme, when the backup service node determines that the first database serving as the main database fails, the initial position of data compensation indicated by the management node and the second database which needs to be switched to serve as the main database are determined, and the incremental binary log in the backup data file is obtained according to the initial position and written into the second database, so that the problem of data loss during switching of the main database and the slave database is solved.
In some embodiments, the backup service node includes a plurality of execution units including the first execution unit, and the task scheduling unit is specifically configured to:
receiving a data compensation task sent by a management node, wherein the data compensation task comprises the initial position, the identifier of the first database and the identifier of the second database, and the data compensation task is used for indicating the backup service node to acquire an incremental binary log from the backup data file according to the initial position and write the incremental binary log into the second database;
determining an execution unit corresponding to the identifier of the first database according to the mapping relation; the mapping relation is used for representing the incidence relation between the identification of the master database and the identification of an execution unit for copying the binary log of the master database in real time;
and the task scheduling unit sends a data recovery task to the execution unit, wherein the data recovery task is used for indicating the execution unit to acquire an incremental binary log from the backup data file of the first database according to the initial position and write the incremental binary log into the second database.
Based on the scheme, different execution units execute the backup tasks of different main databases, and the performance of the backup service node is improved. According to the mapping relation between the execution unit and the backup task, the first execution unit where the data backup file corresponding to the identifier of the first database is located can be accurately found.
In some embodiments, the execution unit is specifically configured to receive the data recovery task sent by the task scheduling unit;
acquiring the incremental binary log from the backup data file of the first database according to the initial position;
and writing the incremental binary log into the second database corresponding to the identifier of the second database.
Based on the scheme, the execution unit receives the data recovery task, acquires the incremental binary log and executes the data compensation task, so that the problem of data loss of the second database relative to the first database is solved.
In some embodiments, the task scheduling unit is further configured to receive a backup task of the first database, determine, according to resource usage of multiple execution units included in the backup service node, the first execution unit that executes the backup task of the first database from the multiple execution units, and send indication information to the first execution unit, where the indication information is used to indicate the first execution unit to copy a binary log from the first database.
Based on the scheme, the task scheduling unit determines the first execution unit of the backup task of the first database according to the resource use condition of the execution unit, so that the problem that the execution unit is interrupted due to excessive load is avoided. The task scheduling unit instructs the first execution unit to copy the binary log from the first database, provides a data backup file for the data compensation task, and ensures the execution of the data compensation task.
In a second aspect, an embodiment of the present application provides a management node, including:
the system comprises a processing unit, a database switching unit and a database switching unit, wherein the processing unit is used for switching a second database in a slave database of a first database into a master database when the first database serving as a current master database fails;
a communication unit, configured to obtain a starting position of data compensation from the second database, where the starting position is a position of a binary log included in the second database that is most recently synchronized from the first database;
the communication unit is also used for indicating the backup service node to acquire an incremental binary log from the backup data file according to the initial position and write the incremental binary log into the second database;
the backup data file comprises a binary log file which is copied from a current main database by the backup service node in real time, and the incremental binary log is a binary log which is positioned behind the initial position in the binary log file copied from the first database.
In some embodiments, the communication unit is specifically configured to send a data compensation task to a backup service node, where the data compensation task includes the starting location, the identifier of the first database, and the identifier of the second database, and the data compensation task is configured to instruct the backup service node to obtain an incremental binary log from the backup data file according to the starting location and write the incremental binary log into the second database.
In a third aspect, an embodiment of the present application provides a method for master-slave switching of a database, where the method is applied to a backup service node, and includes:
when a first database serving as a current main database fails, determining an initial position of data compensation indicated by a management node and a second database to be switched as the main database;
the starting position is the position of a binary log which is contained in the second database and is most recently synchronized from the first database;
acquiring an incremental binary log in a backup data file according to the initial position;
the backup data file comprises a binary log file copied from a current main database by the backup service node in real time, and the incremental binary log is a binary log positioned behind the initial position in the binary log file copied from the first database and included in the backup data file;
writing the incremental binary log to the second database.
In some embodiments, the determining a starting location of data compensation indicated by the management node and a second database to be switched as a master database includes:
and receiving a data compensation task sent by a management node, wherein the data compensation task comprises the initial position, the identifier of the first database and the identifier of the second database, and the data compensation task is used for indicating the backup service node to acquire an incremental binary log from the backup data file according to the initial position and write the incremental binary log into the second database.
In a fourth aspect, an embodiment of the present application provides a master-slave switching method for a database, where the method is applied to a management node, and includes:
when a first database serving as a current master database fails, switching a second database in slave databases of the first database into a master database;
acquiring a starting position of data compensation from the second database, wherein the starting position is a position of a binary log which is included in the second database and is synchronized from the first database recently;
indicating the backup service node to acquire an incremental binary log from the backup data file according to the initial position and writing the incremental binary log into the second database;
the backup data file comprises a binary log file which is copied from a current main database by the backup service node in real time, and the incremental binary log is a binary log which is positioned behind the initial position in the binary log file copied from the first database.
In some embodiments, instructing the backup service node to obtain the incremental binary log from the backup data file according to the starting location, and writing the incremental binary log into the second database may be implemented by: and sending a data compensation task to a backup service node, wherein the data compensation task comprises the initial position, the identifier of the first database and the identifier of the second database, and the data compensation task is used for indicating the backup service node to acquire an incremental binary log from the backup data file according to the initial position and write the incremental binary log into the second database.
In a fifth aspect, an embodiment of the present application provides a master-slave switching apparatus for a database, including:
a memory and a processor;
a memory for storing program instructions;
a processor, configured to call the program instructions stored in the memory, and execute the method according to any one of the third aspect or the third aspect, or execute the method according to any one of the fourth aspect or the fourth aspect, according to the obtained program.
In addition, for technical effects brought by any one implementation manner of the second aspect to the fifth aspect, reference may be made to technical effects brought by different implementation manners of the first aspect, and details are not described here.
Drawings
In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings needed to be used in the description of the embodiments or the prior art will be briefly introduced below, and it is obvious that the drawings in the following description are some embodiments of the present application, and for those skilled in the art, other drawings can be obtained according to these drawings without inventive exercise.
Fig. 1 is a schematic diagram of a service system architecture for master-slave switching of a database according to an embodiment of the present application;
fig. 2A is a schematic structural diagram of a backup service node according to an embodiment of the present disclosure;
fig. 2B is a schematic structural diagram of another backup service node according to an embodiment of the present application;
fig. 3 is a schematic flowchart of a master-slave switching method for a database according to an embodiment of the present application;
fig. 4 is a schematic diagram of an apparatus for master-slave switching of a database according to an embodiment of the present disclosure;
fig. 5 is a schematic diagram of another apparatus for performing master-slave switching on a database according to an embodiment of the present disclosure.
Detailed Description
To make the objects, embodiments and advantages of the present application clearer, the following description of exemplary embodiments of the present application will clearly and completely describe the exemplary embodiments of the present application with reference to the accompanying drawings in the exemplary embodiments of the present application, and it is to be understood that the described exemplary embodiments are only a part of the embodiments of the present application, and not all of the embodiments. It should be noted that the brief descriptions of the terms in the present application are only for the convenience of understanding the embodiments described below, and are not intended to limit the embodiments of the present application. These terms should be understood in their ordinary and customary meaning unless otherwise indicated.
Fig. 1 shows a schematic architecture diagram of a service system for master-slave switching of a database provided in the present application, it should be understood that the present application is not limited to the system shown in fig. 1, and in addition, the apparatus in fig. 1 may be hardware, or may be a structure that is functionally divided by software, or a combination of the two. As shown in fig. 1, the business system provided by the present application includes a management node, a database, and a backup service node. The database may employ MySQL. Currently, MySQL generally includes a main database for providing data management and data query functions, and at least one backup database for backing up data in the main database to provide corresponding functions for the main database when the main database fails. In fig. 1, the database includes a master database and N slave databases corresponding to the master database as an example. It should be noted that the number of the master databases included in the business system is not specifically limited in the embodiment of the present application. The scheme for realizing the main/standby mode in the MySQL database is as follows: the backup database acquires a binary log (bin log) file from the main database, converts the binary log file into a relay log file (relay log), updates data in the backup database according to the relay log file, and generates the binary log file of the backup database.
And the binary log file is used for recording all events for modifying the database data in the MySQL. The event is actually a record of one-time operation on the database, and includes information such as a statement executed by the MySQL server, an execution time point of the statement, a client thread ID initiating execution, and a time overhead taken by the statement execution. Operations may include create operations (create) insert operations (insert), delete operations (drop) to delete tables, update operations (update), and so forth. The binary log files described in the present application include all the binary log files currently used by the MySQL database.
In some embodiments, the format of the relay log file is the same as the format of the binary log. The slave server can parse the binary log of the master database and make the same update to the data of the slave database. Illustratively, there are two threads from the database, I/O threads, SQL threads. The I/O thread is used for downloading the binary log of the master database, a relay log is locally created each time the I/O thread is started, and the SQL thread acquires the data of the relay log to update the slave database. And the SQL thread automatically deletes the relay log file after reading all the update statements.
The management node may be a server or a cluster of servers. A plurality of servers may be included in a server cluster. The management node is used for managing the database, such as executing the switching between a master database and a slave database. And issuing service tasks for the database, such as a storage task or a reading task. The management node is used for providing a function of the high-availability service of the database, or one or more high-availability services of the database can be deployed in the management node. Illustratively, the management node may employ an Orchester service system. Namely, the high-availability service of the database can adopt the high-availability service of the Orchester database.
The backup service node is realized by one server or a plurality of servers, and when a plurality of servers are used, the backup service node can be deployed in a cluster mode. The backup service node supports real-time replication of the binary log from the main database of the business system. When determining that a certain master database fails, the management node refers to the failed master database as a first database. That is, when a first database of the current master database fails, a second database of the slave databases of the first database may be switched to the master database. And then obtaining the initial position of data compensation from the second database. The starting location may be understood as the location of the binary log comprised by the second database that was most recently synchronized from the first database. It will be appreciated that each binary log is stored to a binary log file with a location corresponding to the location of the binary log file. The starting position here can therefore be understood as the position where the second database is in the latest binary log synchronized in real time from the first database. Further, the management node may instruct the backup service node to perform data compensation of the second database according to the starting location. The starting location, the identity of the first database, and the identity of the second database may be indicated to the backup service node. Therefore, when a first database serving as the current main database fails, the backup service node determines the initial position of data compensation indicated by the management node and needs to switch to a second database serving as the main database. And then acquiring the incremental binary log from the backup data file under the instruction of the management node, and writing the incremental binary log into a second database. The backup data file comprises a binary log file which is copied from the current main database by the backup service node in real time, and the incremental binary log is a binary log which is positioned behind the initial position in the binary log file copied from the first database.
In some embodiments, when the management node instructs the backup service node to obtain the incremental binary log from the backup data file according to the starting position and write the incremental binary log into the second database, the method may be implemented as follows: the management node sends a data compensation task to the backup service node, the data compensation task comprises an initial position, an identifier of the first database and an identifier of the second database, and the data compensation task is used for indicating the backup service node to acquire an incremental binary log from the backup data file according to the initial position and write the incremental binary log into the second database.
Further, the backup service receives a data compensation task sent by the management node for the node, the data compensation task includes an initial position, an identifier of the first database and an identifier of the second database, and the data compensation task is used for instructing the backup service node to acquire an incremental binary log from the backup data file according to the initial position and write the incremental binary log into the second database.
In some embodiments, when the backup service node obtains the incremental binary log in the backup data file according to the starting position, the method may be implemented as follows:
the backup service node determines the storage position of the backup data file of the first database according to the incidence relation between the identifier of the first database and the storage position for storing the backup data file of the first database; acquiring an incremental binary log in the backup data file from a storage position according to the initial position; and writing the incremental binary log into the second database corresponding to the identification of the second database.
The structure of the backup service node is described below from the perspective of logical functional partitioning. Referring to fig. 2A, the backup service node includes a storage unit, a task scheduling unit, and an execution unit, where N execution units are illustrated in fig. 2A as an example. The functions of the storage unit may be implemented by one or more storage servers. The function of the task scheduling unit may also be implemented by one or more servers. The execution units may be implemented by one or more servers, and in some embodiments, different execution units may be deployed in different servers. The execution unit may be referred to as an execution node, and other names may also be used, which is not specifically limited in this application. The execution unit may be deployed in a container, such as a pod. In some embodiments, the execution unit is configured to execute a backup task of the binary log for the master database, and the binary log may be copied from the master database in real time. In some embodiments, one execution unit may perform backup tasks for multiple primary databases, with different execution units performing backup tasks for different primary databases. For example, the maximum number of executable backup tasks may be uniformly configured for each execution unit, and different backup tasks correspond to backups of different main databases. When the number of the backup tasks running in the execution unit reaches the maximum number of the backup tasks, the backup tasks distributed by the task scheduling unit are not received any more, so that the backup performance of each execution unit is controllable. In some embodiments, the execution unit may employ a pod deployment manner, and different pods may execute backup tasks for different primary databases, for example, one pod may execute backup tasks of 3 primary databases, see fig. 2B.
The storage unit may store an association relationship between an execution unit executing the backup task and the targeted master database, such as an association relationship between an identifier of the master database and an identifier of an execution unit used for copying a binary log of the master database in real time.
When a primary database is deployed in a business system, an execution unit that executes a backup task of the primary database may be selected from a plurality of execution units by a task scheduling unit. In some embodiments, when receiving the new backup task, the task scheduling unit may select an execution unit from the plurality of execution units to execute the new backup task according to the resource index of each execution unit in the service system. The resource index may include parameters such as the number of current backup tasks, the remaining amount of processing resources, the remaining amount of memory, and the remaining amount of disk. Processing resources may be understood as the amount of Central Processing Unit (CPU) resources remaining. After allocating the execution unit for the new backup task, the task scheduling unit may store the association relationship between the main database and the execution unit corresponding to the new backup task in the storage unit.
Taking a backup task of a first database as an example, a task scheduling unit receives a backup task of the first database, determines a first execution unit for executing the backup task of the first database from a plurality of execution units according to resource usage of the plurality of execution units included in a backup service node, and sends indication information to the first execution unit, wherein the indication information is used for indicating the first execution unit to execute the backup task of the first database, and the backup task indicates the first execution unit to copy a binary log of the first database.
In some embodiments, the first execution unit, upon receiving the backup task, may initiate a binary backup process to execute the backup task. The first execution unit may initiate different binary backup processes for backup tasks of different master databases to execute. The first execution unit can also execute the keep-alive operation of the binary system backup process and monitor whether the backup task executed by the binary system backup process is normal. The first execution unit may also support binary backup process start-stop. For example, when a certain primary database fails, the management node may stop executing the backup task of the failed database when receiving the data recovery task, that is, may stop copying the binary log from the failed primary database, for example, may stop the binary backup process of the failed primary database.
In some embodiments, the service system may further include a monitoring unit, configured to obtain information such as health status information of the backup task, resource index information, and current backup progress of each execution unit. When allocating the execution unit for the backup task, the task scheduling unit may obtain the resource index information from the monitoring unit, and then allocate the execution unit for the backup task according to the resource index information. The backup task health status information may include whether the execution unit is executing normally, whether there is a failure, etc. when executing the backup task. The current backup progress may include the location of the binary log that the execution unit copied from the master database.
For example, the monitoring unit may also monitor that when a certain execution unit fails to execute a backup task of a certain primary database, the execution task scheduling unit may migrate the backup task of the primary data to another execution unit.
In some embodiments, the task scheduling service is deployed in the task scheduling unit to perform functions such as allocation of backup tasks. Illustratively, the task scheduling service deployed by the task scheduling unit may be configured as a stateless service, such as supporting a k8s stateless service (deployment). The task scheduling unit can also support the expansion and contraction capacity in the service system.
In a possible implementation manner, the task scheduling unit may further execute the data compensation task. For example, it is exemplified that a data compensation task is performed for a slave database that is a first database as a master database. And when a first database serving as the current main database fails, the task scheduling unit determines the initial position of data compensation indicated by the management node and a second database required to be switched to serve as the main database. And the execution unit corresponding to the first database in the backup service node acquires the incremental binary log in the backup data file of the first database according to the initial position. Take the execution unit corresponding to the first database as the first execution unit as an example. The first execution unit acquires an incremental binary log in the backup data file according to the initial position; the first execution unit then writes the incremental binary log to a second database.
Illustratively, the highly available service deployed by the management node sends the data compensation task to the task scheduling unit. The data compensation task comprises a starting position of data compensation, an identifier of the first database and an identifier of the second database. The task scheduling unit may find the first execution unit corresponding to the identifier of the first database and used for copying the binary log of the first database in real time according to the association relationship between the identifier of the first database recorded in the storage unit and the identifier of the execution unit. And the task scheduling unit sends a data recovery task to the first execution unit, wherein the data recovery task is used for indicating the first execution unit to acquire the incremental binary log from the backup data file of the first database according to the initial position and write the incremental binary log into the second database.
Illustratively, the first execution unit receives the data recovery task sent by the task scheduling unit; then, acquiring an incremental binary log from a backup data file of the first database according to the initial position; and writing the incremental binary log into the second database corresponding to the identification of the second database.
In some embodiments, the execution unit is further configured to clean the backup data, and instruct migration of the backup task to be executed according to the task scheduling unit. For example, a manager may configure a data cleaning cycle through an interface of the task scheduling unit, so that the task scheduling unit may periodically trigger a cleaning task to the execution unit, so as to clean the copied binary log that exceeds a set time length from the current time. When a certain execution unit (for convenience of description, for example, an old execution unit) cannot execute a backup task due to insufficient resources, the task scheduling unit may select one execution unit (for example, a new execution unit) from other execution units to continue executing the backup task executed by the old execution unit, and the task scheduling unit may migrate the backup task of the old execution unit to the new execution unit.
In some embodiments, the service system may further include a monitoring unit, and the monitoring unit may detect, through a communication interface on each execution unit, information such as an operation state of a backup task on the execution node and a resource usage condition of the execution node. For example, the monitoring unit may periodically acquire information such as an operating state and a resource usage from each execution unit. For another example, the monitoring unit may send a subscription message to each execution unit, indicating the subscribed information and the reporting period. Therefore, each execution unit periodically reports information such as the running state, the resource use condition and the like to the monitoring unit.
Fig. 3 exemplarily shows a flow of the master-slave switching method for a database, where the flow may be performed by a master-slave switching apparatus for a database, and the apparatus may be located in a business system as shown in fig. 1.
Fig. 3 is a method for master-slave switching of a database, which includes the following specific steps:
301, the management node monitors the health status of the master database in real time, and monitors the fault of the master database.
302, when the primary database fails, the management node selects a second database to be switched to the primary database.
Taking the current master database as the first database as an example, when the management node fails to serve as the first database of the current master database, the management node may select one database from the slave databases of the first database as the master database to be switched. For example, the slave database with the largest binary log position in the slave database of the first database may be selected. The slave database with the largest binary log position can be understood as that the speed of the binary logs synchronized in the slave master database in the slave database is fastest and most efficient, so that the binary logs in the slave database have the smallest difference compared with the master database.
In particular, in the MySQL database management system, a management node may be used to provide a function of a database high availability service, or one or more database high availability services may be deployed in the management node. Illustratively, the management node may employ an Orchester service system. Namely, the database high-availability service can adopt an Orchester database high-availability service, which is hereinafter referred to as a high-availability service. The high availability service periodically monitors the running state of a first database as a master database, when the first database is found to be in failure, a second database is selected from slave databases of the first database, and the second database is switched to a new master database.
In some embodiments, if only one slave database with the largest binary log position is in the slave databases of the first database, the slave database is selected as the master database to be switched.
In other embodiments, if there are two or more slave databases with the largest binary log position, the relay log of the slave database with the largest binary log position is obtained, and the slave database with the largest binary log position stored in the relay log is selected as the master database to be switched.
In some embodiments, if there are two or more slave databases with the largest relay log position, one slave database is selected as the master database to be switched from among the slave databases with the largest relay log position, for example, one slave database may be selected randomly.
For convenience of the following description, the slave database to be switched is referred to as a second database as an example.
303, the management node obtains the starting position of the data compensation of the second database.
And after selecting a second database needing to be switched into the main database, the management node acquires the latest position of the binary log of the second database, wherein the latest position is the initial position of data compensation. The starting location is a location of a binary log included by the second database that was most recently synchronized from the first database.
In some embodiments, the binary log in the master database is replicated in real-time from the database. And when the second database is selected as a new main database, the high-availability service determines the position of the binary log which is synchronized from the first database recently in the second database, and takes the position as the starting position of data compensation.
304, the management node sends the data compensation task to the backup service node, and instructs the backup service node to execute the data compensation task.
The data compensation task comprises the starting position, the identifier of the first database and the identifier of the second database, and is used for indicating the backup service node to compensate the data difference of the binary logs in the first database and the second database into the second database according to the starting position.
The backup service node is realized by one server or a plurality of servers, and when a plurality of servers are used, the backup service node can be deployed in a cluster mode. In some embodiments, the log backup and application service may be deployed in a backup service node. Illustratively, the log backup and application services may include log backup and restore services and task scheduling services. The log backup and restore service is used to perform backup tasks for the primary data and data compensation tasks for the secondary databases that are promoted to the primary database. The task scheduling service is used to interface with highly available services, receive data compensation tasks or backup tasks, and so on. In some embodiments, a backup service node may include a plurality of execution units and a task scheduling unit. The execution unit can deploy log backup and recovery service, and the task scheduling unit deploys task scheduling service.
And the high-availability service acquires the synchronous position of the binary log in the second database, sends the data compensation task to the log backup and application service, and the task scheduling service in the log backup and application service instructs the log backup and recovery service to execute the data compensation task according to the data compensation task.
In some embodiments, the secondary database suspends the sync state operation, i.e., stops copying the binary log from the primary database, before the management node sends the data compensation task to the backup service node. And disconnecting the first database from the second database, and finishing the master-slave relationship between the first database and the second database. Further, the management node may set the second database to a read-only mode.
In some embodiments, after the management node sets the second database to read-only mode, the second database is set as the new master database and an interface is configured on the second database through which the management node can trigger data compensation tasks to the backup service node. For example, the highly available service may configure parameters of the data compensation task in hook, and invoke the log backup and application service through hook to instruct an execution node in the backup service node to perform the data compensation task on the second database. The parameters of the data compensation task comprise the identification of the first database, the identification of the second database and the starting position of the data compensation of the second database. Illustratively, hook may be understood as a thread hook or a function, and when there is a data compensation task, the highly available service calls the log backup and application service through the hook function via an interface on the second database, and instructs an execution node on the backup service node to execute the data compensation task on the second database.
305, after receiving the data compensation task, the backup service node acquires the incremental binary log.
The backup service node receives a data compensation task sent by the management node, finds a first execution unit corresponding to the identifier of the first database according to the identifier of the first database provided by the management node, and acquires an incremental binary log from a backup data file of the first execution unit corresponding to the identifier of the first database according to the initial position of data compensation. The backup data file comprises a binary log file which is copied from the current main database by the backup service node in real time. The incremental binary log is a binary log located after the starting location in a binary log file copied from the first database.
In some embodiments, each execution unit stores a backup data file corresponding to the primary database. The execution unit may copy the binary log file from the master database and store it in the backup data file in real time.
After determining the first execution unit corresponding to the identifier of the first database, the backup service node may send the start position of the data compensation, the identifier of the first database, and the identifier of the second database to the first execution unit. And the first execution unit acquires the incremental binary log from the backup data file corresponding to the identifier of the first database.
And 306, writing the acquired incremental binary log into a second database by the backup service node.
For example, the backup service node writes the obtained incremental binary log into the second database corresponding to the identifier of the second database.
In one possible implementation, when the maximum position of the binary log in the backup data file corresponding to the identification of the first database is less than or equal to the starting position of the data compensation, the task scheduling unit no longer instructs the first execution unit to execute the data compensation task on the second database.
307, the backup service node feeds back the compensation result to the management node.
And whether the backup service node executes the data compensation task or not, the data compensation result is fed back to the management node so that the management node can perform further processing.
The management node processes the compensation results 308 to restore service.
And after the backup service node finishes the data compensation task, feeding back the compensation result to the management node, and restoring the service by the management node according to the compensation result.
In one example, after the log backup and application service deployed by the backup service node completes the data compensation task, the compensation result is fed back to the high-availability service through hook. And the high-availability service acquires the compensation result through hook, and if the compensation is successful, the second database is set to be writable so as to recover the service.
In another example, the data compensation fails or times out, and the highly available service performs further operations by configuring a processing policy for the data compensation failing or times out, such as the processing policy being tolerant to partial data loss, wherein the amount of data that is tolerant to loss may be included in the processing policy. When the data compensation fails or times out, the high availability service can inform the management equipment to inform the management personnel, and then the management personnel can process the tasks of the data compensation failure or time out.
In some embodiments, when data compensation fails or times out, the second database may be set to writable and service of the second database may be restored.
Based on the same concept as the method described above, referring to fig. 4, an embodiment of the present application provides a master-slave switching apparatus 400 for a database. The apparatus 400 is capable of performing the steps performed by the management node in the above method, and will not be described in detail herein to avoid redundancy. The apparatus 400 comprises: a processing unit 401 and a communication unit 402.
The processing unit 401 is configured to, when a first database serving as a current master database fails, switch a second database in slave databases of the first database to the master database.
A communication unit 402, configured to obtain a starting position of data compensation from the second database, where the starting position is a position of a binary log included in the second database that is most recently synchronized from the first database.
The communication unit 402 is further configured to instruct the backup service node to obtain an incremental binary log from the backup data file according to the starting position, and write the incremental binary log into the second database.
The backup data file comprises a binary log file which is copied from a current main database by the backup service node in real time, and the incremental binary log is a binary log which is positioned behind the initial position in the binary log file copied from the first database.
In some embodiments, the communication unit 402 is specifically configured to send a data compensation task to a backup service node, where the data compensation task includes the starting location, the identifier of the first database, and the identifier of the second database, and the data compensation task is configured to instruct the backup service node to obtain an incremental binary log from the backup data file according to the starting location and write the incremental binary log into the second database.
The embodiment of the present application further provides a master-slave switching apparatus 500 for a database, which is shown in fig. 5 and includes a memory 501 and a processor 502. Optionally, the apparatus 500 may also include a communication interface 503. Wherein the apparatus 500 communicates with other devices through said communication interface 503. A memory 501 for storing program instructions.
In one possible scenario, the master-slave switching apparatus 500 of the database may be applied to the management node. Such as the processor 502, for calling the program instructions stored in the memory 501, and executing the method executed by the management node in the above embodiments according to the obtained program. The processor 502 may interact with the backup service node via the communication interface 503.
In another possible scenario, the master-slave switching device 500 of the database may be applied to the backup service node. Such as the processor 502, is configured to call the program instructions stored in the memory 501, and execute the method executed by the backup service node in the foregoing embodiment according to the obtained program. The processor 502 may interact with the management node via the communication interface 503.
In the embodiment of the present application, a specific connection medium between the memory 501, the processor 502, and the communication interface 503 is not limited, for example, a bus may be divided into an address bus, a data bus, a control bus, and the like.
In the embodiments of the present application, the processor may be a general-purpose processor, a digital signal processor, an application specific integrated circuit, a field programmable gate array or other programmable logic device, a discrete gate or transistor logic device, or a discrete hardware component, and may implement or execute the methods, steps, and logic blocks disclosed in the embodiments of the present application. A general purpose processor may be a microprocessor or any conventional processor or the like. The steps of a method disclosed in connection with the embodiments of the present application may be directly implemented by a hardware processor, or may be implemented by a combination of hardware and software modules in a processor.
In the embodiment of the present application, the memory may be a nonvolatile memory, such as a Hard Disk Drive (HDD) or a solID-state drive (SSD), and may also be a volatile memory (RAM), for example, a random-access memory (RAM). The memory can also be, but is not limited to, any other medium that can be used to carry or store desired program code in the form of instructions or data structures and that can be accessed by a computer. The memory in the embodiments of the present application may also be circuitry or any other device capable of performing a storage function for storing program instructions and/or data.
Embodiments of the present application also provide a computer-readable storage medium, which includes program code for causing a computer to perform the steps of the method provided by the embodiments of the present application when the program code runs on the computer.
As will be appreciated by one skilled in the art, embodiments of the present application may be provided as a method, system, or computer program product. Accordingly, the present application may take the form of an entirely hardware embodiment, an entirely software embodiment or an embodiment combining software and hardware aspects. Furthermore, the present application may take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, and the like) having computer-usable program code embodied therein.
The present application is described with reference to flowchart illustrations and/or block diagrams of methods, apparatus (systems), and computer program products according to the application. It will be understood that each flow and/or block of the flow diagrams and/or block diagrams, and combinations of flows and/or blocks in the flow diagrams and/or block diagrams, can be implemented by computer program instructions. These computer program instructions may be provided to a processor of a general purpose computer, special purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including instruction means which implement the function specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
It will be apparent to those skilled in the art that various changes and modifications may be made in the present application without departing from the spirit and scope of the application. Thus, if such modifications and variations of the present application fall within the scope of the claims of the present application and their equivalents, the present application is intended to include such modifications and variations as well.
Claims (10)
1. A backup service node, comprising:
the task scheduling unit is used for determining the initial position of data compensation indicated by the management node and a second database needing to be switched to be used as a main database when a first database used as the current main database fails; the starting position is the position of a binary log which is contained in the second database and is most recently synchronized from the first database;
the first execution unit is used for acquiring the incremental binary log in the backup data file according to the initial position;
the backup data file comprises a binary log file copied from a current main database by the first execution unit in real time, and the incremental binary log is a binary log positioned after the starting position in the binary log file copied from the first database and included in the backup data file;
the first execution unit is further configured to write the incremental binary log into the second database.
2. The backup service node of claim 1, wherein the backup service node includes a plurality of execution units including the first execution unit, and the task scheduling unit is specifically configured to:
receiving a data compensation task sent by a management node, wherein the data compensation task comprises the initial position, the identifier of the first database and the identifier of the second database, and the data compensation task is used for indicating the backup service node to acquire an incremental binary log from the backup data file according to the initial position and write the incremental binary log into the second database;
determining an execution unit corresponding to the identifier of the first database according to the mapping relation; the mapping relation is used for representing the incidence relation between the identification of the master database and the identification of an execution unit for copying the binary log of the master database in real time;
and the task scheduling unit sends a data recovery task to the execution unit, wherein the data recovery task is used for indicating the execution unit to acquire an incremental binary log from the backup data file of the first database according to the initial position and write the incremental binary log into the second database.
3. The backup service node of claim 2, wherein the execution unit is specifically configured to:
receiving a data recovery task sent by the task scheduling unit;
acquiring the incremental binary log from the backup data file of the first database according to the initial position;
and writing the incremental binary log into the second database corresponding to the identifier of the second database.
4. The backup service node according to any of claims 1 to 3, wherein the task scheduling unit is further configured to receive a backup task of the first database, determine the first execution unit that executes the backup task of the first database from the plurality of execution units according to resource usage of the plurality of execution units included in the backup service node, and send indication information to the first execution unit, where the indication information is used to instruct the first execution unit to copy a binary log from the first database.
5. A management node, comprising:
the system comprises a processing unit, a database switching unit and a database switching unit, wherein the processing unit is used for switching a second database in a slave database of a first database into a master database when the first database serving as a current master database fails;
a communication unit, configured to obtain a starting position of data compensation from the second database, where the starting position is a position of a binary log included in the second database that is most recently synchronized from the first database;
the communication unit is also used for indicating the backup service node to acquire an incremental binary log from the backup data file according to the initial position and write the incremental binary log into the second database;
the backup data file comprises a binary log file which is copied from a current main database by the backup service node in real time, and the incremental binary log is a binary log which is positioned behind the initial position in the binary log file copied from the first database.
6. The management node according to claim 5, wherein the communication unit is specifically configured to send a data compensation task to a backup service node, where the data compensation task includes the start position, an identifier of the first database, and an identifier of the second database, and the data compensation task is configured to instruct the backup service node to obtain an incremental binary log from the backup data file according to the start position and write the incremental binary log into the second database.
7. A master-slave switching method of a database is characterized in that the method is applied to a backup service node and comprises the following steps:
when a first database serving as a current main database fails, determining an initial position of data compensation indicated by a management node and a second database to be switched as the main database;
the starting position is the position of a binary log which is contained in the second database and is most recently synchronized from the first database;
acquiring an incremental binary log in a backup data file according to the initial position;
the backup data file comprises a binary log file copied from a current main database by the backup service node in real time, and the incremental binary log is a binary log positioned behind the initial position in the binary log file copied from the first database and included in the backup data file;
writing the incremental binary log to the second database.
8. The method of claim 7, wherein determining a starting location of data compensation indicated by a management node and a second database to be switched as a master database comprises:
and receiving a data compensation task sent by a management node, wherein the data compensation task comprises the initial position, the identifier of the first database and the identifier of the second database, and the data compensation task is used for indicating the backup service node to acquire an incremental binary log from the backup data file according to the initial position and write the incremental binary log into the second database.
9. A master-slave switching method of a database is applied to a management node and comprises the following steps:
when a first database serving as a current master database fails, switching a second database in slave databases of the first database into a master database;
acquiring a starting position of data compensation from the second database, wherein the starting position is a position of a binary log which is included in the second database and is synchronized from the first database recently;
indicating the backup service node to acquire an incremental binary log from the backup data file according to the initial position and writing the incremental binary log into the second database;
the backup data file comprises a binary log file which is copied from a current main database by the backup service node in real time, and the incremental binary log is a binary log which is positioned behind the initial position in the binary log file copied from the first database.
10. A master-slave switching apparatus for a database, comprising:
a memory and a processor;
a memory for storing program instructions;
a processor for calling the program instructions stored in the memory and executing the method of any one of claims 7 to 9 according to the obtained program.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111101760.8A CN113868022A (en) | 2021-09-18 | 2021-09-18 | Master-slave switching method and device for database |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111101760.8A CN113868022A (en) | 2021-09-18 | 2021-09-18 | Master-slave switching method and device for database |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN113868022A true CN113868022A (en) | 2021-12-31 |
Family
ID=78992939
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202111101760.8A Pending CN113868022A (en) | 2021-09-18 | 2021-09-18 | Master-slave switching method and device for database |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN113868022A (en) |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN115292408A (en) * | 2022-07-19 | 2022-11-04 | 天翼云科技有限公司 | Master-slave synchronization method, device, equipment and medium for MySQL database |
| CN115599604A (en) * | 2022-10-27 | 2023-01-13 | 北京人大金仓信息技术股份有限公司（Cn） | Automatic data recovery method of database cluster and storage medium |
Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US8041679B1 (en) * | 2008-06-04 | 2011-10-18 | Symantec Operating Corporation | Synthetic differential backups creation for a database using binary log conversion |
| CN105095245A (en) * | 2014-05-04 | 2015-11-25 | 阿里巴巴集团控股有限公司 | Filing log synchronizing method and system based on relevance database |
| CN106383861A (en) * | 2016-08-31 | 2017-02-08 | 网易（杭州）网络有限公司 | Data synchronization method and apparatus used for databases |
| CN106933703A (en) * | 2015-12-30 | 2017-07-07 | 阿里巴巴集团控股有限公司 | A kind of method of database data backup, device and electronic equipment |
| CN108984569A (en) * | 2017-06-05 | 2018-12-11 | 中兴通讯股份有限公司 | Database switching method, system and computer readable storage medium |
| CN110795287A (en) * | 2019-10-30 | 2020-02-14 | 深圳前海环融联易信息科技服务有限公司 | Data recovery method, system, electronic device and computer storage medium |
| CN112286729A (en) * | 2020-11-03 | 2021-01-29 | 浪潮云信息技术股份公司 | Method for restoring specified time |
-
2021
- 2021-09-18 CN CN202111101760.8A patent/CN113868022A/en active Pending
Patent Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US8041679B1 (en) * | 2008-06-04 | 2011-10-18 | Symantec Operating Corporation | Synthetic differential backups creation for a database using binary log conversion |
| CN105095245A (en) * | 2014-05-04 | 2015-11-25 | 阿里巴巴集团控股有限公司 | Filing log synchronizing method and system based on relevance database |
| CN106933703A (en) * | 2015-12-30 | 2017-07-07 | 阿里巴巴集团控股有限公司 | A kind of method of database data backup, device and electronic equipment |
| CN106383861A (en) * | 2016-08-31 | 2017-02-08 | 网易（杭州）网络有限公司 | Data synchronization method and apparatus used for databases |
| CN108984569A (en) * | 2017-06-05 | 2018-12-11 | 中兴通讯股份有限公司 | Database switching method, system and computer readable storage medium |
| CN110795287A (en) * | 2019-10-30 | 2020-02-14 | 深圳前海环融联易信息科技服务有限公司 | Data recovery method, system, electronic device and computer storage medium |
| CN112286729A (en) * | 2020-11-03 | 2021-01-29 | 浪潮云信息技术股份公司 | Method for restoring specified time |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN115292408A (en) * | 2022-07-19 | 2022-11-04 | 天翼云科技有限公司 | Master-slave synchronization method, device, equipment and medium for MySQL database |
| CN115599604A (en) * | 2022-10-27 | 2023-01-13 | 北京人大金仓信息技术股份有限公司（Cn） | Automatic data recovery method of database cluster and storage medium |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN111427728B (en) | State management method, main/standby switching method and electronic equipment | |
| CN103647849B (en) | Method and device for migrating businesses and disaster recovery system | |
| WO2017177941A1 (en) | Active/standby database switching method and apparatus | |
| CN108847982B (en) | Distributed storage cluster and node fault switching method and device thereof | |
| CN101539873B (en) | Data recovery method, data node and distributed file system | |
| CN107357688B (en) | Distributed system and fault recovery method and device thereof | |
| CN105830033B (en) | System and method for supporting persistent storage versioning and integrity in a distributed data grid | |
| JP2008059583A (en) | Cluster system, method for backing up replica in cluster system, and program product | |
| CN111460039A (en) | Relational database processing system, client, server and method | |
| CN113986450B (en) | Virtual machine backup method and device | |
| CN106657167B (en) | Management server, server cluster, and management method | |
| CN115145782A (en) | A server switching method, MooseFS system and storage medium | |
| CN114328033B (en) | Method and device for maintaining service configuration consistency of high-availability equipment group | |
| CN118277488B (en) | Distributed storage cluster management method, device and medium for super fusion system | |
| CN115878361A (en) | Node management method and device for database cluster and electronic equipment | |
| CN110196749B (en) | Virtual machine recovery method and device, storage medium and electronic device | |
| CN113868022A (en) | Master-slave switching method and device for database | |
| CN111158956A (en) | Data backup method and related device for cluster system | |
| CN113946471A (en) | Distributed file-level backup method and system based on object storage | |
| CN109582459A (en) | The method and device that the trustship process of application is migrated | |
| CN104793981A (en) | Online snapshot managing method and device for virtual machine cluster | |
| CN113515574B (en) | Data synchronization method and device | |
| CN106452836A (en) | Method and apparatus for setting host node | |
| CN108900331A (en) | A kind of distributed type assemblies management method and distributed type assemblies | |
| CN116185697B (en) | Container cluster management method, device, system, electronic equipment and storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |