CN108206843B - Cluster access method and device - Google Patents
Cluster access method and device Download PDFInfo
- Publication number
- CN108206843B CN108206843B CN201611170510.9A CN201611170510A CN108206843B CN 108206843 B CN108206843 B CN 108206843B CN 201611170510 A CN201611170510 A CN 201611170510A CN 108206843 B CN108206843 B CN 108206843B
- Authority
- CN
- China
- Prior art keywords
- master node
- target
- request
- node
- cluster
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000000034 method Methods 0.000 title claims abstract description 62
- 230000009849 deactivation Effects 0.000 claims description 41
- 230000002779 inactivation Effects 0.000 claims description 28
- 238000001514 detection method Methods 0.000 claims description 24
- 238000004364 calculation method Methods 0.000 claims description 5
- 230000008569 process Effects 0.000 description 9
- 239000000523 sample Substances 0.000 description 8
- 238000010586 diagram Methods 0.000 description 7
- 230000009471 action Effects 0.000 description 3
- 230000005540 biological transmission Effects 0.000 description 2
- 238000004891 communication Methods 0.000 description 2
- 238000005516 engineering process Methods 0.000 description 1
- 230000003203 everyday effect Effects 0.000 description 1
- 230000006872 improvement Effects 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
Images
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L67/00—Network arrangements or protocols for supporting network services or applications
- H04L67/01—Protocols
- H04L67/10—Protocols in which an application is distributed across nodes in the network
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L67/00—Network arrangements or protocols for supporting network services or applications
- H04L67/50—Network services
- H04L67/56—Provisioning of proxy services
- H04L67/59—Providing operational support to end devices by off-loading in the network or by emulation, e.g. when they are unavailable
Landscapes
- Engineering & Computer Science (AREA)
- Computer Networks & Wireless Communication (AREA)
- Signal Processing (AREA)
- Computer And Data Communications (AREA)
Abstract
本发明实施例提供了一种集群访问方法及装置，应用于代理服务器，该方法首先接收目标客户端发送的用于访问目标集群的目标请求；从目标集群的各个主节点中，确定该目标请求对应的主节点为第一主节点；判断该第一主节点是否处于失活状态；若为是，从目标集群中除该第一主节点外的各个主节点中，确定第二主节点；向该第二主节点发送该目标请求，以使得该第二主节点响应该目标请求；若为为否，向该第一主节点发送该目标请求，以使得该第一主节点响应该目标请求。本发明实施例提供的方案中，代理服务器不会将目标请求发送给处于失效状态的第一主节点，避免集群中主节点失活时造成的代理服务器崩溃的问题。
Embodiments of the present invention provide a cluster access method and device, which are applied to a proxy server. The method first receives a target request sent by a target client for accessing a target cluster; and determines the target request from each master node of the target cluster. The corresponding master node is the first master node; determine whether the first master node is in an inactive state; if so, determine the second master node from each master node in the target cluster except the first master node; The second master node sends the target request, so that the second master node responds to the target request; if no, sends the target request to the first master node, so that the first master node responds to the target request. In the solution provided by the embodiment of the present invention, the proxy server will not send the target request to the first master node in an invalid state, so as to avoid the problem of the proxy server crashing caused by the failure of the master node in the cluster.
Description
Technical Field
The present invention relates to the field of computer technologies, and in particular, to a cluster access method and apparatus.
Background
A cluster is a group of mutually independent computers interconnected by a high-speed network, which form a group and are managed in a single system mode. When a client interacts with the cluster, the cluster is like an independent server, and provides uniform service for the outside. The cluster is composed of a plurality of nodes, wherein the nodes comprise a master node and slave nodes, for the cluster without a central master node, all the master nodes in the cluster have the same position in the whole cluster, and each master node can be responsible for processing a part of client requests.
The cluster access system can comprise a client, a proxy server and a cluster, and the process of accessing the cluster by the client can be as follows: the client sends a request to the proxy server, the proxy server firstly determines a main node corresponding to the request, then directly sends the request to the determined main node, and the main node responds to the request. For example, for a redis (an open-source Key-Value database) cluster, a corresponding relationship between each primary node and a hash Value may be recorded in a proxy server, and after receiving a request sent by a client, the proxy server may calculate and obtain the hash Value corresponding to the Key Value in the request, then determine a primary node corresponding to the request according to the corresponding relationship, and finally send the request to the determined primary node.
Although the above manner can be generally applied to implement cluster access, when the master node determined by the proxy server is an inactive master node, after the proxy server sends a request to the inactive master node, according to a TCP/IP Protocol (Transmission Control Protocol/Internet Protocol), the proxy server may receive Connection rejected (Connection rejected) information.
Disclosure of Invention
The embodiment of the invention aims to provide a cluster access method and a cluster access device so as to avoid the problem of crash of a proxy server caused by the inactivation of a main node in a cluster. The specific technical scheme is as follows:
to achieve the above object, in a first aspect, an embodiment of the present invention provides a cluster access method applied to a proxy server, where the method includes:
receiving a target request sent by a target client for accessing a target cluster;
determining a master node corresponding to the target request as a first master node from all master nodes of the target cluster;
judging whether the first main node is in an inactivated state or not;
if yes, determining a second main node from all main nodes except the first main node in the target cluster; sending the target request to the second host node to enable the second host node to respond to the target request;
if not, the target request is sent to the first main node, so that the first main node responds to the target request.
Preferably, the step of determining, from the master nodes of the target cluster, that the master node corresponding to the target request is the first master node includes:
calculating to obtain a target hash value corresponding to the target key value;
and determining a main node corresponding to the target hash value in a routing table as a first main node corresponding to the target request from all main nodes of the target cluster, wherein the routing table stores a corresponding relation between the hash value and the main node.
Preferably, after the step of sending the target request to the second master node, the method further comprises:
receiving request success information for the target request;
and under the condition that the main node which sends the request success information is the second main node, updating the main node which corresponds to the target hash value in the routing table to be the second main node.
Preferably, the step of determining whether the first master node is in an inactivated state includes:
and judging whether the first main node is in the inactivation state or not according to the inactivation list recorded with the main nodes in the inactivation state.
Preferably, the method further comprises:
receiving feedback information corresponding to the target request as first feedback information;
judging whether the first feedback information is connection rejection information or not;
if yes, recording a third main node into the inactivation list, wherein the third main node is: and the master node sends the first feedback information.
Preferably, the deactivation list is updated by:
sending state detection information to each main node in the target cluster;
receiving feedback information corresponding to each piece of state detection information as second feedback information;
and updating the main node corresponding to the second feedback information belonging to the connection rejection information into the inactivation list.
In a second aspect, an embodiment of the present invention provides a cluster access apparatus, which is applied to a proxy server, where the apparatus includes:
the first receiving module is used for receiving a target request which is sent by a target client and used for accessing a target cluster;
a determining module, configured to determine, from each master node of the target cluster, that a master node corresponding to the target request is a first master node;
the first judgment module is used for judging whether the first main node is in an inactivated state or not;
a first sending module, configured to determine, when a determination result of the first determining module is yes, a second master node from each master node in the target cluster, except the first master node; sending the target request to the second host node to enable the second host node to respond to the target request;
and a second sending module, configured to send the target request to the first host node if the determination result of the determining module is negative, so that the first host node responds to the target request.
Preferably, the target request includes a target key, and the determining module includes:
the calculation submodule is used for calculating and obtaining a target hash value corresponding to the target key value;
and the determining submodule is used for determining a main node corresponding to the target hash value in a routing table as a first main node corresponding to the target request from all main nodes of the target cluster, wherein the routing table stores the corresponding relation between the hash value and the main node.
Preferably, the apparatus further comprises:
a second receiving module, configured to receive request success information for the target request;
and a first updating module, configured to update the master node corresponding to the target hash value in the routing table as the second master node when the master node that sends the request success information is the second master node.
Preferably, the first determining module is specifically configured to:
and judging whether the first main node is in the inactivation state or not according to the inactivation list recorded with the main nodes in the inactivation state.
Preferably, the apparatus further comprises:
a third receiving module, configured to receive feedback information corresponding to the target request as the first feedback information;
the second judging module is used for judging whether the first feedback information is connection rejection information or not;
a recording module, configured to record a third master node in the deactivation list if the determination result of the second determining module is yes, where the third master node is: and the master node sends the first feedback information.
Preferably, the apparatus further comprises:
a third sending module, configured to send status detection information to each master node in the target cluster;
a fourth receiving module, configured to receive feedback information corresponding to each piece of state detection information as second feedback information;
and the second updating module is used for updating the main node corresponding to the second feedback information belonging to the connection rejection information into the inactivation list.
As can be seen from the above, in the solution provided in the embodiment of the present invention, the proxy server first receives a target request for accessing the target cluster, which is sent by the target client; determining a main node corresponding to the target request as a first main node from all main nodes of the target cluster; judging whether the first main node is in an inactivated state; if so, determining a second main node from all main nodes except the first main node in the target cluster; sending the target request to the second host node so that the second host node responds to the target request; if not, the target request is sent to the first main node, so that the first main node responds to the target request. In the solution provided in the embodiment of the present invention, before the proxy server sends the target request to the first master node, it may first determine whether the first master node is in a failure state, and if the first master node is in the failure state, the proxy server may not send the target request to the first master node, so as to avoid a problem of crash of the proxy server when the master node in the cluster is inactivated, and further, ensure stability of the cluster access system configured with the proxy server when the cluster master node fails.
Drawings
In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the embodiments or the prior art will be briefly described below, it is obvious that the drawings in the following description are only some embodiments of the present invention, and for those skilled in the art, other drawings can be obtained according to the drawings without creative efforts.
Fig. 1 is a schematic structural diagram of a cluster access system according to an embodiment of the present invention;
fig. 2 is a first flowchart of a cluster access method according to an embodiment of the present invention;
fig. 3 is a second flowchart of a cluster access method according to an embodiment of the present invention;
fig. 4 is a third flowchart illustrating a cluster access method according to an embodiment of the present invention;
fig. 5 is a fourth flowchart illustrating a cluster access method according to an embodiment of the present invention;
fig. 6 is a fifth flowchart illustrating a cluster access method according to an embodiment of the present invention;
fig. 7 is a schematic diagram of a first structure of a cluster access device according to an embodiment of the present invention;
fig. 8 is a schematic structural diagram of a second structure of a cluster access device according to an embodiment of the present invention;
fig. 9 is a schematic structural diagram of a cluster access apparatus according to an embodiment of the present invention;
fig. 10 is a schematic diagram of a fourth structure of a cluster access device according to an embodiment of the present invention;
fig. 11 is a schematic structural diagram of a fifth structure of a cluster access device according to an embodiment of the present invention.
Detailed Description
The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
First, technical terms related to the embodiments of the present invention will be briefly described below.
Clustering: as previously mentioned, a cluster is a group of mutually independent computers interconnected by a high-speed network, which form a group and are managed in a single system mode. The cluster is composed of a plurality of nodes, and the nodes comprise a master node and a slave node. For a cluster without a central master node, each master node in the cluster has the same status throughout the cluster, and each master node may be responsible for processing a portion of the client requests.
Taking a redis cluster as an example, a request command for a client to access the cluster necessarily includes a key (key) value, and a hash value corresponding to the key value can be obtained through a hash algorithm, in addition, a key space of the redis cluster is divided into 16384 hash slots (slots), where 16384 hash slots respectively correspond to hash values 0-16383, and a request corresponding to a master node (or referred to as a redis instance) in the cluster represents: and requesting the corresponding hash value to be in the hash value range corresponding to the main node.
It should be noted that, in order to ensure high availability of data, the above cluster without a central master node may be configured with a master-slave mode in the cluster, that is: the master node corresponds to at least one slave node, the master node provides data access service for a client of the cluster, the slave node pulls a data backup from the master node, and when the master node is down, one slave node is selected from the slave nodes corresponding to the master node to serve as a new master node, so that the cluster is not hung, namely, the whole cluster can still work normally under the condition.
As shown in fig. 1, a schematic structural diagram of a cluster access system according to an embodiment of the present invention is shown, where the cluster access system may include a client, a proxy server, and a cluster; the cluster comprises a main node 1, main nodes 2, … and a main node N, and the client is in communication connection with each main node in the target cluster through the proxy server. It should be noted that, in fig. 1, the connection relationship of the cluster access system is exemplarily represented by only one client, in practical applications, the number of clients corresponding to the cluster is not limited, and may be thousands of clients, and each client is in communication connection with the proxy server.
The present invention will be described in detail with reference to the following examples.
Fig. 2 is a first flowchart of a cluster access method according to an embodiment of the present invention.
An embodiment of the present invention provides a cluster access method applied to a proxy server, and as shown in fig. 2, the method includes:
s101: and receiving a target request sent by the target client for accessing the target cluster.
It is understood that in the cluster access system configured with the proxy server, the request sent by the client is directed to the proxy server, so in the embodiment of the present invention, the target client sends the target request for accessing the target cluster to the proxy server, and then the target request is received by the proxy server.
S102: and determining the master node corresponding to the target request as a first master node from all master nodes of the target cluster.
As described above, each master node may be responsible for processing a portion of the client's requests, so the proxy server may determine the master node to which the target request corresponds based on the correspondence of the master node to the different types of requests.
In this embodiment of the present invention, the step of determining, in each master node of the slave target cluster, that the master node corresponding to the target request is the first master node (S102), where the target request includes a target key value, for example, a key value is carried in a request sent by a client of the redis cluster, as shown in fig. 3, may include:
s1021: and calculating to obtain a target hash value corresponding to the target key value.
It should be noted that the target request sent by the client will include the key value. In addition, as known to those skilled in the art, the calculation formula of the hash value may be:
HASH_SLOT＝CRC16(key)mod 16384；
in the formula, HASH _ SLOT represents a HASH value, CRC16() represents an existing CRC16 algorithm, a specific algorithm process is not described in detail in the embodiment of the present invention, key represents the target key value, and mod represents a modulo operation. It can be understood that the result obtained by the calculation of the formula is necessarily an integer within 0 to 16383, that is, the value range of the hash value is 0 to 16383.
For example, if the target request for accessing the redis cluster by the target client is "set a 21", the proxy server may parse the target request "set a 21" to obtain a target key value "a 2", and then calculate a hash value corresponding to the target key value "a 2":
CRC16(a2)mod 16384＝x。
s1022: and determining a main node corresponding to the target hash value in a routing table as a first main node corresponding to the target request from all main nodes of the target cluster, wherein the routing table stores the corresponding relation between the hash value and the main node.
As known to those skilled in the art, the key space of a cluster is divided into 16384 hash slots (slots), each master node in the cluster is responsible for processing a part of the 16384 hash slots, and 16384 hash slots correspond to 16384 hash values 0-16383, i.e., hash slots 1-16384 correspond to hash values 0-16383, respectively.
It should be noted that, in practical applications, the correspondence stored in the routing table described herein may be an identification correspondence between a hash value and a master node. It is understood that the identifier is information that can uniquely correspond to the host node, for example, a common identifier may be composed of an IP (Internet Protocol, Protocol for interconnecting networks) address and a TCP (Transmission Control Protocol) port number used by the host node; such as the label "127.0.0.1: 8001", where "127.0.0.1" denotes the IP address used by the primary node and "8001" denotes the port number used by the primary node.
For example, assuming that the target cluster includes master nodes a, b, c, d, e and f, the routing table records:
the hash values 0-2730 correspond to the identification of the main node a, the hash values 2731-5460 correspond to the identification of the main node b, the hash values 5461-8190 correspond to the identification of the main node c, the hash values 8191-10920 correspond to the identification of the main node d, the hash values 10921-13650 correspond to the identification of the main node e, and the hash values 13700-16383 correspond to the identification of the main node f.
Assuming that the proxy server parses a target key value X from a currently received target request, and then calculates a hash value corresponding to the target key value X by the above formula, that is:
CRC16(X)mod 16384＝5113 mod 16384＝5113；
since the routing table records: the identity of the master node corresponding to the hash value 5113 points to the master node b, so that the master node b is determined to be the first master node at this time.
Assuming that the proxy server parses a target key value Y from a currently received target request, and then calculates a hash value corresponding to the target key value Y by the above formula, that is:
CRC16(Y)mod 16384＝30066 mod 16384＝13682；
since the identifier of the master node corresponding to the hash value 13682 is not recorded in the routing table, it is obvious that the proxy server cannot determine the first master node corresponding to the target request through the routing table, and the proxy server may obtain the first master node corresponding to the target request through other technical means.
S103: and judging whether the first main node is in an inactivated state or not.
The inactive state may also be referred to as a failure state, where the master node in the inactive state no longer provides a service to the client, and when sending a request to the master node in the inactive state, the object sending the request may receive Connection rejected (Connection rejected) information according to a TCP/IP protocol.
It should be noted that the reason for the failure of the master node may be: some main nodes are shut down artificially to reduce the number of the main nodes, and the main nodes may also be down due to faults, so that the main nodes enter an inactive state.
There are many specific implementation methods for determining whether the first master node is in the inactivated state (S103), and those skilled in the art can determine which implementation method to apply according to the prior art.
In an embodiment of the present invention, the step of determining whether the first master node is in the inactivated state (S103) may include:
and judging whether the first main node is in the inactivation state or not according to the inactivation list recorded with the main nodes in the inactivation state.
In practical applications, the master nodes in the inactivated state are recorded in the inactivated list, for example, if the identities "127.0.0.1: 8001" and "127.0.0.1: 8003" are recorded in the deactivation list, it indicates that both the master nodes corresponding to the identities "127.0.0.1: 8001" and "127.0.0.1: 8003" are in the inactivated state.
If the judgment result of the step S103 is yes, the step S104 is executed: determining a second main node from all main nodes except the first main node in the target cluster; sending the target request to the second host node so that the second host node responds to the target request.
In general, when the determined first master node is in the inactive state, the client may randomly determine one master node as the second master node from the master nodes in the target cluster except the first master node. For example, the target cluster includes master nodes a, b, c, d, e, and f, the proxy server starts to determine the master node b as the first master node, and then the proxy server finds that the master node b is in the inactive state, at this time, the proxy server randomly selects the master node d from the master nodes a, c, d, e, and f as the second master node, and further sends the target request to the master node d.
It will be appreciated that the randomly determined secondary master node may not be the correct master node currently corresponding to the target request.
If the randomly determined second master node is the correct master node currently corresponding to the target request, at this time, the second master node responds to the target request, which may be understood as: the second main node firstly executes a request command corresponding to the target request and then feeds back request success information to the proxy server.
If the randomly determined second master node is not the correct master node currently corresponding to the target request, the second master node responds to the target request, and it can be understood that the second master node feeds back the steering information to the proxy server.
It is well known to those skilled in the art that the steering information, also referred to as steering error information, includes a hash value corresponding to the target request and an identification of the host node responsible for processing the hash value, such as an IP address and a TCP port number used by the host node.
In the existing cluster, each master node shares the association information of each master node, and the association information includes: the host nodes are respectively responsible for processing the hash grooves. When a host node x is not a host node responsible for processing the target request, after receiving the target request, the host node x first calculates a hash value corresponding to a target key value carried in the target request, and then determines the host node responsible for processing the hash value according to the association information; finally, the master node x returns the turn information to the proxy server, where the turn information includes the identifier of the master node responsible for processing the hash value and the hash value obtained by calculation.
It should be noted that the steering information may include move information and ASK information, different types of steering information, and the access logic of the proxy server is different, and the proxy server first determines the type of the steering information after receiving the steering information fed back by the second master node; if the steering information is MOVED information, the proxy server directly sends the target request to a main node pointed by the MOVED information; if the turn information is ASK information, the proxy server firstly sends an ASKING command to the main node pointed by the ASK information, and then sends the target request to the main node pointed by the ASK information.
It should be noted that the method adopted for determining the type of the turn-around information belongs to the prior art, so the proxy server can determine the type of the turn-around information through the prior art; in addition, the move information and the ASK information are concepts in the prior art, and the difference between the move information and the ASK information can be referred to the prior art, and the embodiment of the present invention is not described in detail herein.
In addition, it should be noted that, on the basis of the embodiment of the method shown in fig. 3, as shown in fig. 4, after the step of sending the target request to the second host node, the method may further include:
s106: receiving request success information for the target request;
s107: and under the condition that the main node which sends the request success information is the second main node, updating the main node which corresponds to the target hash value in the routing table to be the second main node.
It can be understood that the master node corresponding to the target hash value originally recorded in the routing table is the first master node, and after receiving the target request, the current second master node feeds back request success information to the proxy server, indicating that the correct master node corresponding to the target hash value is the second master node. In order to enable all the subsequent requests corresponding to the hash value to directly determine the corresponding correct master node from the routing table, the master node corresponding to the target hash value in the routing table needs to be updated to be the second master node.
For example, the routing table records a corresponding relationship between a hash value x and an identifier Y, and it can be understood that the proxy server first determines the master node Y corresponding to the identifier Y as a first master node, then the proxy server finds that the master node Y is in an inactivated state, then the proxy server sends a target request to another master node Z in the target cluster, and the master node Z feeds back request success information to the proxy server, at this time, the proxy server records the corresponding relationship between the hash value x and the identifier Y in the routing table, and replaces the hash value x with the corresponding relationship between the hash value x and the identifier Z of the master node Z.
If the judgment result of the step S103 is negative, executing the step S105: the target request is sent to the first master node so that the first master node responds to the target request.
It will be appreciated that if the primary master node is not currently in a failed state, the proxy server may now send the target request directly to the primary master node.
It should be noted that the master nodes recorded in the deactivation list need to be updated in time, because the number of hash slots handled by one master node in the cluster is very large, i.e. it indicates that the number of hash values corresponding to one master node is very large.
For example: the hash values 0-2730 correspond to the identification of the main node a, the hash values 2731-5460 correspond to the identification of the main node b, the hash values 5461-8190 correspond to the identification of the main node c, the hash values 8191-10920 correspond to the identification of the main node d, the hash values 10921-13650 correspond to the identification of the main node e, and the hash values 13700-16383 correspond to the identification of the main node f.
And because the number of the clients is huge, the proxy server receives a large number of requests for visiting the cluster in the actual operation process, and a large number of requests are directed to the same main node.
Therefore, if a certain master node enters a failure state, the proxy server will inevitably receive a large amount of connection rejection information because the relationship between the master node and the hash value has not been updated in time in the routing table, or a large amount of requests are sent to the failed master node. It is well known to those skilled in the art that if too many connection rejection messages are received by a proxy server, it will inevitably cause a crash of the proxy server, and further cause a crash of the cluster access system including the proxy server.
In an embodiment of the present invention, two methods for automatically updating the deactivation list are provided, and in the first method for automatically updating the deactivation list, the step of determining whether the first master node is in the deactivation state (S103) is: according to the inactivation list recorded with the master nodes in the inactivated state, judging whether the first master node is in the inactivated state, and based on any one of the method embodiments in fig. 2 to 4, as shown in fig. 5, the method may further include:
s108: and receiving feedback information corresponding to the target request as first feedback information.
It should be noted that the feedback information received by the proxy server includes: after the target request is sent to the first main node, the feedback information received by the proxy server is received; and after sending the target request to the second host node, the proxy server receives the feedback information.
S109: and judging whether the first feedback information is connection rejection information.
If the determination result in step S109 is yes, step S1010 is executed: recording a third master node into the deactivation list, wherein the third master node is: and the master node sends the first feedback information.
It will be appreciated that if the proxy server receives the connection rejection information immediately after sending the target request to a host node, indicating that the host node is in an inactive state at that time, then the host node may need to be recorded in an inactive list at that time.
For example, the proxy server determines that the primary master node is the node identified as "129.0.0.7: 6001 ", the proxy server queries the current deactivation list for any deactivation indicated by" 129.0.0.7: 6001 ", the proxy server sends a target request to the master node a, but, the proxy server immediately receives the connection rejection information, which indicates that the master node a is in the inactivated state at the present time, and records the master node a into the inactivation list, that is, the inactivation list may record the identifier" 129.0.0.7: 6001".
For another example, the first master node determined by the proxy server is: identified as "129.0.0.7: 6002 "the proxy server queries the current deactivation list for the deactivation message indicated as" 129.0.0.7: 6002 ", and then the proxy server re-determines that the master node b, identified as" 129.0.0.7: 6005 "is the second master node, and sends a target request to the master node b, but the proxy server immediately receives the connection rejection information, which indicates that the master node b is also in the inactivated state at the present time, and then records the master node b in the inactivation list, that is, records the identifier" 129.0.0.7: 6005".
In the first method for automatically updating the deactivation list, steps S108 to S1010 relate to how to add a newly deactivated master node to the deactivation list, but in the actual application process, since some master nodes may be reactivated after being deactivated and become a master node that normally works, the record about the reactivated master node should be removed from the deactivation list in time.
In a second method for automatically updating the deactivation list, based on any one of the method embodiments shown in fig. 2 to 4, as shown in fig. 6, the deactivation list is updated as follows:
s201: and sending the state detection information to each master node in the target cluster.
It should be noted that, unlike the above-mentioned target request for accessing the target cluster, the state detection information represents only one type of information for detecting the state of the master node.
In order to ensure that whether each master node is in a failure state is detected in time, in the embodiment of the present invention, the proxy server may actively send the state detection information to each master node in the target cluster according to a certain time interval, or may actively send the state detection information to each master node in the target cluster once the proxy server receives a certain number of requests.
For example, with 12 hours as a time period, sending status probe information to each master node in the target cluster at the end of each period; for example, proxy servers 2: 00 and 14: 00 will actively send status probe information to each master node in the target cluster.
For another example, the proxy server is provided with a module for counting the number of access requests received by the proxy server, and the proxy server actively sends state detection information to each master node in the target cluster once every 5000 requests for accessing the target cluster are received.
S202: receiving feedback information corresponding to each piece of state detection information as second feedback information.
In one case, since the master node that receives the state detection information does not recognize the state detection information and cannot execute the related command, the master node may feed back feedback information that the request cannot be executed, that is, the second feedback information is feedback information indicating that the request cannot be executed; alternatively, if the status probe information is sent to a primary node, and the primary node is in an inactive state, the second feedback information is obviously the connection rejection information.
Of course, whether the specific second feedback information is the connection rejection information may be determined by the prior art, and the embodiment of the present invention is not described in detail herein.
S203: and updating the main node corresponding to the second feedback information belonging to the connection rejection information into the inactivation list.
It should be noted that, since step S101 is to send status probe information to each master node in the target cluster, in the current deactivation list, only all the master nodes corresponding to the connection rejection information that are currently determined need to replace the master node that is originally recorded in the deactivation list.
For example, the master nodes recorded in the deactivation list are represented by the identities of the master nodes, the master node identities "127.0.0.1: 8001" and "127.0.0.1: 8003" are recorded in the deactivation list, and after steps S201 and S202 are executed, the proxy server finds that: after status probe information is sent to two master nodes labeled "127.0.0.1: 8002" and "127.0.0.1: 8004", connection rejection information for the status probe information is received, and after status probe information is sent to other master nodes in the target cluster, connection rejection information for the status probe information is not received.
Then, at this time, the proxy server may directly replace the identifiers "127.0.0.1: 8001" and "127.0.0.1: 8003" recorded in the deactivation list with the identifiers "127.0.0.1: 8002" and "127.0.0.1: 8004" in the deactivation list, that is, after step S103 is executed, the identifiers recorded in the deactivation list only exist: labels "127.0.0.1: 8002" and "127.0.0.1: 8004".
For the above steps S201 to S203, for example, it is assumed that the target cluster includes the master nodes a to g, the execution subjects of S201 to S203 are proxy servers, and the proxy servers actively send status detection information to each master node in the target cluster at 8:00, 16:00, and 24:00 every day.
At 16:00 of a certain day, the proxy server actively sends the state detection information to each master node in the target cluster, and as a result, only after sending the state detection information to the master node b, the proxy server receives the connection rejection information aiming at the state detection information, and after sending the state detection information to other master nodes in the target cluster, the proxy server does not receive the connection rejection information aiming at the state detection information.
The proxy server will then update the deactivation list so that only the identity of the master node b is recorded in the deactivation list.
It should be noted that steps S201 to S203 may be preferably executed by a proxy server, but steps S201 to S203 may also be executed by another object, for example, a server dedicated to updating the deactivation list is provided.
Of course, in the embodiment of the present invention, besides the two methods for automatically updating the deactivation list, other methods for automatically updating the deactivation list may also be used, and those skilled in the art may determine an appropriate method for automatically updating the deactivation list according to actual situations in practical applications.
Obviously, after the method for automatically updating the deactivation list is adopted, if a certain main node enters a failure state, although the relationship between the main node and the hash value is not updated in time in the routing table, the main node recorded in the deactivation list is updated in time, so that the proxy server can not send a request to the newly aged main node, the crash of the proxy server is avoided, and the stability of a cluster access system is ensured.
As can be seen from the above, in the scheme provided in this embodiment, the proxy server first receives a target request for accessing the target cluster, which is sent by the target client; determining a main node corresponding to the target request as a first main node from all main nodes of the target cluster; judging whether the first main node is in an inactivated state; if so, determining a second main node from all main nodes except the first main node in the target cluster; sending the target request to the second host node so that the second host node responds to the target request; if not, the target request is sent to the first main node, so that the first main node responds to the target request. In the solution provided in this embodiment, before sending the target request to the first master node, the proxy server may first determine whether the first master node is in a failure state, and if the first master node is in the failure state, the proxy server may not send the target request to the first master node, thereby avoiding a problem of crash of the proxy server caused when the master node in the cluster is inactivated, and further, ensuring stability of the cluster access system configured with the proxy server in the case of failure of the cluster master node.
It is to be understood that the names of the first master node and the second master node determined by the proxy server in the foregoing embodiments are only given for convenience of describing specific operations, and in practical applications, any master node in the cluster may be determined as the first master node or the second master node according to an actual target request.
Similarly, the target client and the target request in the foregoing embodiments are also names given for convenience of explanation of specific operations; in practical applications, any client corresponding to the cluster may be the target client, and any request sent by the client to access the cluster may be the target cluster.
In other embodiments of the present application, there are also definitions similar to the foregoing for convenience of describing specific operations, which are for convenience of description and do not substantially limit what function a certain device or network has. There is no longer a single row.
The following is a brief description of an embodiment of the present invention by way of a specific example.
An existing redis cluster includes host nodes a to g, and a routing table in a proxy server records:
the hash values 0-2340 correspond to the identification of the main node a, the hash values 2341-4680 correspond to the identification of the main node b, the hash values 4681-7020 correspond to the identification of the main node c, the hash values 7021-9360 correspond to the identification of the main node d, the hash values 9361-11700 correspond to the identification of the main node e, the hash values 11701-14040 correspond to the identification of the main node f, and the hash values 14041-16383 correspond to the identification of the main node g.
In addition, the inactivation list of the proxy server records the identification of the master node g.
Client m sends request X and request Y to the proxy server for access to the redis cluster.
For the request X, the proxy server analyzes a key value X carried in the request X, then calculates a hash value corresponding to the key value X to be 8008 through a hash algorithm, determines that the master node d is the master node corresponding to the request X according to the routing table, and then judges that the master node d is not in an inactivated state according to the inactivation list, so that the proxy server directly sends the request X to the master node d, and the master node d processes a request instruction corresponding to the request X and feeds back request success information to the proxy server.
For the request Y, the proxy server analyzes the key value Y carried in the request Y, then calculates a hash value 13228 corresponding to the key value Y through a hash algorithm, at the moment, the proxy server determines the main node f as the main node corresponding to the request Y according to the routing table, and then the proxy server judges that the main node f is not in the inactivation state according to the inactivation list.
After sending the request Y to the master node f, the proxy server receives connection rejection information for the master node f, that is, the proxy server indicates that the master node f is actually in an inactivated state at the current time.
Then the proxy server randomly selects a master node a from the master nodes a-e, then the proxy server sends a request Y to the master node a, the master node a analyzes a key value Y carried in the request Y, then a hash value corresponding to the key value Y is calculated to be 13228 through a hash algorithm, the master node a finds that the current hash value 13228 corresponds to the master node c, the master node a feeds back MOVED steering information to the proxy server, and the MOVED steering information contains the identification of the master node c.
And then the proxy server analyzes and obtains the identifier of the main node c contained in the MOVED steering information, sends the request Y to the main node c, and the main node c processes the request instruction corresponding to the request Y and feeds back the request success information to the proxy server.
In addition, in the routing table, the corresponding relationship between the updated hash value 13228 and the identifier of the master node f is as follows: the correspondence of the hash value 13228 to the identity of the master node c; and the proxy server will also add the identity of the master node f to the deactivation list.
Corresponding to the embodiment of the method shown in fig. 2, as shown in fig. 7, an embodiment of the present invention further provides a cluster access apparatus, which is applied to a proxy server, where the apparatus includes:
a first receiving module 110, configured to receive a target request sent by a target client for accessing a target cluster;
a determining module 120, configured to determine, from the master nodes of the target cluster, that a master node corresponding to the target request is a first master node;
a first determining module 130, configured to determine whether the first master node is in an inactive state;
a first sending module 140, configured to determine a second master node from each master node in the target cluster except the first master node if the determination result of the first determining module 130 is yes; sending the target request to the second host node to enable the second host node to respond to the target request;
a second sending module 150, configured to send the target request to the first host node if the determination result of the determining module 130 is negative, so that the first host node responds to the target request.
In an embodiment of the present invention, the target request includes a target key, and as shown in fig. 8, the determining module 120 may include:
a calculating submodule 1201, configured to calculate and obtain a target hash value corresponding to the target key value;
the determining sub-module 1202 is configured to determine, from the master nodes of the target cluster, that a master node corresponding to the target hash value in a routing table is a first master node corresponding to the target request, where the routing table stores a correspondence between hash values and master nodes.
Corresponding to the method embodiment shown in fig. 4, as shown in fig. 9, the apparatus may further include:
a second receiving module 160, configured to receive request success information for the target request;
a first updating module 170, configured to update the master node corresponding to the target hash value in the routing table as the second master node when the master node that sends the request success information is the second master node.
Specifically, in practical applications, in any one of the cluster access devices shown in fig. 7 to 9, the first determining module 130 may be specifically configured to:
and judging whether the first main node is in the inactivation state or not according to the inactivation list recorded with the main nodes in the inactivation state.
In the case of configuring the deactivation list, corresponding to the method embodiment shown in fig. 5, as shown in fig. 10, the apparatus may further include:
a third receiving module 180, configured to receive feedback information corresponding to the target request as the first feedback information;
a second determining module 190, configured to determine whether the first feedback information is connection rejection information;
a recording module 1100, configured to record a third master node into the deactivation list if the determination result of the second determining module 190 is yes, where the third master node is: and the master node sends the first feedback information.
In the case of configuring the deactivation list, corresponding to the method embodiment shown in fig. 6, as shown in fig. 11, the apparatus may further include:
a third sending module 210, configured to send status detection information to each master node in the target cluster;
a fourth receiving module 220, configured to receive feedback information corresponding to each piece of state detection information as second feedback information;
a second updating module 230, configured to update the master node corresponding to the second feedback information belonging to the connection rejection information into the inactivation list.
It should be noted that, in the method embodiment, the execution subject of steps S201 to S203 is preferably a proxy server, but is not limited to the proxy server. In this embodiment of the apparatus, the third sending module 210, the fourth receiving module 220 and the second updating module 230 respectively corresponding to the steps S201 to S203 are obviously disposed in the proxy server.
As can be seen from the above, in the scheme provided in this embodiment, the proxy server first receives a target request for accessing the target cluster, which is sent by the target client; determining a main node corresponding to the target request as a first main node from all main nodes of the target cluster; judging whether the first main node is in an inactivated state; if so, determining a second main node from all main nodes except the first main node in the target cluster; sending the target request to the second host node so that the second host node responds to the target request; if not, the target request is sent to the first main node, so that the first main node responds to the target request. In the solution provided in this embodiment, before sending the target request to the first master node, the proxy server may first determine whether the first master node is in a failure state, and if the first master node is in the failure state, the proxy server may not send the target request to the first master node, thereby avoiding a problem of crash of the proxy server caused when the master node in the cluster is inactivated, and further, ensuring stability of the cluster access system configured with the proxy server in the case of failure of the cluster master node.
It is noted that, herein, relational terms such as first and second, and the like may be used solely to distinguish one entity or action from another entity or action without necessarily requiring or implying any actual such relationship or order between such entities or actions. Also, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising an … …" does not exclude the presence of other identical elements in a process, method, article, or apparatus that comprises the element.
All the embodiments in the present specification are described in a related manner, and the same and similar parts among the embodiments may be referred to each other, and each embodiment focuses on the differences from the other embodiments. In particular, for the system embodiment, since it is substantially similar to the method embodiment, the description is simple, and for the relevant points, reference may be made to the partial description of the method embodiment.
The above description is only for the preferred embodiment of the present invention, and is not intended to limit the scope of the present invention. Any modification, equivalent replacement, or improvement made within the spirit and principle of the present invention shall fall within the protection scope of the present invention.
Claims (12)
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201611170510.9A CN108206843B (en) | 2016-12-16 | 2016-12-16 | Cluster access method and device |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201611170510.9A CN108206843B (en) | 2016-12-16 | 2016-12-16 | Cluster access method and device |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN108206843A CN108206843A (en) | 2018-06-26 |
| CN108206843B true CN108206843B (en) | 2021-06-04 |
Family
ID=62602971
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201611170510.9A Active CN108206843B (en) | 2016-12-16 | 2016-12-16 | Cluster access method and device |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN108206843B (en) |
Families Citing this family (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110224871B (en) * | 2019-06-21 | 2022-11-08 | 深圳前海微众银行股份有限公司 | High-availability method and device for Redis cluster |
| CN111104266A (en) * | 2019-12-23 | 2020-05-05 | 北京大米科技有限公司 | Access resource allocation method and device, storage medium and electronic equipment |
| CN113472823B (en) * | 2020-03-30 | 2023-06-27 | 深圳Tcl数字技术有限公司 | Server access method and device, intelligent terminal and storage medium |
| CN113760901A (en) * | 2021-02-19 | 2021-12-07 | 北京沃东天骏信息技术有限公司 | Data processing method, device, equipment and storage medium |
| CN113794764A (en) * | 2021-09-10 | 2021-12-14 | 小红书科技有限公司 | Request processing method and medium for server cluster and electronic device |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102591970A (en) * | 2011-12-31 | 2012-07-18 | 北京奇虎科技有限公司 | Distributed key-value query method and query engine system |
| CN103747073A (en) * | 2013-12-30 | 2014-04-23 | 乐视网信息技术（北京）股份有限公司 | Distributed caching method and system |
| US9317548B2 (en) * | 2013-01-30 | 2016-04-19 | International Business Machines Corporation | Reducing collisions within a hash table |
| CN105933407A (en) * | 2016-04-20 | 2016-09-07 | 中国银联股份有限公司 | Method and system for achieving high availability of Redis cluster |
Family Cites Families (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US7734648B2 (en) * | 2006-04-11 | 2010-06-08 | Sap Ag | Update manager for database system |
-
2016
- 2016-12-16 CN CN201611170510.9A patent/CN108206843B/en active Active
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102591970A (en) * | 2011-12-31 | 2012-07-18 | 北京奇虎科技有限公司 | Distributed key-value query method and query engine system |
| US9317548B2 (en) * | 2013-01-30 | 2016-04-19 | International Business Machines Corporation | Reducing collisions within a hash table |
| CN103747073A (en) * | 2013-12-30 | 2014-04-23 | 乐视网信息技术（北京）股份有限公司 | Distributed caching method and system |
| CN105933407A (en) * | 2016-04-20 | 2016-09-07 | 中国银联股份有限公司 | Method and system for achieving high availability of Redis cluster |
Also Published As
| Publication number | Publication date |
|---|---|
| CN108206843A (en) | 2018-06-26 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN108206843B (en) | Cluster access method and device | |
| CN108206779B (en) | A cluster access system, method and device | |
| US11706102B2 (en) | Dynamically deployable self configuring distributed network management system | |
| CN109729111B (en) | Method, apparatus and computer program product for managing distributed systems | |
| CN103460203B (en) | cluster unique identifier | |
| US6973473B1 (en) | Method, system and program products for managing identifiers of components of a clustered environment | |
| CN111259072B (en) | Data synchronization method, device, electronic equipment and computer readable storage medium | |
| US20140059315A1 (en) | Computer system, data management method and data management program | |
| US9729560B2 (en) | Method and device for synchronizing network data flow detection status | |
| US7356531B1 (en) | Network file system record lock recovery in a highly available environment | |
| CN109101196A (en) | Host node switching method, device, electronic equipment and computer storage medium | |
| WO2016197346A1 (en) | Method and apparatus for managing life circle of virtualized network function | |
| CN111901705A (en) | OMCI function virtualization system of OLT equipment | |
| JP5617304B2 (en) | Switching device, information processing device, and fault notification control program | |
| CN115086330A (en) | Cross-cluster load balancing system | |
| CN111708668B (en) | Cluster fault processing method and device and electronic equipment | |
| WO2020010906A1 (en) | Method and device for operating system (os) batch installation, and network device | |
| EP3570169B1 (en) | Method and system for processing device failure | |
| CN113064732B (en) | Distributed system and management method thereof | |
| US9866651B2 (en) | Entity creation for constrained devices | |
| CN109947451B (en) | Cluster application file updating method, system, medium and equipment | |
| CN118381739A (en) | Positive resource response to pending resources | |
| CN104618414A (en) | Implementation method of distributed service, service agent device and distributed system | |
| CN107493308B (en) | Method and device for sending message and distributed equipment cluster system | |
| JP6644902B2 (en) | Neighbor monitoring in a hyperscale environment |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |