CN104038569A - Trunking communication model based on address mapping - Google Patents

Abstract

The invention provides a trunking communication model based on address mapping. The trunking communication model is composed of a plurality of mainframe systems and a plurality of communication modules, wherein the plurality of mainframe systems respectively include a memory address mapping device and a communication module based on memory address mapping, and the memory address mapping device is used for performing memory address mapping among different mainframe systems; compared with the prior art, the trunking communication model based on address mapping achieves RDMA based on memory address mapping, is packaged according to a special communication protocol, directly achieves package forwarding at the communication modules, achieves zero-copy data transmission, fully utilizes physical transmission links among computer mainframes, and achieves improved transmission performance and reduced system losses; the trunking communication model based on address mapping achieves memory address mapping and data transmission based on NTB, and transmits data by using a PCI-E2.0 protocol, thus having good popularization and application values.

Description

A Cluster Communication Model Based on Address Mapping

technical field

The invention relates to the field of computer cluster system and storage, in particular to a cluster communication model based on address mapping.

Background technique

A computer cluster is a computer system that is connected by a group of loosely integrated computer software and/or hardware to perform computing tasks in a highly co-operative manner. In a sense, they can be seen as a computer. The individual computers in a cluster system are usually called nodes and are usually connected by a local area network, but there are other possible connections. Cluster computers are often used to improve the computing speed and/or reliability of individual computers. In general, cluster computers are much more cost-effective than individual computers, such as workstations or supercomputers.

The starting point of computer cluster technology is to provide computer systems with higher availability, manageability, and scalability. A cluster includes multiple servers with shared data storage space, and each server communicates with each other through an internal LAN. When a node fails, the applications it was running are automatically taken over by other nodes. In most modes, all nodes in the cluster have a common name, and services running on any node in the cluster can be used by all network clients.

A large amount of data interaction is required between the various nodes in the cluster. Usually, the cluster system relies on high-speed LAN to realize. The common high-speed LAN includes FDDI optical fiber ring network, 100BASE-T high-speed Ethernet, Gigabit Ethernet, 10Gbit/s Ethernet, etc. Transmission media include optical fiber, Ethernet, etc. The traditional iscsi and TCP/IP protocols are used, and the protocol encapsulation is more complicated. When transmitting data, a large number of protocol packet conversion, encapsulation, codec, verification, and other steps are required. These protocols are more suitable for transmission in long-distance and unreliable environments. data.

In a large-scale cluster system, more hosts are transmitted in a short-distance, reliable, and stable transmission environment. High-bandwidth, high-reliability, and large-capacity data transmission links are required. When using traditional high-speed transmission protocols, the computer host needs to consume a lot of calculations for data protocol packet encapsulation, encoding and decoding, and verification, and the communication link bandwidth and node computing capabilities will become the bottleneck of the overall response and performance of the entire computer cluster system.

Contents of the invention

The purpose of the present invention is to overcome the deficiencies in the prior art and provide a cluster communication model based on address mapping, which can meet the requirements of high-speed, high-reliability, and large-capacity data transmission between nodes of a large-scale cluster system.

The technical solution of the present invention is realized in the following manner, and its structure is composed of multiple host systems and multiple communication modules, and the multiple host systems all include memory address mapping devices and communication modules based on memory address mapping;

The memory address mapping device is used for memory address mapping between different host systems;

The communication model is used for data communication between different host systems, and realizes a point-to-point data transmission model. The communication module analyzes the address information of the data packet, and directly transmits it to the destination host system without being forwarded by switches and other equipment, thereby realizing high bandwidth and low cost. The delayed data transmission model is suitable for large-scale data transmission between cluster systems.

The multiple host systems are connected through a memory address mapping device, such as NTB;

The communication model is used for communication requirements between computer hosts, and provides a unified interface for computer system applications. The communication module is based on the address mapping device between computers, and reads and writes data by accessing the mapped memory, thereby completing the data communication.

The memory address mapping device is used to connect multiple host systems to each other, and each host performs memory address mapping with all other hosts. The locations of the hosts are equal, and all hosts have complete peer-to-peer access.

The communication module mentioned also includes special communication protocol encapsulation, based on an effective, reliable, and concise communication link between hosts, reduces the loss of data communication in protocol processing, improves the effective data duty cycle, and improves data communication efficiency.

Each of the hosts is connected through an interconnection bus, and each of the hosts includes a heartbeat module.

The heartbeat module is configured to reset the power supply of the peer storage controller when detecting that the peer storage controller is in a fault state through the interconnection bus.

The advantages of the present invention are:

Compared with the prior art, a cluster communication model based on address mapping in the present invention realizes RDMA based on memory address mapping, encapsulates according to a special communication protocol, directly implements packet forwarding in the communication module, realizes zero-copy data transmission, and makes full use of The physical transmission link between computer hosts is improved, the transmission performance is improved, and the system loss is reduced; the present invention realizes memory address mapping based on NTB, and realizes data transmission; uses PCI-E 2.0 protocol for data transmission, thus, has good promotion use value.

Description of drawings

FIG. 1 is a schematic structural diagram of a cluster communication model based on address mapping.

Implementation

A cluster communication model based on address mapping of the present invention will be described in detail below in conjunction with the accompanying drawings.

As shown in FIG. 1 , a high-speed communication model in the structure of a cluster communication model based on address mapping in the present invention is shown in FIG. 1 , and different nodes in the cluster are connected to each other through a memory address mapping device. Each host opens up a section of space in its own memory as the memory access area of other nodes. For sending and receiving data.

The memory address mapping device (such as NTB, the following chapters of this article uses NTB temporary memory address mapping device), can realize memory access between different host systems, and realize the purpose of communication through interrupt registers and temporary registers. Through address translation technology, a section of memory in the local machine is mapped to the MMIO area of the remote NTB. Copying data from the remote host to MMIO is equivalent to directly copying data to the corresponding memory area on the local end. So that the data is sent. Combined with the doorbell interrupt register and temporary storage register, the process of interrupt notification and configuration interaction is realized. In this way, data transmission between different host systems can be realized.

As shown in Figure 1, all nodes in the cluster are mapped to a memory space through NTB (the mapping window is generally applied for less than 1MB). Forms an accessible memory window that can be used to receive data. Coupled with the MMIO area of NTB, the sending and receiving buffer areas are formed.

During the system startup process, when enumerating PCI devices, apply for different memory spaces according to the NTB mapping intervals of different hosts, and establish a one-to-one correspondence for query. Establish different interrupt processing functions for NTB devices of different nodes. Determine which node it belongs to according to the interrupt offset, and add it to the hash table.

At the end of initialization, a LINK UP signal is sent to all registered NTB device nodes, and then the status of other nodes is judged according to the LINK status register. When the corresponding node also sends a LINK UP signal, the connection establishment process is started. Data interaction is performed through scratch registers and a connection is established.

After the initialization is completed, the node can transmit data with other nodes through the communication module.

After the host node application submits the transmission request, the data packet is split and encapsulated according to the MTU, and the header is added. According to the node serial number, the hash table is searched and copied to the MMIO of the corresponding node. Then set the sending completion flag to 1, and finally trigger the corresponding interrupt to notify the remote node to receive the data.

When the remote node interrupts the response, wake up the processing thread corresponding to the memory mapping window. First judge whether the sending completion flag is set to 1. If it is set to 1, it indicates that the data packet transmission is completed, and data reception can be performed. Parse the packet header, extract valid data from the received data packets, combine them, and submit them to the corresponding application layer when the data transmission is complete. Indicates that the data has been sent this time.

The processing and manufacture of a cluster communication model based on address mapping in the present invention is very simple and convenient, and can be processed as shown in the accompanying drawings.

Except for the technical features described in the instructions, all are known technologies by those skilled in the art.

Claims (6)

1. the trunking communication model based on address mapping, is characterized in that being made up of multiple host computer systems and multiple communication module, and multiple host computer systems all comprise memory address mapped device and the communication module based on memory address mapping;

Described memory address mapped device, for memory address mapping between different host system;

Described traffic model, for data communication between different host system, realize point-to-point data transfer model, communication module resolution data packet address information, without device forwards such as switches, directly be transferred to destination host system, realized the data transfer model of high bandwidth, low delay, be applicable to large-scale data transmission between group system.

2. a kind of trunking communication model based on address mapping according to claim 1, is characterized in that connecting by memory address mapped device between described multiple host computer system, as NTB;

Described traffic model, for communication requirement between main frame, provides unified interface, use for computer system application, the address mapping device of communication module based between computer, by access map internal memory, carry out reading and writing data, thereby complete data communication.

3. a kind of trunking communication model based on address mapping according to claim 1, is characterized in that described memory address mapped device, and for interconnecting between multiple host computer systems, each main frame carries out memory address mapping with other All hosts; Position equity between main frame, complete reciprocal access between All hosts.

4. a kind of trunking communication model based on address mapping according to claim 1, it is characterized in that told communication module, also comprise special communication protocol encapsulation, communication link between main frame based on effective, reliable, succinct, reduce the loss of data communication in protocol processes, improve valid data duty ratio, promote data communication efficiency.

5. a kind of trunking communication model based on address mapping according to claim 1, is characterized in that being connected by interconnection between described each main frame, all comprises heartbeat module between described each main frame.

6. a kind of trunking communication model based on address mapping according to claim 1, it is characterized in that described heartbeat module,, the power supply of described opposite end storage control is reset when the malfunction for opposite end storage control detected by described interconnection.