RU209333U1 - HIGH DENSITY COMPUTING NODE - Google Patents

Abstract

The utility model relates to the field of computer technology and can be used to create high-performance computing systems (HPC). The high-density computing node contains a 4U chassis, blade servers, communication devices, and power supplies. The chassis is divided into front and rear sections by information and electrical backplanes, the server blades are installed in the front of the chassis, and the communication devices and power supplies are installed in the back of the chassis. Blade servers and communication devices are connected from different sides to the backplanes. Each communication device has three groups of ports. The first groups of ports connect communication devices to each other in a fully connected topology. The second groups of ports of communication devices are external outputs of a high-density computing node, which are designed to combine high-density computing nodes into a single high-performance computing system of unlimited capacity. Up to two corresponding blade servers are connected to the third group of ports on each communication device. Blade servers and communication devices are equipped with a contact liquid cooling system. The technical result consists in increasing the efficiency of a high-density computing node. 5 ill.

Description

The utility model relates to the field of computer technology and can be used to create high-performance computing systems (HPC).

Known server platform (computing node) (patent RU No. 2389058 priority dated 08/07/2008, "Server platform", author Slepukhin A.F.. IPC G06F 1/12, H05K 7/20, published on 10.05.2010 Bull. No. 13) , containing a chassis that is divided in the direction of cooling air into front and rear sections, with rails for blade servers, blade servers, power supplies, and air-cooling fans. Guides for blade servers are made on part of the length of the case and are located in its rear section. The fans are combined in pairs into cooling modules and placed in the front section of the case, free from guides. 10 dual-socket blade servers are installed from the back of the chassis inside the rear section of the chassis. Between the blade servers and cooling modules there is a free cavity that acts as an air collector. The power supplies are placed above the blade servers in such a way that said cavity is in fluid communication with the housing space in which the power supplies are installed. On the end surfaces of the blade servers, facing outward from the rear section of the case, connectors for inter-interface exchange are made. A standard 42 U 19" telecom rack accommodates up to eight server platforms.

The disadvantages of the known device are the lack of communication equipment in its composition that ensures efficient exchange of information between the blade servers of the server platform and the need to use additional external communication equipment, the use of air cooling in the server platform does not allow achieving a high density of computing resources and good energy efficiency.

Known blade system (high-density computing node) 8U SuperBlade SBE-820C manufactured by Supermicro (8U SuperBlade User's Manual revision 1.2a, published 10/24/2018. https://www.supermicro.com/manuals/superblade/MNL-SuperBlade_8U.pdf ). The SuperBlade SBE-820C blade system contains a chassis that contains a power supply subsystem, an air cooling subsystem and a communication subsystem that combines from 2 to 20 blade servers. The communication subsystem uses the communication hardware. consisting of two types of communication devices: a 40-port InfiniBand switch and Mellanox ConnectX-4 channel adapters or a 44-port Intel Omni-Path switch and Intel Omni-Path Wolfriver channel adapters. A standard 42 U 19" telecom rack accommodates up to 5 blade systems.

The disadvantages of the known blade system are: the value of the average distance of the communication network of the blade system is two hops, the message passes at least three communication devices, which leads to an increase in the delay in message transmission: the use of two different types of communication devices in the communication subsystem: a switch and an adapter, failure of a multiport switch leads to failure of the entire blade system; the use of air cooling, which increases energy consumption and noise level.

The closest analogue, in terms of essential features, to the claimed utility model is the TC4600E-LX blade system developed by the Chinese company Sugon (Sugon TC4600E-LC: Updated blade system with LSS, published on 11/20/2015, http://hpc-lc .ru/news/sugon_tc4600e__liquid_blade_rack).

The TC4600E-LC blade system contains a chassis that houses the power subsystem, hybrid cooling subsystem, communication subsystem and can accommodate 10 blade servers or 5 heterogeneous blades plus 10 GPUs or Intel Xeon Phi accelerators. The communication subsystem uses communication hardware consisting of two types of communication devices: a 36 or 40 port InfiniBand switch and Mellanox ConnectX-4 channel adapters. The TC4600E-LC Blade System is 6U high and can accommodate up to 6 blade systems in a standard 42U rack. In a hybrid cooling system, only the blade servers are cooled with water, while air cooling is used for the rest of the blade system.

The disadvantages of this device are: the value of the average distance of the communication network of the blade system, equal to two hops; the use of two different types of communication devices in the communication subsystem: a switch and an adapter; leads to a decrease in the packing density.

The technical problem to be solved by the claimed utility model is the creation of a fault-tolerant high-density computing node, 4U high, providing accommodation for up to 16 blade servers, each of which contains N processors, where N = 1, 2, ... and RAM. The value of the average message transfer distance between the blade servers of one high-density computing node should not exceed one hop.

The technical result consists in increasing the efficiency of a high-density computing node by reducing the value of the average distance of the communication network of a high-density computing node to one hone and. hence, reducing the delay in the transmission of messages.

This technical result is achieved by the fact that in a high-density computing node containing a housing, power supplies, a cooling system, communication devices and blade servers, each of which contains N processors, where N = 1, 2, ... and RAM, while blocks power supplies are connected to the respective blade servers and communication devices, what is new is that the chassis is divided into front and rear sections by information and electrical backplanes, while rails for blade servers and blade server are installed in the front section of the chassis, and rails for the blade servers and the blade server are installed in the rear section switching devices are installed, made on the basis of low-port, full-matrix switches and interconnected through an information backplane by the first groups of ports in a fully connected topology, the second groups of ports of communication devices are external outputs of a high-density computing node, and to the third group of ports of each communication device up to two corresponding blade servers are connected through the information backplane, while the cooling system is made using a contact liquid cooling system for the electronic components of the blade servers and communication devices.

In the inventive high-density computing node, due to the connection of communication devices and their corresponding blade servers to each other in a fully connected topology ("each with each"), the diameter and average distance of the communication network is reduced to one hop, which leads to a decrease in the delay in the transmission of messages and in in turn to increase efficiency.

Blade servers and communication devices are equipped with a contact liquid cooling system. The outer end surfaces of each blade server and each communication device have two quick-disconnect connections for coolant inlet and outlet, which provide leakage protection when connected to an external fluid supply and drainage system.

Power supplies are located in the back of the case along its walls, to the right and left of the communication devices. Power supplies can be made in the form of stand-alone modules, each of which has a housing with batteries placed inside it and an exhaust fan. The power supplies plug into the connectors on the electrical backplane.

The case of a high-density computing node can be made with a removable cover, and inside the case there are guides directed along its length for blade servers installed in front and communication devices at the back.

Blade servers and communications devices are placed vertically in the chassis, making it easy to independently remove blade servers and communications devices from the high-density compute node for repair work without shutting down the high-density compute node.

Figure 1 shows a front isometric view of a high-density computing node. Figure 2 shows a rear isometric view of a high-density computing node. Figure 3 shows an isometric view of a blade server of a high-density computing node. Figure 4 shows an isometric view of a communication device of a high-density computing node. Figure 5 shows a diagram of the communication network of a high-density computing node.

A high-density computing node (figure 1) contains a case 1 (shown transparent), which is divided into front 2 and rear 3 sections by backplanes: information 4 and electrical 5. Blade servers 6 are installed in the front section 2 of case 1. Outside the front section 2 there is a system for supplying 7 coolant to each blade server 6 and removing 8 the heated coolant from the blade servers 6. Chassis ! high-density computing node is equipped with a top cover 9.

Communication devices (10) and power supplies 11 (figure 2) are installed in the rear section 3 of the high-density computing node. Outside the rear section 3 there is a system for supplying 12 coolant to the communication devices 10 and removing 13 the heated coolant from the communication devices 10.

Each blade server (figure 3) contains processors 14 and RAM 15, equipped with a contact system 16 liquid cooling. On the blade server motherboard 17, an information connector 18 is installed to connect the blade server to the information backplane 4 (in Fig. 1), and a connection board with a connector 19 to connect to the electrical backplane 5 (in Fig. 1). The outer end surfaces of the blade server have two quick connectors 20 each for coolant inlet and outlet, which provide leak protection when connected to an external coolant supply and discharge system.

Each communication device (figure 4) contains a very large-scale integrated circuit (VLSI) of a low-port, full-matrix switch 21 and three QSFP-DD 22 transceivers equipped with a contact liquid cooling system 23. An information connector 25 is installed on the back side of the board 24 of the communication device, with which the communication device is connected to the information backplane 4 (in Fig.1) and the connector 26 for connection to the electrical backplane 5 (in Fig.1). Three holes 27 are made on the outer end surfaces of the communication device, into which the connectors of the QSFP-DD 22 transceivers and two quick-disconnect connections 28 for the inlet and outlet of the liquid are made, which provide protection against leaks when connected to an external system for supplying and discharging coolant.

Figure 5 shows a diagram of a communication network of a high-density computing node, consisting of eight VLSI low-port, full-matrix switch 21.0 - 21.7. Each full matrix switch 21 has twelve ports divided into three groups. The first group of ports includes ports with numbers 1-7 (21) providing the combination of switches 21.0 - 21.7 of a high-density computing node in the "each with each" topology with the diameter and average distance of the communication network equal to one hop. The second group of ports includes ports numbered 8, 9 and 10, set aside for external switching and allow through connectors QSFP-DD 22 (in Fig.4) to combine high-density computing nodes with each other, creating high-performance computing systems. The third group of ports includes ports numbered 11 and 12, which provide information transfer from switches 21.0 - 21.7 to blade servers 6.0 - 6.12. The third group of ports on the same switch can connect either to one blade server, such as the 6.4 blade server, or to two different blade servers, such as the 6.2 and 6.3 blade servers.

The high-density computing node operates as follows.

A high-density computing node is a basic element in creating high-performance computing systems of any performance designed for high-performance computing in industry and science.

High-density computing nodes are installed in a rack with a standard height of 42 U. Up to ten high-density computing nodes are placed in one rack, which are connected by cables through QSFP-DD 22 connectors (figure 4). Blade servers 6 and communication devices 2 I are connected to an external system for supplying and discharging coolant. Power supplies are connected to the power supply system.

The exchange of information between the processes of one application running on processors located in different blade servers. is carried out as follows.

For example, process A runs on a 6.0 blade server processor and process B runs on a 6.1 blade server processor. The transfer of information from process A to process B is carried out as follows: from the blade server processor 6.0, information is transmitted to the switch 21.0 through port 11, and then from the switch 21.0 through port 12 to the blade server processor 6.1. The information transfer route passes through only one communication device.

Process A runs on a 6.0 blade server processor and process B runs on a 6.6 blade server processor. The transfer of information from process A to process B is carried out as follows: from the blade server processor 6.0, information is transmitted through port 11 to switch 21.0, then through port 3 of switch 21.0 it is transferred to port 1 of switch 21.3, and then through port 12 of switch 21.3 it is transferred to the processor blade servers 6.6. The information transfer path has a distance of one hop, and passes through two communication devices.

Processes A and B run on blade server processors located in different high-density computing nodes. The transfer of information from process A to process B is carried out as follows: from the blade server processor located in the first high-density computing node, information is transmitted through the corresponding port 11 or 12 to the corresponding switch. Then, depending on the topology of the communication network of a high-performance computing system, through ports 8, 9 or 10 of this switch, information is transmitted to ports 8, 9 or 10 of an adjacent switch located in the second high-density computing node. Then, through ports 11 or 12 of the adjacent switch, it is transmitted to the blade server processor located in the second high-density computing node.

Claims (1)

A high-density computing node containing a chassis, power supplies, a cooling system, communication devices and blade servers, each of which contains N processors, where N = 1, 2, ... and RAM, while the power supplies are connected to the corresponding blade servers and communication devices, characterized in that the case is divided into front and rear sections by information and electrical backplanes, while in the front section of the case there are guides for blade servers and identical blade servers, and in the rear section there are switching devices made on the basis of low-port , full-matrix switches and interconnected through an information backplane by the first groups of ports in a fully connected topology, the second groups of ports of communication devices are external outputs of a high-density computing node, and to the third group of ports of each communication device through an information backplane up to two corresponding blade servers are connected, while the cooling system is made using a contact liquid cooling system for the electronic components of the blade servers and communication devices.

Images