TWI883861B - Data server system - Google Patents

Abstract

A data server system includes a central processing unit and a baseboard management controller disposed on a motherboard, an Ethernet switch, a plurality of processor nodes, two buses and first optical transceiver disposed on a substrate, and second and third optical transceivers and a physical layer chip disposed on an optical module board. The baseboard management controller is connected to the central processing unit. The Ethernet switch is connected to the central processing unit and baseboard management controller. The processor nodes is connected to the Ethernet switch and the central processing unit. The two buses are connected to the central processing unit and the processor nodes. The first optical transceiver is connected to the Ethernet switch. The second optical transceiver is connected to the first optical transceiver. The physical layer chip is connected to the second optical transceiver. The third optical transceiver is connected to the physical layer chip.

Description

Data Server System

The present invention relates to a data server system.

In response to the current business model of the extension of online short video platforms, the main needs are video transcoding and game processing. One business model is to provide high-definition video transcoding marketing products, broadcasting through the short video platform to attract customers to purchase product interest; the second business model is to provide real-time processing of online games, allowing online users to enjoy online game platform services. In order to meet the needs of the above platforms, it is necessary to provide a server system that can process large amounts of data.

In view of the above, the present invention provides a data server system.

According to an embodiment of the present invention, a data server system includes a central processor and a baseboard management controller arranged on a mainboard, an Ethernet switch, a plurality of processor nodes, a first bus, a second bus and a first optical transceiver arranged on a baseboard, and a second and third optical transceivers and a physical layer chip arranged on an optical module board. The baseboard management controller is connected to the central processor. The Ethernet switch is connected to the central processor and the baseboard management controller. The processor node is connected to the Ethernet switch and the central processor. The first and second buses are connected to the central processor and the processor node. The first optical transceiver is connected to the Ethernet switch. The second optical transceiver is connected to the first optical transceiver. The physical layer chip is connected to the second optical transceiver. The third optical transceiver is connected to the physical layer chip.

Through the above structure, the data server system disclosed in this case receives and transmits large amounts of data through optical transceivers, and uses Ethernet switches and the first and second buses as data transmission interfaces between the central processor and multiple processor nodes, which can improve the data storage density and processing capabilities of the server, allowing users to enjoy the big data services of online short video platforms and online game platforms in real time.

The above description of the disclosed content and the following description of the implementation method are used to demonstrate and explain the spirit and principle of the present invention, and provide a further explanation of the scope of the patent application of the present invention.

1:Data server system

11: Motherboard

12:Substrate

13: Optical module board

101:Central Processing Unit

102: Baseboard management controller

103:Ethernet switch

104,104-1 to 104-n: Processor nodes

105: First bus

106: Second bus

107: First optical transceiver

108: Second optical transceiver

109: Physical layer chip

110: The third optical transceiver

111:Ethernet transceiver

112:USB 3.0 controller

113:USB hub

114:USB 2.0 controller

115: USB interface chip

116:UART controller

117: Complex programmable logic device

1031: First transmission chip

1032: Second transmission chip

1033: The third transmission chip

1034: The fourth transmission chip

T1: First end

T2: Second end

T3: The third terminal

T4: The fourth end

N1: First node

N2: Second node

N3: The third node

N4: The fourth node

N5: Fifth Node

N6: Node 6

G1: Group 1

G2: The second group

G3: The third group

FIG1 is a block diagram of a data server system according to an embodiment of the present invention.

FIG2 is a schematic diagram of the internal structure of an Ethernet switch of a data server system according to an embodiment of the present invention.

FIG3 shows the signal connection relationship between the first and second transmission chips of the data server system of an embodiment of the present invention in different modes.

FIG4 is a schematic diagram of the management architecture of a data server system according to an embodiment of the present invention.

FIG5 is a schematic diagram of a USB topology design of a data server system according to another embodiment of the present invention.

FIG6 is a schematic diagram of a UART topology design of a data server system according to another embodiment of the present invention.

The detailed features and advantages of the present invention are described in detail in the following implementation method. The content is sufficient for anyone familiar with the relevant technology to understand the technical content of the present invention and implement it accordingly. According to the content disclosed in this specification, the scope of the patent application and the drawings, anyone familiar with the relevant technology can easily understand the relevant purposes and advantages of the present invention. The following embodiments are to further illustrate the viewpoints of the present invention, but do not limit the scope of the present invention by any viewpoint.

Please refer to FIG. 1, which is a block diagram of a data server system according to an embodiment of the present invention. As shown in FIG. 1, the data server system 1 includes a central processing unit 101 and a baseboard management controller 102 disposed on a mainboard 11, an Ethernet switch 103, a plurality of processor nodes 104, a first bus 105, a second bus 106 and a first optical transceiver 107 disposed on a substrate 12, and a second optical transceiver 108, a physical layer chip 109 and a third optical transceiver 110 disposed on an optical module board 13. The baseboard management controller 102 is connected to the central processing unit 101. The Ethernet switch 103 is connected to the central processing unit 101 and the baseboard management controller 102. The processor node 104 is connected to the Ethernet switch 103 and the central processing unit 101. The first and second buses 105 and 106 are connected to the central processor 101 and the processor node 104. The first optical transceiver 107 is connected to the Ethernet switch 103. The second optical transceiver 108 is connected to the first optical transceiver 107. The physical layer chip 109 is connected to the second optical transceiver 108. The third optical transceiver 110 is connected to the physical layer chip 109.

For example, the central processor 101 may be, for example, a D2775 chip of Intel Corporation, or the data server system 1 may also use other high-performance processors that can process large data, but the present invention is not limited thereto. The baseboard management controller 102 may be, for example, an AST2600 chip of ASPEED Corporation, which is used to execute the processing flow of server management, or the data server system 1 of the present invention may also use other baseboard management controllers (BMC). The central processor 101 and the baseboard management controller 102 may be connected to each other through a variety of connector specifications, such as PCIe, USB, UART, NCSI, etc. The Ethernet switch 103 may include a plurality of data transmission chips, which are used as the main data transmission interface between the mainboard 11, the baseboard 12 and the optical module board 13. The Ethernet switch 103 and the central processor 101 can be connected to each other through a variety of connector specifications, such as PCIe, CAUI, etc. The Ethernet switch 103 and each of the multiple processor nodes 104-1 to 104-n can be connected to each other through a 2.5GBaseT connector specification. The Ethernet switch 103 and the first optical transceiver 107 can be connected to each other through a 100G CAUI connector specification.

Each of the multiple processor nodes 104 can be a processing unit, such as Qualcomm's QCS8550 chip, or the data server system 1 can also use other high-performance processors that can perform operations on large data, but the present case is not limited thereto. In one embodiment, the number of the multiple processor nodes 104-1 to 104-n can be 240, and every four of these processor nodes 104-1 to 104-n can be set on a computing card (POD card), and each computing card (60 in total) can support hot-swap function. Specifically, each computing card can be installed in a slot of the computing card base, and through the gold finger mechanism design and related line protection mechanism, the hot-swap function of each computing card can be realized, supporting online hot operation mode.

The first bus 105 and the second bus 106 may be, for example, PI7C9X2G608GP switches based on the PCIe connector specification. The first bus 105 may be connected to each processor node 104 via a PCIe to UART adapter. The second bus 106 may be connected to each processor node 104 via a PCIe to USB 3.0 adapter. The first and second optical transceivers 107 and 108 may be, for example, Quad Small Form-factor Pluggable (QSFP) transceivers, and support a data transmission rate of 100G. The third optical transceiver 110 may include a dual Small Form-factor Pluggable (DSFP) and a quad Small Form-factor Pluggable (QSFP) transceiver, and support a 100G data transmission rate. In particular, a set of optical transceivers may include two optical transceiver components. The physical layer chip 109 may be, for example, a BCM81358 chip from Broadcom, which is a low-power component and has the functions of retime and equalize.

Through the above configuration, 240 processor nodes can transmit data through the 2.5G Base-T network interface, and these 240 2.5G Base-Ts are aggregated into two 100G uplink optical transmission interfaces through a two-level Ethernet switch; it supports two UARTs to each processor node, the main UART uses USB to UART, and the backup UART uses PCIe to UART and switches through a complex programmable logic device multiplexer (CPLD MUX). The latter is a backup design solution to improve the reliability of system management; it supports USB3.0 to each processor node to realize the firmware upgrade of each processor node.

Please refer to FIG. 2 in conjunction with FIG. 1. FIG. 2 is a schematic diagram of the internal structure of an Ethernet switch of a data server system according to an embodiment of the present invention. As shown in FIG. 2, the Ethernet switch 103 of this example may include two first transmission chips 1031, a second transmission chip 1032, a plurality of third transmission chips 1033, and a plurality of fourth transmission chips 1034. The two first transmission chips 1031 may be connected to the central processor 101 through the first end T1, and connected to the first optical transceiver 107 through the second end T2. The second transmission chip 1032 is connected to the two first transmission chips 1031. A plurality of third transmission chips 1033 are connected to the second transmission chip 1032. Multiple fourth transmission chips 1034 are connected to multiple third transmission chips 1033, and are connected to multiple processor nodes 104 through the third terminal T3.

In this example, the first transmission chip 1031 may be a BCM81381 chip of Broadcom, the second transmission chip 1032 may be a BCM56771 chip of Broadcom, the third transmission chip 1033 may be a BCM56072 chip of Broadcom, and the fourth transmission chip 1034 may be a BCM54908E chip of Broadcom. The first transmission chip 1031 and the second transmission chip 1032 may be connected via a CAUI connector specification and support a data transmission rate of 100G. The second transmission chip 1032 and the third transmission chip 1033 may be connected via a Base-KR4 connector specification and support a data transmission rate of 100G. The third transmission chip 1033 and the fourth transmission chip 1034 may be connected via a QXGMII connector specification and support a data transmission rate of 100G. Furthermore, the number of third transmission chips 1033 in this example can be 5, the number of fourth transmission chips can be 30, and the number of processor nodes 104 can be 240. That is, the second transmission chip 1032 can be connected to 5 third transmission chips 1033, each third transmission chip 1033 can be connected to 6 fourth transmission chips 1034, and each fourth transmission chip 1034 can be connected to 8 processor nodes 104. Furthermore, the data server system in this example can include multiple switching components. These switching components are connected to the multiple processor nodes 104 through the Base-T connection port, and are connected to the fourth transmission chip 1034 of the Ethernet switch 103 through the PCIe connection port.

The PCIe of each processor node is converted to 2.5GBase-T through Intel's Ethernet controller I226 and enters the system's Ethernet link; the system uses 30 BCM54908E to convert each 8 2.5GBase-T to 2 10G QXGMII, which are then converted to 5 100G Base-KR by 5 BCM56072, and then aggregated into 2 100G uplink ports through BCM56771; the system's 2 100G uplink ports are connected to the optical module board through active optical cables, and retimer/gearbox is used by BCM81358 to support QSFP or DSFP; 2 BCM81381 are used to run the normal mode (Normal Mode) to achieve 100G dual uplink; use two BCM81381s to run multiplexing mode (MUX mode) to achieve a dual-port preboot execution environment (Preboot eXecution Environment, PXE).

Please refer to FIG. 3 in conjunction with FIG. 1 and FIG. 2. FIG. 3 shows the signal connection relationship between the first and second transmission chips of the data server system of an embodiment of the present invention in different modes. As shown in FIG. 3, the two first transmission chips 1031 support a data transmission speed of 100G and are used to be switched to a first mode or a second mode under the control of the baseboard management controller 102. In the first mode, the central processor 101 can directly receive the signal from the first optical transceiver 107 through the first node N1 and the second node N2 of the two first transmission chips 1031. In the second mode, the central processor 101 receives the signal from the first optical transceiver 107 through the first node N1 and the third node N3 of the two first transmission chips 1031 and the fifth node N5 of the second transmission chip 1032. That is, in the first mode, the signal received from the third optical transceiver 110 is transmitted to the first optical transceiver 107 through the physical layer chip 109 and the second optical transceiver 108, and is transmitted to the central processor 101 through the second node N2 and the first node N1 of the first transmission chip 1031. In the second mode, the signal received from the third optical transceiver 110 is transmitted to the first optical transceiver 107 through the physical layer chip 109 and the second optical transceiver 108, and is transmitted to the second transmission chip 1032 (through the sixth node N6) through the second node N2 and the fourth node N4 of the first transmission chip 1031, and then transmitted to the central processor 101 through the fifth node N5 of the second transmission chip 1032 and the third node N3 and the first node N1 of the first transmission chip. The first mode corresponds to the above-mentioned general mode, and the second mode corresponds to the above-mentioned multiplexing mode.

Please refer to FIG. 4 in conjunction with FIG. 1 and FIG. 2. FIG. 4 is a schematic diagram of a management architecture of a data server system according to an embodiment of the present invention. As shown in FIG. 4, the central processor 101 can be used to manage the second transmission chip 1032 and multiple third transmission chips 1033, and each third transmission chip 1033 can be used to manage multiple fourth transmission chips 1034. Specifically, the central processor 101 can manage the second transmission chip 1032 through 4 PCIeGen3s, and manage 5 second transmission chips 1032 through 5 PCIeGen3s respectively. The baseboard management controller 102 can be connected to an Ethernet transceiver 111, and is used to manage the physical layer chip 109 and the two first transmission chips 1031. With this configuration, the baseboard management controller 102 can complete the initialization of two BCM81381s and BCM81358s through the MDC/MDIO interface; the central processor 101 manages the BCM56771 and five BCM56072s through the PCIe interface; and each BCM56072 manages six BCM54908Es through two MDC/MDIO interfaces.

Please refer to FIG. 5 in conjunction with FIG. 1 , which is a schematic diagram of a USB topology design of a data server system according to another embodiment of the present invention. The data server system of this example may further include a plurality of universal serial buses connected between the second bus 106 and the processor node 104. As shown in FIG. 5 , the central processing unit 101 may be connected to the second bus 106 via a PCIe2.0×4 interface. The second bus 106 may be connected to the four USB 3.0 controllers 112 of the first group G1 via four PCIe2.0×1 interfaces. The central processing unit 101 may be connected to the eight USB 3.0 controllers 112 of the first group G1 via eight PCIe2.0×1 interfaces. That is, the central processor 101 can be connected to a total of 12 USB 3.0 controllers 112 of the first group G1. The central processor 101 can also be connected to a USB 3.0 controller 112 of the second group G2 through a PCIe2.0×1 interface. In each first group G1, the USB 3.0 controller 112 can be connected to three USB hubs 113 through three USB 3.0 interfaces, and the three USB hubs 113 are connected to a total of 18 processor nodes 104 through six USB 3.0 interfaces. In each first group G1, the USB 3.0 controller 112 can also be connected to one processor node 104 through one USB 3.0 interface. That is, in the first group G1, each USB 3.0 controller 112 can be connected to 19 processor nodes 104.

In the second group G2, the USB 3.0 controller 112 can be connected to two USB hubs 113 through two USB 3.0 interfaces, and the two USB hubs 113 are connected to a total of 12 processor nodes 104 through six USB 3.0 interfaces. In other words, in the second group G2, each USB 3.0 controller 112 can be connected to 12 processor nodes 104. Under this architecture, the central processor 101 can be connected to 12×19+12=240 processor nodes 104 through the USB 3.0 controllers 112 of the first group G1 and the second group G2 via a universal serial bus. In addition, the central processor 101 can also be connected to the baseboard management controller 102 through a USB port, and has a spare USB port for other devices to connect.

Please refer to FIG. 6, which is a schematic diagram of a UART topology design of a data server system according to another embodiment of the present invention. The data server system of this example may further include a plurality of universal asynchronous transceivers connected between the first bus 105 and the plurality of processor nodes 104, and these universal asynchronous transceivers include a plurality of main transmitters and a plurality of backup transmitters. As shown in FIG. 6, the central processor 101 may be connected to the first bus 105 via a PCIe2.0×1 interface. The first bus 105 may be connected to the four USB 3.0 controllers 112 of the third group G3 via four PCIe2.0×1 interfaces. In other words, the central processor 101 may be connected to a total of four USB 3.0 controllers 112 of the third group G3. In each third group G3, the USB 3.0 controller 112 can be connected to two USB 2.0 controllers 114 through two USB 2.0 interfaces respectively, and the four USB 2.0 controllers 114 can be connected to seven USB interface chips 115 through seven USB 2.0 interfaces respectively. Each USB interface chip 115 can be connected to four processor nodes 104 through a UART port. In each third group G3, the USB 3.0 controller 112 can also be connected to one USB interface chip 115 through one USB 2.0 interface to connect to four processor nodes 104 through a UART port. That is, in the third group G3, each USB 3.0 controller 112 can be connected to 4×7×2+4=60 processor nodes 104. In addition, the first bus 105 can also be connected to the UART controller 116 through a PCIe1.0×1 interface. The UART controller 116 can be connected to the complex programmable logic device (CPLD) 117 through 8 UART ports. The complex programmable logic device 117 can be connected to 240 processor nodes 104 through 240 UART ports as a backup transmitter.

For example, the USB 3.0 controller of the above embodiment can be implemented using the UPD720201 chip of Renesas. The USB hub 113 can be implemented using the USB5806 chip of Microchip. The USB 2.0 controller 114 can be implemented using the USB2517 chip of Microchip. The USB interface chip 115 can be implemented using the FT4232 chip of FTDI. The UART controller 116 can be implemented using the XR17V358 chip of MAXLinear. However, the present invention is not limited to the above components. Through the configuration of this embodiment, the data server system can support 2 front USB3.0 interfaces, and a USB2.0 interface is reserved between the CPU and the BMC for communication. The system provides 240 USB3.0 channels to realize the function of upgrading the firmware of the QCS8550 in each node through USB3.0. In this solution, the CPU PCIe port is allocated and expanded to 13 PCIe2.0 through PCIe Switch PI7C9X2G608GP, and then expanded to 240 USB3.0 through USB controller uPD720201 and USB hub. Limited by the number of slots and endpoints supported by uPD720201, one uPD720201 can expand to 19 USB3.0, so this solution uses 12 sets of 1×uPD720201+3×USB5806 combination solutions, plus 1 set of 1×uPD720201+2×USB5806 combination solutions, to achieve the 240-channel USB3.0 requirement in this system.

Through the configuration of this embodiment, this system supports two UARTs to each node, one for the main UART and one for the backup UART. In this solution, the CPU's 1-way X1PCIe2.0 is allocated and expanded to 5-way PCIe through the PCIe switch PI7C9X2G608GP, of which 4-way is expanded to 240-way main UART through the PCIe to USB and USB to UART solution. The implementation solution is: 4 sets of 1×uPD720201+2×USB2517+7×FT4232; the other PCIe is converted to 8-way URAT through XR17V358, and then the CPLD realizes the expansion and switching of 240-way backup UART.

For PCIe allocation, please refer to the following description. The network switch BCM56771 occupies one PCIe3.0x4, and five BCM56072s occupy five PCIe3.0×1s. UART: PCIe2.0x1 is expanded through the switch to four PCIe2.0×1s connected to four USB controllers for USB-UART conversion chips + one PCIe2.0×1 is expanded to connect to PCIe-UART controllers. USB: PCIe2.0×4 is expanded through the switch to four PCIe2.0×1s connected to four USB controllers + HSIO leads to nine PCIe2.0×1s connected to nine USB controllers, for a total of 13. BMC: PCIe2.0×1; two USB3.0×1s; two M.2 SATAs.

This system also supports a 4-power supply (2+2) architecture, and the system power distribution board (PDB) supplies power to each board. The PDB supplies power to the baseboard through 4 2×12 power connectors; the PDB supplies power to the optical module board through 1 2×2 power connector; the baseboard supplies power to the mainboard through 1 2X2 power connector.

Through the above structure, the data server system disclosed in this case receives and transmits large amounts of data through optical transceivers, and uses an Ethernet switch and the first and second buses as data transmission interfaces between a central processor and multiple processor nodes, thereby improving the data storage density and processing capabilities of the server, allowing users to enjoy the big data services of online short video platforms and online game platforms in real time. In addition, this system supports 240 nodes of PCIe to 2.5G Base-T, and aggregates them into 2 100G uplink ports through the system network to achieve video and audio transcoding and game data transmission; the system supports 240 main UARTs for individual node control and management, and supports 240 backup UARTs as a redundant design to improve the reliability of system management; the system supports front 100G dual uplink function; the system supports PXE function, through BMC switching, supports transparent and connected switch chip transfer mode; each POD is equipped with 4 nodes, which can save the maintenance cost of bad cards, and supports hot plugging, through the gold finger mechanism design, and related line protection mechanism, supports online hot operation mode.

Although the present invention is disclosed as above by the aforementioned embodiments, it is not intended to limit the present invention. Any changes and modifications made within the spirit and scope of the present invention are within the scope of patent protection of the present invention. Please refer to the attached patent application for the scope of protection defined by the present invention.

1:Data server system

11: Motherboard

12:Substrate

13: Optical module board

101:Central Processing Unit

102: Baseboard management controller

103:Ethernet switch

104,104-1 to 104-n: Processor nodes

105: First bus

106: Second bus

107: First optical transceiver

108: Second optical transceiver

109: Physical layer chip

110: The third optical transceiver

Claims (10)

A data server system includes: a mainboard; a central processing unit, arranged on the mainboard; a baseboard management controller, arranged on the mainboard and connected to the central processing unit; a baseboard; an Ethernet switch, arranged on the baseboard and connected to the central processing unit and the baseboard management controller; a plurality of processor nodes, arranged on the baseboard, connected to the Ethernet switch and connected to the central processing unit; a first bus and a second bus, arranged on the baseboard and connected to the central processing unit and the processor nodes; a set of first optical transceivers, arranged on the baseboard and connected to the Ethernet switch; an optical module board; a set of second optical transceivers, arranged on the optical module board and connected to the set of first optical transceivers; A physical layer chip is disposed on the optical module board and connected to the second optical transceiver group; and a third optical transceiver group is disposed on the optical module board and connected to the physical layer chip. A data server system as described in claim 1, wherein the Ethernet switch includes two first transmission chips, a second transmission chip, multiple third transmission chips and multiple fourth transmission chips, the two first transmission chips are connected to the group of first optical transceivers, the second transmission chip is connected to the two first transmission chips, the third transmission chips are connected to the second transmission chip, and the fourth transmission chips are connected to the third transmission chips and the processor nodes. A data server system as described in claim 2, wherein the number of the processor nodes is 240, the number of the third transmission chips is 5, and the number of the fourth transmission chips is 30. A data server system as described in claim 2, wherein the two first transmission chips support a data transmission speed of 100G and are used to be switched to a first mode or a second mode under the control of the baseboard management controller, in which the central processing unit directly receives signals from the group of first optical transceivers through the two first transmission chips, and in which the central processing unit receives signals from the group of first optical transceivers through the two first transmission chips and the second transmission chip. A data server system as described in claim 2, wherein the central processor is used to manage the second transmission chip and the third transmission chips, and the third transmission chips are used to manage the fourth transmission chips. The data server system as described in claim 1 further includes multiple adapter components, which are connected to the processor nodes via Base-T ports and connected to the network switch via PCIe ports. The data server system as described in claim 1 further includes a plurality of universal serial buses connected between the second bus and the processor nodes. The data server system as described in claim 1 further includes a plurality of universal asynchronous receiver/transmitters connected between the first bus and the processor nodes, wherein the universal asynchronous receiver/transmitters include a plurality of main transmitters and a plurality of backup transmitters. A data server system as described in claim 1, wherein the set of third optical transceivers includes a DSFP port and a QSFP port. A data server system as described in claim 1, wherein the number of the processor nodes is 240, every four of the processor nodes are arranged on a computing card, and the computing card supports hot-swap function.

Images