CN100372334C - A device and method for transmitting InfiniBand data in an optical network - Google Patents

Abstract

The invention discloses a device for transmitting InfiniBand data in an optical network. The device is located between an InfiniBand port and an optical network port, and includes an InfiniBand interface module, a mapping and flow control module, a data processing module, an optical network interface module and a management Module, using this device can realize the reliable transmission of InfiniBand protocol on SDH/SONET. The invention also discloses a method for transmitting InfiniBand data in an optical network. In the method, the InfiniBand data frame is used as the payload of an advanced data link control protocol data frame that can be transmitted by the optical network, and is encapsulated in the advanced data link control protocol. In this way, the transmission of InfiniBand data frames in the optical network can be realized. Utilize widely used SDH/SONET network to realize reliable transmission in InfiniBand wide area. Compared with the prior art, the present invention well realizes functions such as server remote clustering and remote storage based on InfiniBand, and greatly expands the application range of InfiniBand.

Description

A device and method for transmitting InfiniBand data in an optical network

technical field

The invention relates to the data transmission technology in the optical network, in particular to a device and a method for realizing the transmission of InfiniBand data in the optical network.

Background technique

InfiniBand (hereinafter referred to as IB) technology is a switched serial bus technology. Its technical standards stipulate all operating protocols from the physical layer to the transport layer, and the functions from one layer to four layers are realized by hardware, thus greatly The CPU load is reduced, and IB technology has also formulated detailed flow control strategies and security mechanisms, which can realize port-level and connection-level flow control and point-to-point security protection.

Refer to FIG. 1 for the network configuration of the IB technology. FIG. 1 is a schematic diagram of the network configuration of the IB technology in the prior art. As shown in FIG. 1 , the devices included in the IB network include: a host channel adapter (HCA) 105 , a target channel adapter (TCA) 106 , an IB switch (Switch) 107 , an IB router (Router) 108 and the like. The IB network is connected to the CPU 101 through the system memory controller 103 connected to the system memory 104 and the system bus 102 . In Fig. 1, what connects Switch 107, HCA 105, TCA 106, and Router 108 is an IB link (IB Link).

HCA and TCA are the end (EndNode) devices in the InfiniBand architecture. HCA is mainly a device connecting the CPU and the IB link, while TCA is mainly connecting the I/O device and the IB link; Switch is an InfiniBand link layer device that can Realize the forwarding function of InfiniBand messages in the same IB subnet; Router is the InfiniBand network layer device, which can realize the message forwarding function between IB subnets or between IB subnets and different IB networks; IB Link is a specific physical link The transmission medium can be printed circuit board (PCB), copper cable and optical fiber.

At present, the use of IB technology in the communication system can realize the interconnection between boards, frames and devices. As an emerging switched serial bus technology, IB technology will be widely used in future data storage area network (SAN), high-performance clusters, and data communications.

However, the farthest distance of copper cable connection described in the current IB specification is 17m, and the furthest distance of optical fiber connection is only about a dozen kilometers. Therefore, IB technology is mainly used in server clusters and local storage, and it is limited to system A regional network cannot realize wide-area IB interconnection. In this way, the wide application and further development of IB technology are restricted.

In the optical network transmission technology, Synchronous Digital Hierarchy (SDH)/Synchronous Optical Network (SONET) is a very mature technology, which has the advantages of good security and high reliability, and has been widely used in telecommunication networks. SDH/SONET itself is a transparent transmission channel, which can realize long-distance transmission of multiple protocol data.

Contents of the invention

In view of this, the main purpose of the present invention is to provide a device for realizing the transmission of InfiniBand data in the optical network, and the reliable transmission of IB protocol on SDH/SONET can be realized by using the device.

Another main purpose of the present invention is to provide a method for realizing the transmission of InfiniBand data in the optical network, utilizing the widely used SDH/SONET network to realize the reliable transmission of IB in the wide area.

According to one aspect of the purpose of the above invention, the present invention provides a device for transmitting InfiniBand data in an optical network, the device is located between the InfiniBand port and the optical network port, and includes an InfiniBand interface module, which is connected to the InfiniBand port and mapping and flow The control modules are connected separately to send and receive data transmitted between the InfiniBand port and the mapping and flow control modules;

The mapping and flow control module receives the InfiniBand data packet from the InfiniBand interface module, adds the flow control information to the InfiniBand data packet, and sends it to the data processing module; or removes the flow control information from the data packet received from the data processing module After that, send it to the InfiniBand interface module;

Data processing module, which receives data packets from the mapping and flow control module, and sends the data packets to the optical network interface module after adding header information, or receives data packets from the optical network interface module, removes the header information and sends them to the mapping and flow control module;

An optical network interface module, which receives data sent by the data processing module and sends it to the optical network port, or receives data sent by the optical network port and sends it to the data processing module;

The management module is connected to other modules by sending management information to the other modules and receiving the reported information from the other modules.

Wherein, the InfiniBand interface module may include an InfiniBand interface circuit and a virtual channel receiving and sending buffer; the InfiniBand interface circuit receives the data sent by the InfiniBand port and sends it to the virtual channel sending buffer, or receives the data sent by the virtual channel receiving buffer and sends it It is sent to the InfiniBand port; the virtual channel send buffer temporarily stores the data received from the InfiniBand interface circuit and then sends it to the mapping and flow control module; the virtual channel receive buffer temporarily stores the data received from the mapping and flow control module and then sends it to the InfiniBand interface circuit.

The virtual channel sending buffer can further include a virtual channel sending queue and a virtual channel sending multiplexing module, and the data received by the virtual channel sending buffer is first temporarily stored in the virtual channel queue, and then sent to the mapping and the virtual channel multiplexing module through the virtual channel multiplexing module. Flow control module. The virtual channel receiving buffer can further include a virtual channel receiving queue and a virtual channel receiving multiplexing module. The data received by the virtual channel receiving buffer is temporarily stored in the virtual channel queue, and then sent to the InfiniBand interface circuit through the virtual channel multiplexing module.

The virtual channel sending buffer can also be further connected to the InfiniBand interface circuit by sending a sending buffer status signal to the InfiniBand interface circuit; the virtual channel receiving buffer can also further send a receiving buffer status signal to the mapping and flow control module Connected to the Mapping and Fluidics modules.

The data processing module may further include a data calculation checksum module, which calculates a checksum for the data received from the mapping and flow control module; The data received by the network interface module is checked to calculate a checksum.

The management information sent by the management module to other modules may include: sending port control information to the InfiniBand interface module, flow control policy information sent to the mapping and flow control module, control information sent to the data processing module, The network interface module sends control information for the port. The reporting information received by the management module from other modules may include: port status reporting information sent by the InfiniBand interface module, flow control status reporting information sent by the mapping and flow control module, data packet statistics reporting information sent by the data processing module, optical network interface module The sent port status report information.

The InfiniBand port can be an InfiniBand switch, or an InfiniBand router, or an InfiniBand channel adapter.

The optical network port may be an add/drop multiplexer.

According to another aspect of the purpose of the above invention, the present invention provides a method for transmitting InfiniBand data in an optical network. At the InfiniBand data sending end, the InfiniBand data frame is used as an advanced data link control protocol data frame that can be transmitted by the optical network. The payload is encapsulated in the advanced data link control protocol data frame, and the data frame is sent to the optical network port, the data frame is transmitted to the optical network port of the data receiving end through the optical network, and the data receiving end will be encapsulated in the advanced The InfiniBand data frame in the data link control protocol data frame is taken out and sent to the data InfiniBand receiving end.

The payload of the data frame described in the method may further set a flow control message header for identifying a flow control state.

The method can use the credit-based flow control mechanism in the InfiniBand communication protocol to perform flow control.

It can be seen from the above technical solutions that the device and method for transmitting InfiniBand data in the optical network of the present invention utilize the widely used SDH/SONET network to realize reliable transmission of IB in a wide area. Compared with the prior art, the present invention well solves the problem of IB long-distance transmission, can realize IB-based server remote clustering and remote storage, and greatly expands the application range of IB.

Description of drawings

FIG. 1 is a schematic diagram of the network configuration of the prior art IB technology;

Fig. 2 realizes the block diagram of an embodiment of the equipment of transmitting InfiniBand data in optical network for the present invention;

Fig. 3 is the connection schematic diagram that the transmission equipment of embodiment shown in Fig. 2 connects IB network and SDH network;

Fig. 4 is the frame structure schematic diagram of the HDLC data frame used in the transmission method of the present invention;

FIG. 5 is a schematic structural diagram of an IB data frame.

Detailed ways

In order to make the object, technical solution and advantages of the present invention clearer, the present invention will be further described in detail below in conjunction with the embodiments and with reference to the accompanying drawings.

FIG. 2 is a block diagram of an embodiment of a device for transmitting InfiniBand data in an optical network according to the present invention. As shown in FIG. 2 , the transmission device is located between the InfiniBand port and the optical network port, and includes an InfiniBand interface module 201 , a mapping and flow control module 202 , a data processing module 203 , an optical network interface module 204 and a management module 205 .

Wherein, InfiniBand interface module 201 comprises InfiniBand interface circuit 2011, virtual channel sending buffer 2012 and virtual channel receiving buffer 2013; InfiniBand interface module 201 is connected with InfiniBand port by InfiniBand interface circuit 2011, InfiniBand interface circuit 2011 receives the data that InfiniBand port sends and sends It sends to the virtual channel sending buffer 2012, or receives the data sent by the virtual channel receiving buffer 2013 and sends it to the InfiniBand port; the virtual channel sending buffer 2012 temporarily stores the data received from the InfiniBand interface circuit 2011 and sends it to mapping and flow control Module 202; the virtual channel receiving buffer 2013 temporarily stores the data received from the mapping and flow control module 202 and sends it to the InfiniBand interface circuit 2011; in this embodiment, the virtual channel sending buffer 2012 also includes a virtual channel (VL) sending queue VL0 -VLn and the virtual channel sending multiplexing module VL MUX, the data received by the virtual channel sending buffer 2012 is first temporarily stored in the virtual channel sending queue VL0-VLn, and then sent to the mapping and flow control module 202 through the virtual channel sending multiplexing module VLMUX The virtual channel receiving buffer 2013 also further includes the virtual channel receiving queue VL0-VLn and the virtual channel receiving multiplexing module VL MUX, the data received by the virtual channel receiving buffer 2013 is temporarily stored in the virtual channel receiving queue VL0-VLn first, and then passed through The virtual channel receiving and multiplexing module VL MUX sends it to the InfiniBand interface circuit 2011. The virtual channel transmit buffer 2012 is also connected to the InfiniBand interface circuit by sending the transmit buffer status signal TXBS to the InfiniBand interface circuit; The modules 202 are connected, and the transmission device of the present invention implements flow control according to TXBS and RXBS.

The mapping and flow control module 202 receives the data sent by the virtual channel transmission buffer 2012 of the InfiniBand interface module 201, and according to the current state of the virtual channel transmission buffer 2012, adds corresponding flow control information to the InfiniBand data packet and sends it to the data processing module 203; Or receive the data sent by the data processing module 203 , extract the flow control information from the data packet of the data, and send the data packet without the flow control information to the InfiniBand interface module 201 .

The data processing module 203 receives the data sent by the mapping and flow control module 202, adds header information to the data packet of the data and calculates the checksum and sends it to the optical network interface module 204, or receives the data sent by the optical network interface module 204, and sends the data to the optical network interface module 204. The data packet of the data is sent to the mapping and flow control module 202 after removing the header information and checking and calculating the checksum.

The optical network interface module 204 receives the data sent by the data processing module 203 and sends it to the optical network port, or receives the data sent by the optical network port and sends it to the data processing module 203 .

The management module 205 is connected to other modules by sending management information to other modules and receiving reported information from other modules. The management module 205 sends port control information to the InfiniBand interface module 201, receives the port status report information sent by the InfiniBand interface module 201, sends flow control policy information to the mapping and flow control module 202, and receives the flow control status sent by the mapping and flow control module 202 Report information, send control information to the data processing module 203, receive data packet statistics report information sent by the data processing module 203, send port control information to the optical network interface module 204, and receive port status report information sent by the optical network interface module 204.

Fig. 3 is the connection schematic diagram of the transmission equipment of the embodiment shown in Fig. 2 connecting the IB network and the SDH network. As shown in Figure 3, one end of the IB transmission equipment 302 based on the optical network of the present embodiment is connected with the add/drop multiplexer 303 in the SDH ring network, and the other end is connected with the IB switch/IB router/IB channel adapter 301 of the IB network .

The present invention realizes the method for transmitting InfiniBand data in the optical network as follows: at the InfiniBand data sending end, the InfiniBand data frame is used as the payload of the high-level data link control protocol (HDLC) data frame that the optical network can transmit, and is encapsulated in the HDLC data frame In, and send the data frame to the optical network port, the data frame is transmitted to the optical network port of the data receiving end through the optical network, and the InfiniBand data frame encapsulated in the HDLC data frame is taken out and sent to the data InfiniBand receiving end at the data receiving end .

FIG. 4 is a schematic diagram of the frame structure of the HDLC data frame used in the implementation method of the present invention. As shown in FIG. 4 , the data frame is formed by adding a 6-byte flow control information header to the IB data frame, and encapsulating it into the HDLC data frame as the payload of the HDLC data frame.

Wherein, the IB data frame includes a header, an IB data payload, an invariant cyclic redundancy check (ICRC) and a variable cyclic redundancy check (VCRC). Its specific structure refers to Fig. 5, and Fig. 5 is the structural diagram of IB data frame, and as shown in Fig. 5, the information header of IB data frame includes local routing header information (LRH), global routing header information (GRH), basic transmission Layer header information (BTH), transport layer extension headers (ETHs) and immediate data (ImmData). See Table 1 for a brief description of each field of the IB data frame.

field Length (Bytes) illustrate LRH 8 Local Routing Header is the header information of the IB link layer, including the VL number, and each IB message has this field. GRH 40 Global Routing Header is the header information of the IB network layer, and only the packets that need to be forwarded between two subnets have this field. BTH 12 Base Transport Header is one of the header information of the IB transport layer, and whether to include it is determined according to different messages. ETHs 8 Transport layer extension header. ImmData 4 Immediate value, IB stipulates that the immediate value can be appended after the transport layer header and extension header of the last request message, which can be customized by the user. Payload 0~4096 Payload, the IB device can support a maximum of 4096 bytes, which needs to be negotiated. ICRC 4 The constant CRC check refers to the checksum of the field that will not be modified during the entire forwarding process of the message. The algorithm is CRC32, and whether to include it is determined according to different messages. VCRC 2 Variable CRC check, check the entire message, all messages have.

Table I

The other fields are described as follows:

◆Flag, occupying 1 byte: Flag is used to mark the start and end of a frame, and it is fixed at 0x7E. If there are two consecutive frames, only one Flag is needed between the previous frame and the next frame, instead of two consecutive 0x7E to distinguish consecutive frames; when there is no data to send, send consecutive Flag to SDH/SONET, that is, consecutive Flags are considered as an empty frame.

◆Address, occupying 1 byte: the address field is used to identify the destination address of an HDLC frame. When this field is 0xFF, it means broadcast address, and all nodes will receive the frame data. If the address is an invalid address, then the frame data will not be received or ignored.

◆Control (Control), occupying 1 byte: the Control field is used to identify the type of an HDLC frame, indicating that the frame is Information, Supervisory or Unnumbered in the HDLC protocol.

◆Protocol (Protocol), occupying 2bytes: the Protocol field is used to identify the protocol type in the Payload. In the present invention, it is used to indicate that the payload (Payload) contains SDH/SONET data based on IB. The definition source of this field According to the Internet standard—RFC1661, user-defined fields need to apply to the Internet Assigned Numbers Authority (IANA), the Internet Assigned Numbers Authority (IANA). In this embodiment, this field is tentatively set to 0xC031.

◆Flow Control Header (Flow Control Header), occupying 6 bytes: used to implement flow control between two devices that transmit InfiniBand data in the optical network. The method of the present invention uses the same credit-based flow control mechanism as the InfiniBand protocol.

◆Reserved (Reserved), occupying 1 byte: Reserved field for future expansion.

◆Checksum (FCS), occupying 4bytes: the checksum from the address field to the end of the Payload field.

The above method can be specifically realized by using the above SDH-based IB transmission equipment, and the process still refers to FIG. 2 . Wherein, the sending direction refers to the direction from InfiniBand to the SDH/SONET port, and the receiving direction refers to the direction from SDH/SONET to the InfiniBand port.

The sending process is as follows:

The first step: After the IB interface circuit 2011 of the IB interface module 201 receives the data from the IB port, according to the corresponding VL number of the data, the received data is put into the corresponding VL buffer queue VL0-VLn in the VL receiving buffer 2012 At the same time, according to the state TXBS of the VL buffer sent back to the IB interface circuit 2011 by the VL receiving buffer 2012, the VL-based flow control at the IB interface side is realized. The realization of the flow control mechanism conforms to the credit-based flow control adopted by the IB protocol According to the regulation of the mechanism, according to the scheduling mechanism sent by the management module 205, for example, a weighted round-robin scheduling method (WRR) is adopted between VLs, and the data in the VL buffer queue is output to the mapping and flow control module 202 through the VL MUX.

Step 2: After the mapping and flow control module 202 receives the data sent by the IB interface module, according to the current state of the VL receiving buffer, add the corresponding flow control information to the IB data packet, and then send the data to the data processing Module 203.

Step 3: the data processing module 203 receives the packet containing the flow control information, uses it as the payload of the HDLC data frame that the optical network can transmit, encapsulates it in the HDLC data frame, and completes the framing and checksum of the data Calculate the work, and then send the data to the SDH port through the SDH interface module 204.

The receiving process is as follows:

Step 1: The SDH interface module 204 receives data from the SDH port and sends it to the data processing module 203 .

Step 2: The data processing module 203 removes HDLC header information such as FLAG, Address, Control, and Protocol from the received data, checks the checksum and sends the processed data to the mapping and flow control module 202 .

Step 3: the mapping and flow control module 202 takes out the 6-byte flow control information from the data, sends the data packet with the flow control information removed to the IB interface module 201, and sends it back to the mapping and flow control information according to the VL sending buffer 2013 The VL of the flow control module 202 receives the state RXBS of the buffer, and implements flow control based on the VL.

Step 4: The IB interface module 201 puts the data into the corresponding position in the VL queue in the VL receive cache according to the different VL numbers of the received data, and according to the scheduling mechanism sent by the management module 205, for example, weighted polling is adopted between VLs Scheduling mode (WRR), the data in the VL buffer queue is output to the IB port through the IB interface circuit 2011 by the VL MUX.

It can be seen that the transmission equipment and transmission method based on the InfiniBand communication protocol of the optical network of the present invention utilize the widely used SDH/SONET network to realize reliable transmission of IB in a wide area. Compared with the prior art, the present invention well solves the problem of IB long-distance transmission, can realize functions such as server remote clustering and remote storage based on IB, and greatly expands the application range of IB.

Claims (13)

1. one kind is implemented in the equipment that transmits the InfiniBand data in the optical-fiber network, and it is characterized in that: this equipment comprises between InfiniBand port and optical network port

The InfiniBand interface module, it links to each other respectively with the Flow Control module with mapping with the InfiniBand port, is used to receive and dispatch the data of transmitting between InfiniBand port and mapping and the Flow Control module;

Mapping and Flow Control module, it is from InfiniBand interface module reception InfiniBand packet, this InfiniBand packet is added flow-control information after, send to data processing module; Or receive packet from data processing module, it is removed flow-control information after, send to the InfiniBand interface module;

Data processing module, it receives packet from mapping and Flow Control module, and sends to the optical network interface module after this packet added header, or receives packet from the optical network interface module, it is removed to send to behind the header shine upon and the Flow Control module;

The optical network interface module, its data that receive the data processing module transmission send to optical network port, or the data that the reception optical network port sends send to data processing module;

Administration module, it links to each other with above-mentioned each module with the reporting information that receives above-mentioned each module by sending management information to above-mentioned each module.

2. transmission equipment as claimed in claim 1 is characterized in that: described InfiniBand interface module comprises the InfiniBand interface circuit, buffer memory is sent in the Virtual Channel transmitting-receiving; The data that InfiniBand interface circuit reception InfiniBand port sends also send it to Virtual Channel transmission buffer memory, or receive the data of Virtual Channel reception buffer memory transmission and send it to the InfiniBand port; Virtual Channel sends buffer memory will send to mapping and Flow Control module after the data that the InfiniBand interface circuit receives are kept in; Virtual Channel receives buffer memory will send to the InfiniBand interface circuit after the data of mapping and the reception of Flow Control module are kept in.

3. transmission equipment as claimed in claim 2, it is characterized in that: described Virtual Channel sends buffer memory and further comprises Virtual Channel transmit queue and Virtual Channel transmission Multiplexing module, Virtual Channel sends the data of buffer memory reception and keeps in earlier in the Virtual Channel formation, sends to mapping and Flow Control module by the Virtual Channel Multiplexing module again.

4. transmission equipment as claimed in claim 2, it is characterized in that: described Virtual Channel receives buffer memory and further comprises Virtual Channel reception formation and Virtual Channel reception Multiplexing module, Virtual Channel receives the data of buffer memory reception and keeps in the Virtual Channel formation earlier, sends to the InfiniBand interface circuit by the Virtual Channel Multiplexing module again.

5. transmission equipment as claimed in claim 2 is characterized in that: described Virtual Channel sends buffer memory and further links to each other with the InfiniBand interface circuit by sending the transmission buffer status signal to the InfiniBand interface circuit; Described Virtual Channel receives buffer memory and further links to each other with the Flow Control module with mapping by sending the reception buffer status signal to mapping and Flow Control module.

6. transmission equipment as claimed in claim 1 is characterized in that: described data processing module further comprises data computation verification and module, its to the data computation verification that receives from mapping and Flow Control module with; With check calculation check and module, its to the data checks calculation check that receives from the optical network interface module and.

7. transmission equipment as claimed in claim 1 is characterized in that: described administration module comprises to the management information that other each module sends: the Flow Control policy information that sends to the control information of InfiniBand interface module transmit port, to mapping and Flow Control module, the control information that sends to data processing module, to the control information of optical network interface module transmit port.

8. transmission equipment as claimed in claim 1 is characterized in that: the reporting information that described administration module receives other each module comprises: the port status reporting information that the packet statistical report information that the Flow Control state reporting information that the port status reporting information that the InfiniBand interface module sends, mapping and Flow Control module send, data processing module send, optical network interface module send.

9. transmission equipment as claimed in claim 1 is characterized in that: described InfiniBand port is the InfiniBand switch, or the InfiniBand router, or the InfiniBand channel adapter.

10. transmission equipment as claimed in claim 1 is characterized in that: described optical network port is an add-drop multiplexer.

11. method that is implemented in transmission InfiniBand data in the optical-fiber network, it is characterized in that: at the InfiniBand data sending terminal, the payload of the High level data link control Frame that the InfiniBand Frame can be transmitted as optical-fiber network, be encapsulated in the High level data link control Frame, and this Frame sent to optical network port, this Frame is transferred to the optical network port of data receiver by optical-fiber network, will be encapsulated in InfiniBand Frame in the High level data link control Frame at data receiver and take out and send to data I nfiniBand receiving terminal.

12. method as claimed in claim 11 is characterized in that: the payload of described High level data link control Frame further is provided with a Flow Control message header that is used to identify the Flow Control state.

13. as claim 11 or 12 described methods, it is characterized in that: the flow-control mechanism based on credit rating in this method employing InfiniBand communication protocol carries out Flow Control.

Images