JP4930554B2 - Input/Output Control Unit - Google Patents

Description

The present invention relates to an input/output control device for transmitting and receiving data within a computer network, and in particular to a device for receiving, generating, and transmitting frame data across computer network boundaries.

There has been a great technological advancement in the field of ultra-high speed data links. High performance computers are the focus of the data communications industry. Performance requirements and improvements have given rise to data-intensive, high speed network applications such as multimedia, scientific visualization, and network expansion design. And there is a demand for ever faster network interconnections between computers and I/O devices.

Fibre Channel (FC) was developed to provide a practical, inexpensive, and scalable means for quickly transferring data between workstations, mainframes, supercomputers, desktop computers, storage devices, network servers, and other peripheral devices. Fibre Channel is the general name for a set of standards that consolidates standards created by the American National Standards Institute (ANSI), and related specifications are available at http://www.t11.org/ and elsewhere. In order to enable even faster data links, the Institute of Electrical and Electronics Engineers (IEEE) developed 802.3ae in 2002, which achieves a link speed of 10 Gbps, and 10GFC, based on this standard, is currently being standardized by the IEEE.

As a means of connecting this Fibre Channel to a host processor, a configuration using the Peripheral Component Interchange (PCI) standard established by PCI-SIG is widely used, and various vendors are providing Fibre Channel Host Bus Adapters (HBAs) on the market. In addition, these HBAs are generally equipped with a protocol processor that interprets the Fibre Channel protocol and controls data transfers with the main memory device (MS).

Recently, the market has been demanding more efficient use of the host bus. For example, in the aforementioned PCI host bus, the number of connections is regulated for each bus segment, and in order to use the PCI bus more efficiently, it is necessary to connect multiple fibre channels to one HBA. In HBAs available on the market, one method for implementing multiple fibre channels on one adapter is to implement two fibre channels on a host adapter by installing an I/O control device with a single interface and a single protocol processor, as described in Patent Document 1, for two channels, and further by installing a PCI bridge to separate the bus segment on the system side from that in the HBA.

There is also method 2, which takes advantage of technological advances in highly integrated LSIs to integrate multiple completely independent fibre channel control circuits into a single LSI. With this method, it is only necessary to integrate independent logic circuit cores into a single LSI.

Japanese Patent Application Publication No. 5-334223

However, in applying the conventional technology, the following problems are raised in order to satisfy further functional demands from the market.
(Challenge 1)
In response to market demand for more ports, for example, to realize a single adapter with four ports, the conventional method 1 obviously tends to increase the number of parts. This increases the wiring density of the target adapter board, making design more difficult and increasing the board cost, and there is also concern that the failure rate of the adapter as a whole may worsen. In addition, the latency of requests increases due to the use of a PCI bridge, which may have a negative impact on data transfer performance.
(Challenge 2)
On the other hand, according to method 2, since the multiple fibre channel functions are completely independent logic circuit cores, each core wastes processing capacity. In general, input/output processing is not always performed while maintaining peak performance, and there are periods when no processing is performed and periods when peak performance is required. For this reason, the protocol processor processes only about 50% of the total over a long period of time (there is a lot of idle time). However, if the performance of the protocol processor is about 50% of its peak performance, it takes a long time when the input/output processing requires peak performance, so it is usually designed to meet peak processing demands. However, although about 50% of peak performance is sufficient on average, the protocol processor maintains peak performance that occurs temporarily.

For example, consider the case where two completely independent logical cores are integrated into one LSI for two fibre channel functions. The combined processing capacity of these two protocol processors is twice the peak performance of one fibre channel function in the above configuration. In other words, because they are completely independent, the processing load cannot be distributed between the two protocol processors, and on average, there is surplus processing capacity.
(Challenge 3)
As described in the prior art, a high-speed technology for a single fiber channel interface (10GFC) is being standardized, and in order to realize this, the processing capacity of the protocol processor must be at least five times that of 2 Gbps, in terms of a simple multiple of the transfer rate. Both method 1 and method 2 described in the prior art cannot be realized by simply mounting multiple elements, and it is necessary to introduce or develop a processor capable of high-speed processing. Therefore, the transferability of processing from 2 Gbps to 10 Gbps is also an issue that must be considered.

The objective of the present invention is to provide a flexible input/output control device that supports multiple 2 Gbps fiber channels and takes into consideration the transition to 10 Gbps fiber channels in order to solve the above problems.

The input/output control device of the present invention has multiple fibre channel interfaces within a single device and an interface control circuit that can independently control the interfaces.

Furthermore, in order to interpret and process protocols that are used on multiple Fibre Channel interfaces, the protocol processing circuit is provided with a port identification means that allows the protocol processing circuit to identify the port of the target interface when a frame is received, etc., and conversely, a port designation means that allows the protocol processing circuit to designate the port of the target interface when a frame is transmitted, etc.

Furthermore, by using the port identification means and the port designation means, when a 2 Gbps fibre channel with multiple ports is operated, protocol processing can be performed for each port, and a single protocol processing circuit is provided that can ensure the necessary performance even when a 10 Gbps fibre channel is operated.

According to the present invention, it is possible to provide an input/output control device that has a multi-port fibre channel interface, has a small number of parts, does not occupy a physical host bus, and allows for flexible distribution of protocol processing capabilities. It is also possible to provide an input/output control device that is compatible with a high-speed fibre channel interface. It is also possible to provide an input/output control device that is flexible in its compatibility with long-distance transmission.

1 is a configuration diagram of an input/output control device to which an embodiment of the present invention is applied; FIG. 2 is a diagram showing the configuration of a receive data buffer used in an embodiment of the present invention. FIG. 2 is a diagram showing the configuration of a receive data stack used in one embodiment of the present invention. FIG. 1 is a diagram showing a frame format used in Fibre Channel. FIG. 2 is a diagram showing the configuration of a transmission data buffer used in one embodiment of the present invention. FIG. 2 is a diagram showing the configuration of a transmission data stack used in one embodiment of the present invention. 1 is a diagram showing the configuration of an entire system equipped with an input/output control device realized by the present invention;

The circuit and its operation of one embodiment of the present invention will be explained below with reference to the drawings.

Figure 7 shows an example of the overall configuration of a system equipped with an input/output control device realized by the present invention. The host 1000 is composed of one or more central processing units 1001, a host bus controller 1002, a main memory device 1003, and a PCI/PCI-X bridge 1004, and the disk control device 2000 is composed of a disk controller 2002, a disk device 2001, and a PCI/PCI-X bridge 2003. The host 1000 and the disk control device 2000 are connected to an input/output control device 1 and an input/output control device 2, respectively, and data can be exchanged via a fibre channel interface.

Figure 1 shows an embodiment of an input/output control device that best illustrates the features of the present invention. This input/output control device has four ports of low-speed fibre channel interfaces 101, 201, 301, 401 (2 Gbps is assumed in this embodiment, and hereafter referred to as the 2 Gbps fibre channel interface), and one port of high-speed fibre channel interface 501 (10 Gbps is assumed in this embodiment, and hereafter referred to as the 10 Gbps fibre channel interface). At this time, the 2 Gbps fibre channel interface and the 10 Gbps fibre channel interface can only operate exclusively. Also, a PCI/PCI-X interface is used to exchange information with the host 1000, which includes a central processing unit 1001, a main memory unit 1003, etc.

The other ends of the fibre channel interfaces are connected to their respective communication destinations via optical fibre cables, and the four 2 Gbps fibre channel interfaces are controlled by fibre channel interface control circuits 100, 200, 300, 400 which operate independently and asynchronously to control the fibre channel interfaces. The 10 Gbps fibre channel interface is connected to a fibre channel interface control circuit 500 and is capable of supporting link speeds faster than 2 Gbps. This input/output control device also uses a single protocol processing circuit 600 to collectively handle protocol processing on multiple fibre channel interfaces.

First, the operation when data is received from the Fibre Channel interface will be explained. Normally, an I/O operation request generated by an operating system (OS) or application running on a central processing unit included in the host 1000 is transmitted to an I/O control device via a PCI/PCI-X interface or the like. Various means are provided for this transmission, and details will not be described here, but in this embodiment, an I/O operation request is conveyed to a protocol processing mechanism.

Figure 4 shows the frame format of the fibre channel that is recognized and assembled by the fibre channel interface control circuits 100, 200, 300, and 400. The frame is composed of SOF and EOF, which are ordered sets for identifying frame delimiters, a header containing various information related to the frame, a CRC (Cyclic Redundancy Check) for error detection, and a payload whose validity is guaranteed by the CRC. The data received by each fibre channel interface control circuit passes through a serializer/deserializer (SerDes) that converts serial data received via an optical transceiver into parallel data, and then through a frame analysis circuit 105 that recognizes ordered sets from the parallel data, assembles frames, and detects errors using CRC for error detection, etc., before being prepared for writing to the receive data buffer 21. The fibre channel interface control circuits 100, 200, 300, 400 count the data length recognized as a frame while sending the payload portion, and after sending the last payload portion, send the header portion and the counted payload length, etc. to the receiving data lines 102, 202, 302, 402.

When the receive buffer control circuit 20 recognizes a write request on the receive data lines 102, 202, 302, and 402, it selects an available RLR number from the receive data buffer 21. If multiple receive data lines 102, 202, 302, and 402 are sending requests at the same time, the write requests are processed sequentially by a method such as determining the port to be serviced for each operation cycle. This write processing capacity is sufficient to satisfy a link speed of 10 Gbps, and of course, there is no problem even if four ports operate at 2 Gbps simultaneously. The receive data buffer 21 is configured as shown in Figure 2, and data is written to one of the RLRs 2000 to 2255 corresponding to each frame. When writing from a certain port is completed, the receive buffer control circuit 20 writes the number identifying the port and the RLR number in which the frame data received from the port is stored into the receive data stack 22 (Figure 3), and sets the busy bit 29 indicating the usage status of the corresponding RLR to a busy state. The busy bit 29 is set to an empty state when the processing of the frame stored in the corresponding RLR number is completed by the protocol processing circuit 600. The receive buffer control circuit 20 is connected to a 10GFC flag line 601 from the protocol processing circuit, and when the 10GFC flag line 601 is '1', a means is provided for suppressing write requests from the 2 Gbps Fibre Channel interface control circuit.

The receive data stack 22 shown in FIG. 3 has a FIFO configuration with an input pointer 23 and an output pointer 24, and in the initial state, the input pointer and the output pointer point to the same stack position. In this embodiment, 256 stacks are possible. The input pointer 23 is updated when the frame reception is completed. This causes the mismatch detection circuit 25, which checks the state of the input pointer 23 and the output pointer 24, to be established, and the interrupt pending register 26 is set. At the same time, the contents of the stack indicated by the output pointer 24 are read into the receive port identification register RPORT 27 and the receive buffer plane identification register RLR 28, and the output pointer 24 is updated by one stack. At this time, while the interrupt pending register 26 is set, updates to the receive port identification register RPORT 27, the receive buffer plane identification register RLR 28, and the output pointer 24 are suppressed, which serves to prevent the registers and the output pointer from being erroneously updated.

When the interrupt pending register 26 is set, the protocol processing circuit 600 recognizes that a new frame has been received, and by reading the receiving port identification register RPORT 27 and the receiving buffer plane identification register RLR 28, it can identify which port of the multiple fiber channel interfaces the frame came from and which RLR number in the receiving data buffer 21 the frame is stored in. The protocol processing circuit 600 has a means for clearing the interrupt pending register 26, and clears the interrupt pending register 26 using the clearing means after saving the contents of the receiving port identification register RPORT 27 and the receiving buffer plane identification register RLR 28 to local storage, etc. If multiple frames are reserved in the receiving data stack 22, the interrupt pending register 26 is set again. The protocol processing circuit 600 performs the processing required for the protocol from the port number and RLR number obtained here, and if data transfer to the host is required, it can instruct the DMA control circuit 900 of the address in the receiving data buffer, the main memory address, and the data transfer length indicated by the RLR number. The DMA control circuit 900, which has been instructed to transfer data, identifies the data position in the receive data buffer 21 from the RLR number, and starts the data transfer with the host 1000 via the PCI/PCI-X control circuit 800. By providing a means for identifying the port number and RLR number when an interrupt is pending, it becomes possible for the common protocol processing circuit 600 to perform protocol processing across multiple ports, and furthermore, because frame reception reports are made in the order in which the frames arrive, the service period for each port becomes longer as the frame multiplexing increases, and load balancing for input/output requests can be achieved obviously.

Furthermore, by sharing the receive data buffers related to multiple fiber channel interfaces, a flexible buffer configuration can be provided. That is, a register RFCNT106 is provided in each fiber channel interface control circuit to define the number of receive data buffers that the port can use, and the number of receive data buffers allowed for each port is set in the RFCNT within the range of the total number of buffer planes when the I/O control device is initialized. In the embodiment of the present invention, the number of buffer planes is 255, so for example, port 0 can be set to 128 planes, port 1 to 64 planes, and ports 2 to 3 to 32 planes. Alternatively, if only ports 0 to 1 are operated, 128 planes each can be set for ports 0 to 1, or if only port 0 is operated, 256 planes can be set for port 0. By providing this means, it is also possible to provide the number of buffer credits required for long-distance transmission without changing the HW.

Next, we will explain the operation when sending data to the Fibre Channel interface.

As in the case of a receiving operation, the protocol processing circuit 600 that recognizes an input/output operation request must first secure a buffer in the transmission data buffer 11 and store the data to be transmitted in the buffer in order to transmit data to the fiber channel interface. The buffer may be secured by busy management within the protocol processing circuit, or a buffer number setting in the transmission buffer plane designation register TLR12 and a busy state setting command may be sent to the transmission buffer control circuit 10. Either method does not define the configuration of the present invention, and it is sufficient that a busy flag corresponding to the buffer plane is provided. There are two methods for writing data to the transmission data buffer 11: Method 1, in which the protocol processing circuit 600 generates target data and performs the process via the processing circuit write data line 602, and Method 2, in which the protocol processing circuit 600 issues a data transfer instruction and the DMA control circuit 900 performs the process via the DMA write data line 902. In Method 2, when the data transfer instructed by the protocol processing circuit 600 is completed, the DMA control circuit 900 has a means for reporting completion (such as an interrupt). The transmission data buffer 11 is divided into buffer planes as shown in FIG. 5, and has a capacity to store the payload portion in the Fibre Channel frame format in FIG. 4, for example. In addition, a separate storage means corresponding to the number of buffer planes in the transmission data buffer 11 is provided for the SOF, EOF, and header information in the frame format, and a write means from the protocol processing circuit 600 to the storage means is also provided.

When the data to be transmitted is prepared, the protocol processing circuit 600 stores the buffer number of the transmission data buffer 11 in the transmission buffer plane designation register TLR12, and stores the port number of the fiber channel interface to transmit the contents of the transmission data buffer in the transmission port designation register TPORT13. The port address decoder (DEC) 14 at the rear of the transmission port designation register TPORT13 is composed of decode logic with enable, and when the protocol processing circuit 600 sends a trigger signal to the transmission command line 603, the decode logic is enabled, and an actual transmission instruction signal is sent to one of the transmission command decode lines 17. Therefore, by setting the transmission port designation register TPORT13, one protocol processing circuit 600 can execute transmission instructions to multiple fiber channel interfaces. In addition, the 10GFC flag line 601 from the protocol processing circuit is connected to AND 15, which is located in front of the enable signal input of the port address decoder (DEC) 14, and when the 10GFC flag line 601 is '1', the transmission command decode line 17 to the four 2 Gbps Fibre Channel interface control circuits is always disabled.

One of the fibre channel interface control circuits that receives the transmission command decode line 17 sent in the above process stores the contents of the transmission buffer plane designation register TLR 12 in the transmission data stack 111 in the format shown in FIG. 6. The transmission data stack 111 is configured as a FIFO, and transmission requests are generated sequentially in the same way as the reception buffer control circuit 20, and the TLR number stored in the transmission data stack 111 is transmitted to the frame generation circuit 110. The frame generation circuit 110 is connected to a transmission data read line 18 for reading data to be transmitted from the transmission data buffer 11, and is also provided with means for reading SOF, EOF, and header information. It also generates a CRC code and assembles them to generate the frame shown in FIG. 4. The generated frame is converted from parallel data to bit serial data by a serializer/deserializer (SerDes), and is sent to the other end of the fibre channel interface via an optical transceiver.

As shown in FIG. 1, the transmission data read line 18 is shared by four 2 Gbps Fibre Channel interface control circuits 100, 200, 300, and 400, and when transmission commands instructed by the protocol processing circuit 600 are issued continuously, data to different ports is sent out on the transmission read data line 18 while being alternated in chronological order. As with the writing of received data to the received data buffer 21, the transmission data buffer 11 has a configuration that satisfies a link speed of 10 Gbps and has a configuration that does not cause any problems even if 2 Gbps reads from four ports simultaneously.

When frame transmission is completed, the fibre channel interface control circuits 100, 200, 300, and 400 release the transmission data buffer. The means for releasing the transmission data buffer is to set the busy flag for each transmission data buffer surface described above to '0'. The released transmission data buffer surface is reused for frame transmission by the protocol processing circuit 600 for any port.

The above explanation of the operation and configuration of one embodiment of the present invention has been mainly given for the case where a 2 Gbps Fibre Channel interface operates simultaneously on four ports. A sublayer circuit such as XGXS defined in the IEEE 10 Gbps specification is incorporated into the 10 Gbps Fibre Channel interface control circuit to operate faster than at 2 Gbps. However, the basic configuration is the same, and the only difference is that the 10GFC flag line 601 from the protocol processing circuit 600 is set to '1', and there is no access contention to the buffer that occurs during 2 Gbps multi-port operation, and the protocol processing circuit 600 can allocate all of its processing capacity to one 10 Gbps Fibre Channel port.

1 I/O control device connected to host device 2 I/O control device connected to disk control device 10 Transmit buffer control circuit 11 Transmit data buffer 12 Transmit port designation register 13 Transmit buffer plane designation register 20 Receive buffer control circuit 21 Receive data buffer 22 Receive data stack 27 Receive port identification register 28 Receive buffer plane identification register 100, 200, 300, 400 Fibre channel interface control circuits for 2 Gbps 101, 201, 301, 401 2 Gbps fibre channel interface 500 Fibre channel interface control circuit for 10 Gbps 501 10 Gbps fibre channel interface 600 Protocol processing circuit 800 PCI/PCI-X control circuit 900 DMA control circuit 1000 Host device 2000 Disk control device

Claims (2)

an input/output control device in which an input/output operation request generated by a host is transmitted via a PCI interface, and which transmits and receives data related to the input/output operation request to and from the input/output device in frame units via a Fibre Channel interface , the input / output control device comprising: a plurality of interface control circuits independently controlling the Fibre Channel interface for each port; a receive buffer control circuit storing frame data received by the plurality of interface control circuits in a common receive data buffer; a common transmit data buffer storing frame data to be sent to each interface control circuit; a single protocol processing circuit processing transmit and receive data at each port via the plurality of interface control circuits; a transmit port designation register for the protocol processing circuit to designate an interface control circuit to which frame data should be transmitted; a decode circuit for generating a transmit instruction signal to the specified interface control circuit from the content of the transmit port designation register when the protocol processing circuit issues a transmit instruction; and a transmit buffer address register for the protocol processing circuit to set buffer address information of the transmit data buffer in which the frame data to be transmitted is stored, the interface control circuit comprising a transmit stack for reading and storing the content of the transmit buffer address register. 2. The input/output control device according to claim 1, wherein said interface control circuit comprises a frame generation circuit which sequentially reads out the transmission buffer address information stored in said transmission stack, reads out the corresponding transmission data from said transmission data buffer, and generates a transmission frame.

Images