HK1182561B - Embedded end-to-end delay information for data networks - Google Patents

Description

Embedded end-to-end delay information for data networks

Cross reference to related applications

This application is related to the following co-pending U.S. patent applications, all of which are incorporated herein by reference:

U.S. patent application serial No. 13/073,260 (attorney docket No. H0028046-5409), entitled "universal source port implementation for data networks," filed 3/28/2011, and referred to herein as the "046 application"; and

U.S. patent application serial No. 13/073,269 (attorney docket No. H0028047-5409), entitled "centralized traffic shaping for data networks," filed 3/28/2011, and referred to herein as the "048 application.

Background

Some conventional data networks use virtual links. For example, ARNIC664 section 7 defines a shaped ethernet network that allows traffic flow between endpoints of a full duplex switched ethernet network to be analyzed with respect to transmission timing. At the ethernet network level, virtual linking is implemented by a locally administered multicast group with a network-wide unique multicast ethernet address. I.e. all frames of a virtual link use the same ethernet multicast destination address, while different frames of a virtual link use different ethernet multicast destination addresses. At the ethernet network level, the virtually linked frames can thus be identified by their destination ethernet address.

Disclosure of Invention

In one embodiment, a system is provided. The system includes a plurality of nodes, at least one of the plurality of nodes configured to insert a delay value on a per virtual link basis into a dynamic delay field of a frame corresponding to a respective virtual link, wherein the dynamic delay value represents a latency of the frame of the respective virtual link. The system also includes a switch having a plurality of ports, each port coupled to one of the plurality of nodes. The switch is configured to route frames received from the plurality of nodes to one or more of the plurality of nodes. At least one of the plurality of nodes is configured to store frames received from the switch in a buffer and update a value in a dynamic delay field to reflect an end-to-end system delay.

Drawings

Understanding that the drawings depict only exemplary embodiments and are not therefore to be considered to be limiting in scope, the exemplary embodiments will be described with additional specificity and detail through use of the accompanying drawings, in which:

FIG. 1 is a block diagram of one embodiment of a system.

FIG. 2 is a block diagram of one embodiment of an exemplary frame.

FIG. 3 is a flow chart depicting one embodiment of a method of transmitting a frame.

In accordance with common practice, the various described features are not drawn to scale, but are drawn to emphasize specific features relevant to the exemplary embodiments.

Detailed Description

In the following detailed description, reference is made to the accompanying drawings that form a part hereof, and in which is shown by way of illustration specific illustrative embodiments. However, it is to be understood that other embodiments may be utilized and that logical, mechanical and electrical changes may be made. Furthermore, the methods presented in the figures and description are not to be construed as limiting the order in which individual actions may be performed. The following detailed description is, therefore, not to be taken in a limiting sense.

FIG. 1 is a block diagram of one embodiment of an exemplary system 100. System 100 includes a plurality of nodes 102-1.. 102-N (also referred to as end systems) and at least one switch 104. System 100 is configured to use virtual links to transfer frames between nodes 102-1.. 102-N through switch 104. As used herein, a virtual link is a unidirectional logical path connecting two or more nodes 102 through a switch 104. For example, in some embodiments, the system 100 is a full duplex switched ethernet network configured to implement a protocol compatible with part 7 of the aeronautical radio corporation (ARINC) standard 644, also known as avionics full duplex switched ethernet (AFDX). In some embodiments, the virtual links are limited to have one and only one source node 102 as defined in section 7 of the ARNIC standard 664. However, in other embodiments, the switch 104 is configured to accept multiple nodes 102 as active source nodes for a single virtual link as described in the' 046 application.

Each node 102 is communicatively coupled to a respective subsystem 120. The implementation of each subsystem 120 depends on the implementation of the system 100. For example, in this example, system 100 is implemented as an avionics system. Thus, each subsystem 120 is implemented as, but not limited to, one of: in-flight computer systems, navigation systems, Global Navigation Satellite Systems (GNSS), and the like. Accordingly, the subsystems 120-1 and 120-N are each coupled to one or more sensors 126 and one or more actuators 128 corresponding to respective implementations of the subsystems 120-1 and 120-N. Further, in the present embodiment, subsystem 120-2 is implemented as a gateway coupled to another network 130 (such as the Internet).

Each subsystem 120 provides data to a respective node 102. In addition, each subsystem 120 includes one or more higher level applications 125 implemented at the application layer. The term "application layer" is well known to those skilled in the art and, as used herein, refers to programs and services that implement high-level functionality for accomplishing tasks on the network, such as implementing protocols for specific user applications. Each node 102 in turn processes and outputs data through one or more corresponding virtual links. In particular, each node 102 includes a respective controller or processing unit 124 configured to insert one or more delay values into fields of a corresponding ethernet frame based on corresponding delay rules 122 stored in a memory 132 of the respective node 102 on a per virtual link basis. The one or more delay values represent the delay or latency experienced by the corresponding ethernet frame while residing in the node 102 or being transmitted from the node 102 to another device in the system 100. For example, in some embodiments, the delay value is a dynamic value measured by the respective node 102, such as, but not limited to, a queuing delay of a frame in an output queue. Further, in some embodiments, the delay value is a static value that is configured a priori and stored in the node 102, such as, but not limited to, the transmission delay on the line from a transmitting port of the node to a receiving port of another device in the system 100.

Further, the controller 124 is configured to determine which delay value or values to add to the corresponding frame on a per virtual link basis based on the delay rules 122. For example, one or more static delay values may be selected based on the content of the frames corresponding to the respective virtual links. That is, the one or more static delay values may be selected based on one or more of a virtual link ID, an Internet Protocol (IP) source address, an IP destination address, a User Datagram Protocol (UDP) source port, a UDP destination port, or other fields contained in the ethernet frame payload. In some embodiments, the controller 124 is further configured to update a checksum (also referred to as a Frame Check Sequence (FCS) or Cyclic Redundancy Check (CRC)) based on the inserted delay value. In some embodiments, such as embodiments implementing fragmented frame grouping, it should be understood that the checksum may reside in a different frame than the inserted delay value. Further, in some embodiments, the delay value is only added to a particular frame of the virtual link, such as the first fragment of a fragmented UDP packet, based on the delay rules 122.

After inserting the delay values and performing other processing on the frames, such as policing to ensure that the virtual links comply with Broadband Allocation Gap (BAG) requirements, node 102 sends the frames of the virtual links to switch 104. The switches 104 receive the frames through the corresponding ports 106-1.. 106-N. Switch 104 processes each received frame in processing unit 114. For example, processing unit 114 is configured to determine whether the frame is received on an active port of a corresponding virtual link for the frame. Further, based on routing table 118 stored in memory 116, processing unit 114 routes valid received frames to one or more ports 106-1.. 106-N for output to one or more nodes 102-1.. 102-N.

Further, as described above with respect to node 102, in this embodiment, processing unit 114 is configured to insert one or more delay values into fields (also referred to as delay fields) of the received frame on a per virtual link basis based on delay rules 123 stored in memory 116. For example, in some embodiments, the delay value to be inserted into a given frame is determined based on the port 106 through which the frame is received. Furthermore, in some embodiments, processing unit 104 is configured to insert delay values into frames corresponding to certain virtual links, rather than frames corresponding to all virtual links. In some embodiments, the delay values inserted into the frame represent a range of values, rather than specific values.

The processing unit 114 includes or functions with: software programs, firmware, or other computer readable instructions for performing various methods, process tasks, calculations, and control functions are used to insert delay values into fields of ethernet frames on a per virtual link basis. The instructions are typically stored on any suitable computer readable medium for storing computer readable instructions or data structures. The computer-readable media can be implemented as any available media that can be accessed by a general purpose or special purpose computer or processor, or any programmable logic device. Suitable processor-readable media may include memory or storage media, such as magnetic or optical media. For example, the memory or storage medium may include a conventional hard disk, a compact disk-read only memory (CD-ROM), a volatile or non-volatile medium such as Random Access Memory (RAM) (including, but not limited to, Synchronous Dynamic Random Access Memory (SDRAM), Double Data Rate (DDR) RAM, RAMBUS Dynamic RAM (RDRAM), Static RAM (SRAM), and the like), Read Only Memory (ROM), Electrically Erasable Programmable ROM (EEPROM), flash memory, and the like. Suitable processor-readable media may also include transmission media such as electrical, electromagnetic, or digital signals conveyed via a communication medium such as a network and/or a wireless link.

As discussed above, the processing unit 104 in the switch routes the received frame to the node 102 based on the routing table 118. The received frames are placed in the buffer 127 at the respective node until accessed by a higher level application 125 in the corresponding subsystem 120. The receiving node 102 inserts the dynamic timestamp into the delay field at the physical layer. The term "physical layer" is well known to those skilled in the art and, as used herein, refers to the physical hardware components of a system, such as, but not limited to, transmission media, pins, network adapters, host bus adapters, circuit elements, and the like. Further, the term "dynamic timestamp" is used herein to refer to a timestamp that is updated to reflect changes in delay time. For example, the controller 124 inserts a timestamp into the dynamic delay field by using a component in the physical layer when the corresponding message is placed into the buffer 127. In addition, the processing unit updates the timestamp, such as when other events occur. For example, the timestamp in the dynamic delay field can be updated at certain intervals or when a message is accessed by an application/service 125 operating in the application layer.

When an application at the application layer reads or writes a message stored in buffer 127, controller 124 updates the value in the dynamic delay field. Thus, the dynamic delay field is continuously updated to provide real-time end-to-end system delay to applications 125 residing at the application layer. Real-time end-to-end system delay is the delay associated with the source from the time the message is sent to the time the message is accessed. Thus, as described above, although a static value may be initially inserted into the delay field by the corresponding node 102, the static value may be updated to reflect the real-time delay.

Fig. 2 is a block diagram illustrating an exemplary frame 200 transmitted between node 102 and switch 104. As is well known to those skilled in the art, the frame 200 includes a preamble 202, a start of frame (SOF)204, and an intra-frame gap (IFG) 234. Further, frame 200 includes an ethernet frame 206. The ethernet frame includes a header 208 and a payload 210. The ethernet header 208 is comprised of a destination address 212 and a source address 214. The destination address 212 is comprised of a constant field 216 and a virtual link ID 218. The virtual link ID is used to route the frame in the switch by using a routing table that indicates which output ports the virtual link is associated with. The ethernet header 208 also includes a type field 220 as is well known to those skilled in the art.

The ethernet payload 210 includes a delay field 222, an IP header 224, and an IP payload 226. Node 102 and/or switch 104 are configured to insert a delay value into delay field 222 as described above. Notably, the location of the delay field 222 in the ethernet payload 210 of fig. 2 is provided by way of example and not limitation. In particular, the location and/or length of the delay field 222 may be dynamically determined by checking the contents of the frame against rules (e.g., delay rules 122) stored in the node 102 or switch 104 as the frame passes through the apparatus. Further, in some implementations, the stored rules indicate which delay values to insert based on one or more of a virtual link ID, an IP address, or a UDP port number. Thus, it is determined for each virtual link ID individually whether to insert a static delay value, or a dynamic delay value, or a combination of static and dynamic delay values, and the value or range of values to be inserted into the delay field 222.

In this example, the IP payload 226 consists of a UDP header 228 and a UDP payload 230, as is well known to those skilled in the art. In addition, ethernet frame 206 includes a frame check sequence 232. As described above, in some embodiments, FCS232 is updated to reflect the delay value inserted into delay field 222.

Fig. 3 is a flow chart depicting one embodiment of a method 300 of transmitting frames in a system such as system 100 described above. At block 302, a delay value is inserted at the source node on a per virtual link basis into the delay field of the corresponding frame of the respective virtual link, as described above. The delay value represents the latency of the corresponding virtually linked frame. As used herein, a source node is a node designated as the source of a frame for a corresponding virtual link. Thus, as used herein, a destination node is a node for a respective virtual link that is designated as the node to which frames are routed from a corresponding source node.

In some embodiments, inserting the delay value into the delay field includes inserting at least one of a dynamic delay value and a static delay value, as described above. Further, in some embodiments, inserting the delay value includes updating a checksum in the corresponding frame based on the inserted delay value, as described above.

At block 304, the corresponding frame with the delay field is transmitted from the source node to the switch. In some embodiments, as described above, at block 306, the switch is configured to insert a delay value into the delay field. In such embodiments, inserting the delay value may include adding a static value, a dynamic value, or a combination of static and dynamic values to the received value contained in the delay field and replacing the received value with the result of adding a value to the received value. Further, inserting the delay value includes updating any checksums affected by the inserted delay value. In other embodiments, the switch does not insert a delay value. At block 308, the corresponding frame is routed to one or more destination nodes based on the respective virtual links. In this embodiment, the frames are transmitted and routed using a protocol compatible with the ARINC644, part 7 standard. However, it should be understood that other protocols may be used in other embodiments.

At block 310, the value in the delay field is updated to reflect the real-time end-to-end system delay. For example, the delay field may be updated upon the occurrence of a particular event, such as the frame being accessed by an application in the application layer, as discussed above.

Example embodiments

Example 1 includes a system comprising a plurality of nodes, at least one node of the plurality of nodes configured to insert, on a per virtual link basis, a delay value into a dynamic delay field of a frame corresponding to a respective virtual link, wherein the dynamic delay value represents a latency of the frame of the respective virtual link; and a switch having a plurality of ports, each port coupled to one of the plurality of nodes; wherein the switch is configured to route frames received from the plurality of nodes to one or more of the plurality of nodes; and wherein at least one of the plurality of nodes is configured to store frames received from the switch in a buffer and update a value in the dynamic delay field to reflect the end-to-end system delay.

Example 2 includes the system of example 1, wherein the at least one node configured to update the value in the dynamic latency field is configured to update the dynamic latency field when the frame is accessed by an application.

Example 3 includes the system of any of examples 1-2, wherein the at least one node is configured to determine the delay value to be inserted based on one or more of a virtual link ID of the corresponding frame, an Internet Protocol (IP) source address of the corresponding frame, an IP destination address of the corresponding frame, a User Datagram Protocol (UDP) source port of the corresponding frame, and a UDP destination port of the corresponding frame.

Example 4 includes the system of any of examples 1-3, wherein the at least one node is configured to insert the delay values into a subset of all frames corresponding to the respective virtual links.

Example 5 includes the system of any of examples 1-4, wherein the at least one node is configured to update the checksum of the corresponding frame based on the delay value inserted into the delay field.

Example 6 includes the system of any of examples 1-5, wherein the at least one node is configured to dynamically determine one of a position or a length of the delay field in the corresponding frame.

Example 7 includes the system of any of examples 1-6, wherein the switch is configured to insert a delay value into a delay field of a corresponding frame received from the at least one node on a per virtual link basis.

Example 8 includes the system of any of examples 1-7, wherein the switch and the plurality of nodes are configured to implement a protocol compatible with ARINC644 part 7.

Example 9 includes a communications apparatus comprising a memory having a delay rule stored thereon, wherein the delay rule includes information for use in inserting one or more delay values into a delay field of a corresponding frame of a respective virtual link on a per virtual link basis; and a processing unit configured to insert one or more delay values into the corresponding frame on a per virtual link basis based on the delay rule by comparing information contained in the corresponding frame with the delay rule; the processing unit is further configured to update a delay field in the received frame until the frame is accessed by an application.

Example 10 includes the communication device of example 9, wherein the processing unit is configured to determine which delay values to insert based on a comparison of the delay rules to at least one of a virtual link ID of the corresponding frame, an Internet Protocol (IP) source address of the corresponding frame, an IP destination address of the corresponding frame, a User Datagram Protocol (UDP) source port of the corresponding frame, and a UDP destination port of the corresponding frame.

Example 11 includes the communication apparatus of any of examples 9-10, wherein the processing unit is configured to insert at least one of the dynamic delay value and the static delay value on a per virtual link basis.

Example 12 includes the communication apparatus of any of examples 9-11, wherein the processing unit is configured to update the checksum in the corresponding frame based on the delay value inserted into the corresponding frame.

Example 13 includes the communication device of any of examples 9-12, wherein the communication device is one of: an end system configured to implement a protocol compatible with the ARINC664 part 7 standard, or a switch configured to implement a protocol compatible with the ARINC664 part 7 standard.

Example 14 includes the communication apparatus of any of examples 9-13, wherein the processing unit is configured to dynamically determine one of a position or a length of the delay field in the corresponding frame.

Example 15 includes the communication device of any of examples 9-14, wherein the processing unit is configured to insert the delay values into a subset of all frames corresponding to the respective virtual links based on a delay rule.

Example 16 includes a method of transmitting frames, the method comprising inserting, at a source node, a delay value into a delay field of a corresponding frame of a respective virtual link on a per virtual link basis, wherein the delay value represents a latency of the frame of the respective virtual link; transmitting a corresponding frame with a delay field from the node to the switch; routing a corresponding frame having a delay field from the switch to one or more destination nodes based on the respective virtual links; and updating a value in the delay field at the one or more destination nodes to reflect the real-time end-to-end system delay.

Example 17 includes the method of example 16, wherein updating the value in the delay field includes updating the value in the delay field when the frame is accessed by the application.

Example 18 includes the method of any of examples 16-17, wherein inserting the delay value includes updating a checksum in the corresponding frame based on the inserted delay value.

Example 19 includes the method of any of examples 16-18, wherein transmitting the corresponding frame comprises transmitting the corresponding frame using a protocol compatible with ARINC664 part 7 standard; and wherein routing the corresponding frame comprises routing the corresponding frame using a protocol compatible with the ARINC664 part 7 standard.

Example 20 includes the method of any of examples 16-19, further comprising inserting, at the switch, a delay value into a delay field of a corresponding frame on a per virtual link basis.

Although specific embodiments have been illustrated and described herein, it will be appreciated by those of ordinary skill in the art that any arrangement which is calculated to achieve the same purpose may be substituted for the specific embodiments shown. Therefore, it is manifestly intended that this invention be limited only by the claims and the equivalents thereof.

Claims (9)

1. A system (100) for transmitting frames, comprising:

a plurality of nodes (102-1 … 102-N), at least one node of the plurality of nodes configured to insert, on a per virtual link basis, a delay value into a delay field of a frame corresponding to a respective virtual link, wherein the delay value represents a latency of the frame of the respective virtual link; and

a switch (104) having a plurality of ports (106-1 … 106-N), each port coupled to one of the plurality of nodes (102-1 … 102-N);

wherein the switch (104) is configured to route frames received from the plurality of nodes (102-1 … 102-N) to one or more of the plurality of nodes (102-1 … 102-N); and

wherein at least one node of the plurality of nodes (102-1 … 102-N) is configured to store frames received from the switch (104) in a buffer (127) and update the latency value in the latency field to reflect an end-to-end system latency;

wherein at least one of the plurality of nodes configured to insert a delay value is configured to determine on a per virtual link basis whether to insert a static delay value, a dynamic delay value, or a combination of static and dynamic delay values;

wherein the dynamic delay value is a value measured by a respective at least one of the plurality of nodes and the static measurement value is configured a priori and stored in the respective at least one of the plurality of nodes;

wherein the at least one node is configured to determine the delay value to be inserted based on one or more of a virtual link ID of the corresponding frame, an Internet Protocol (IP) source address of the corresponding frame, an IP destination address of the corresponding frame, a User Datagram Protocol (UDP) source port of the corresponding frame, and a UDP destination port of the corresponding frame.

2. The system (100) of claim 1, wherein the at least one node configured to update the delay value in the delay field is configured to update the delay value when the frame is accessed by an application.

3. The system (100) of claim 1, wherein the at least one node is configured to insert the delay value into a subset of all frames corresponding to a respective virtual link.

4. The system (100) of claim 1, wherein the switch (104) is configured to insert the delay value into a delay field of a corresponding frame received from the at least one node on a per virtual link basis.

5. A method (300) of transmitting a frame, the method comprising:

inserting, at the source node, a delay value into a delay field of a corresponding frame of a respective virtual link on a per virtual link basis, wherein the delay value represents a latency of the frame of the respective virtual link (302);

transmitting (304) the corresponding frame with the delay field from the node to a switch;

routing the corresponding frame with the delay field from the switch to one or more destination nodes based on the respective virtual links (308); and

updating a delay value in the delay field at the one or more destination nodes to reflect a real-time end-to-end system delay (310);

wherein inserting the delay value comprises determining, on a per virtual link basis, whether to insert a static delay value, a dynamic delay value, or a combination of static and dynamic delay values based on one or more of a virtual link ID of the corresponding frame, an Internet Protocol (IP) source address of the corresponding frame, an IP destination address of the corresponding frame, a User Datagram Protocol (UDP) source port of the corresponding frame, and a UDP destination port of the corresponding frame;

wherein the dynamic delay value is a value measured by a respective at least one of the plurality of nodes and the static measurement value is configured a priori and stored in the respective at least one of the plurality of nodes.

6. The method (300) of claim 5, wherein updating the delay value in the delay field comprises updating the delay value in the delay field when the frame is accessed by an application (310).

7. The method (300) of claim 5, wherein inserting a delay value comprises updating a checksum (302) in the corresponding frame based on the inserted delay value.

8. The method (300) of claim 5, wherein transmitting the corresponding frame comprises transmitting the corresponding frame (304) by using a protocol compatible with the ARINC664 part 7 standard; and is

Wherein routing the corresponding frame comprises routing the corresponding frame using a protocol compatible with the ARINC664 part 7 standard (308).

9. The method (300) of claim 5, further comprising inserting a delay value into the delay field of the corresponding frame at the switch on a per virtual link basis.