WO2026001943A1 - Data packet deduplication method, apparatus, device, storage medium and program product - Google Patents

Abstract

The present application belongs to the technical field of networks. Disclosed are a data packet deduplication method, an apparatus, a device, a storage medium and a program product. The method comprises: receiving a first data packet; and, if the first data packet carries a retransmission identifier, deduplicating the first data packet on the basis of a deduplication matching table, such that there is one first data packet in a receiving end device, the deduplication matching table storing information about data packets that have been received by the receiving end device. It can be seen that the receiving end device in the present application only deduplicates a small number of data packets carrying retransmission identifiers, not only ensuring that the receiving end device can process exactly once the data packets forwarded by a network, but also realizing lightweight deduplication of the receiving end device.

Description

Data packet deduplication methods, apparatus, equipment, storage media, and program products

This application claims priority to Chinese Patent Application No. 202410866893.1, filed on June 29, 2024, entitled “Data Packet Deduplication Method, Apparatus, Device, Storage Medium and Program Product”, the entire contents of which are incorporated herein by reference.

Technical Field

This application relates to the field of network technology, and in particular to a method, apparatus, device, storage medium, and program product for deduplicating data packets.

Background Technology

In the Unified Bus (UB) protocol, the data link layer supports sending and receiving data packets and link layer control packets, with a fixed minimum transmission unit (Flit) of 20 bytes. When the first forwarding device sends a data packet to the receiving device, it forwards a Flit of the packet from the corresponding port and backs up the sent Flit to its local retransmission buffer (also known as a Retry-Buff). Upon receiving this Flit, the second forwarding device performs a cyclic redundancy check (CRC) or decodes it. If the CRC check is correct, or the forward error correction (FEC) decoding is successful, it returns an acknowledgment (ACK) message to the first forwarding device, notifying it to release the retransmission buffer and continue forwarding the Flit to the receiving device. When the first forwarding device receives an acknowledgment message, it will delete the Flit from the retransmission buffer. However, if the second forwarding device fails, or if the communication path between the first and second forwarding devices fails, the first forwarding device may not be able to receive the ACK message from the second forwarding device. In this case, the first forwarding device will determine an alternative communication path through the fast reroute (FRR) mechanism and retransmit the Flits in the retransmission buffer to the receiving device along the alternative communication path. The receiving device may receive duplicate data packets.

Therefore, there is an urgent need for a data packet deduplication method to ensure that the receiving device can process the network-forwarded data packets exactly once.

Summary of the Invention

This application provides a data packet deduplication method, apparatus, device, storage medium, and program product, enabling a receiving device to deduplicatize data packets carrying retransmission identifiers after receiving them. This not only ensures that the receiving device can process network-forwarded data packets exactly once, but also achieves lightweight deduplication of data packets by the receiving device. The technical solution is as follows:

Firstly, a data packet deduplication method is provided, the method comprising:

The receiving device receives a first data packet; if the first data packet carries a retransmission flag, the receiving device performs deduplication processing on the first data packet according to the deduplication matching table, so that there is a first data packet in the receiving device, and the deduplication matching table stores information about the data packets that the receiving device has received.

In one possible implementation, if the first data packet does not carry a retransmission flag, then the first data packet is retained. Further, if the first data packet is retained, the deduplication matching table is updated based on the information in the first data packet; that is, the information of the first data packet is recorded in the deduplication matching table.

It should be noted that, for the receiving device, the deduplication matching table it maintains and manages can store only the information of the data packets received within a specified time period, or it can continuously store the information of all the data packets received. This application does not limit the time range of the information of the received data packets stored in the deduplication matching table, nor does it limit the amount of information in the deduplication matching table.

Therefore, when a receiving device receives a data packet, if the packet carries a retransmission flag, it indicates that the packet is a retransmitted packet from the network. Since the receiving device may have already received the same retransmitted packet, it needs to perform deduplication on the packet. This ensures that there are no duplicate data packets in the receiving device's memory, allowing it to process network-forwarded data packets exactly once.

Moreover, compared to related technologies that detect duplicate packets based on packet sequence number (PSN), which requires determining whether a packet is a duplicate packet based on the PSN carried by each packet, the receiving device in this application can directly predict whether a packet is a duplicate packet based on whether the packet carries a retransmission flag during the packet reception process. Thus, it only needs to perform deduplication processing on a small number of packets marked with retransmission flags based on the deduplication matching table, without having to perform the operation of determining whether each packet is a duplicate packet. This greatly reduces the resource consumption of the receiving device and achieves lightweight deduplication of packets by the receiving device.

In one possible implementation, the step of deduplicating the first data packet according to the deduplication matching table includes: if the first data packet is determined to be a duplicate data packet according to the deduplication matching table, then discarding the first data packet; if the first data packet is determined to be a non-duplicate data packet according to the deduplication matching table, then retaining the first data packet.

In other words, if the first data packet carries a retransmission flag, it is necessary to further confirm whether the receiving device has received the first data packet based on the deduplication matching table, so as to determine whether the first data packet is a duplicate data packet based on the deduplication matching table.

In one possible implementation, retaining the first data packet includes: storing the first data packet and recording the information of the first data packet in the deduplication matching table to update the deduplication matching table.

In other words, if the first data packet is not a duplicate data packet, the deduplication matching table needs to be updated based on the first data packet to avoid receiving the first data packet repeatedly in the subsequent data packet reception process.

In one possible implementation, the method further includes: receiving a first message carrying information about forwarded data packets, wherein the forwarded data packets are data packets forwarded by the target forwarding device to the receiving device within a target fault detection period, the first message being used to indicate that a fault is detected in the target communication path within the target fault detection period, and the target communication path being the communication path between the sending device and the receiving device; and updating the deduplication matching table based on the information about the forwarded data packets.

Specifically, the target forwarding device stores information about all data packets forwarded during the fault detection period and can detect and identify faults in the communication path of the transmitted data packets during the forwarding process. Therefore, if a fault is detected in the target communication path during the target fault detection period, a first message can be constructed and sent to the receiving device to indicate that the receiving device will receive retransmitted data packets in the subsequent data packets, requiring deduplication.

Furthermore, the first message carries information about the data packets that have been forwarded within the target fault detection period when a fault is detected in the target communication path. This allows the receiving device to update the deduplication matching table based on the information of the forwarded data packets, and then use the updated deduplication matching table to determine whether there are duplicate data packets that have been retransmitted within the target fault detection period in the subsequently received data packets.

In one possible implementation, the first message includes a first field carrying a message type, the message type indicating that the first message is a deduplication trigger message, the deduplication trigger message being used to instruct the receiving device to perform deduplication processing on data packets received after the first message.

In other words, the first message can be a deduplication trigger message. When the receiving device receives the first message, it can not only parse the information of the forwarded data packets from the first message, but also perform deduplication processing on the data packets received after the first message based on the message type of the first message.

In other words, the first message, as a deduplication trigger message, can instruct the receiving device to start the deduplication process.

In one possible implementation, the target forwarding device is located on the target communication path, and the target forwarding device includes at least one forwarding device that detects a fault in the target device or the target path segment on the target communication path; if the target device is faulty, the target forwarding device includes at least one forwarding device connected to the target device; if the target path segment is faulty, the target forwarding device includes a first forwarding device and/or a second forwarding device, the second forwarding device being a next-hop forwarding device in the target communication path that is communicatively connected to the first forwarding device, and the target path segment being a path segment between the first forwarding device and the second forwarding device.

In other words, a fault in the target communication path includes a fault in the link corresponding to a certain path segment, or a fault in a forwarding device on the target communication path. When a link corresponding to a path segment is faulty, both forwarding devices connected to that link can detect the link fault, and both of these forwarding devices can construct and send the first message. When a forwarding device on the target communication path is faulty, other forwarding devices connected to that forwarding device can detect the device fault, and one or more of these other forwarding devices can construct and send the first message.

In one possible implementation, the information of the forwarded data packet includes the identification information of the forwarded data packet, which includes the transaction layer segment number of the forwarded data packet or the packet sequence number information of the forwarded data packet.

It should be noted that this application uses only the transaction layer segment number or packet sequence number of the data packet to uniquely identify a data packet. In practical applications, other information can also be used to identify and distinguish data packets in order to better distinguish them. This application does not limit the specific content of the identification information of the data packet.

In one possible implementation, the deduplication trigger message includes a first field, which carries identification information of the forwarded data packet.

In one possible implementation, the information of the forwarded data packet also includes device address information, which at least indicates the address of the sending device.

In one possible implementation, the deduplication trigger message includes a first field and a second field, wherein the first field is used to carry the identification information of the forwarded data packet, and the second field is used to carry the device address information.

In one possible implementation, the first data packet also carries an end-of-retransmission flag, which indicates that the first data packet is the last data packet to undergo deduplication.

In other words, the first data packet can carry both a retransmission flag and an end-of-retransmission flag to indicate that it is the last retransmitted data packet and the last data packet that needs to be deduplicated. For the receiving device, based on the end-of-retransmission flag carried in the first data packet, after deduplicating the first data packet, no further deduplication processing is performed on subsequent received data packets.

In other words, in this application, the end retransmission flag can be used to instruct the receiving device to end the deduplication operation.

Secondly, a data packet deduplication method is provided, applied to a target forwarding device, the method comprising:

A first data packet is sent to the receiving device to instruct the receiving device to perform deduplication processing on the first data packet according to the deduplication matching table, so that there is a first data packet in the receiving device; wherein, the first data packet carries a retransmission identifier, and the deduplication matching table stores information on data packets that the receiving device has received.

Therefore, if a data packet is a retransmitted packet from a forwarding device, it will carry a retransmission flag. In other words, when a forwarding device in the network forwards data packets to the receiving device, it marks the few retransmitted packets to carry a retransmission flag.

Moreover, compared to related technologies that rely on PSN to detect duplicate packets, requiring each packet to be identified by its PSN, the forwarding device in this application can tag a small number of retransmitted packets. This allows the receiving device to directly predict whether a packet is a retransmitted packet based on whether it carries a retransmission flag. Consequently, only a small number of packets with retransmission flags need to be deduplicated using a deduplication matching table. This eliminates the need to perform a deduplication check on every single packet, significantly reducing the resource consumption of the receiving device and enabling lightweight deduplication of packets.

In one possible implementation, the method further includes:

A first message is sent to the receiving device to instruct the receiving device to update the deduplication matching table based on the information of the forwarded data packets carried in the first message; wherein, the forwarded data packets are data packets forwarded by the target forwarding device to the receiving device within the target fault detection period, and the first message is used to indicate that a fault is detected in the target communication path within the target fault detection period, and the target communication path is the communication path between the sending device and the receiving device.

Specifically, the target forwarding device stores information about all data packets forwarded during the fault detection period and can detect and identify faults in the communication path of the transmitted data packets during the forwarding process. Therefore, if a fault is detected in the target communication path during the target fault detection period, a first message can be constructed and sent to the receiving device to indicate that the receiving device will receive retransmitted data packets in the subsequent data packets, requiring deduplication.

In one possible implementation, the first message includes a first field carrying a message type, the message type indicating that the first message is a deduplication trigger message, the deduplication trigger message being used to instruct the receiving device to perform deduplication processing on data packets received after the first message.

In one possible implementation, the target forwarding device is located on the target communication path, and the target forwarding device includes at least one forwarding device that detects a fault in the target device or the target path segment on the target communication path; if the target device is faulty, the target forwarding device includes at least one forwarding device connected to the target device; if the target path segment is faulty, the target forwarding device includes a first forwarding device and/or a second forwarding device, the second forwarding device being a next-hop forwarding device in the target communication path that is communicatively connected to the first forwarding device, and the target path segment being a path segment between the first forwarding device and the second forwarding device.

In one possible implementation, the information of the forwarded data packet includes the identification information of the forwarded data packet, which includes the transaction layer segment number of the forwarded data packet or the packet sequence number information of the forwarded data packet.

In one possible implementation, the information of the forwarded data packet also includes device address information, which at least indicates the address of the sending device.

In one possible implementation, the first data packet also carries an end-of-retransmission flag, which indicates that the first data packet is the last data packet to undergo deduplication.

Thirdly, a data packet deduplication device is provided, which has the function of implementing the data packet deduplication method described in the first aspect. The data packet deduplication device includes at least one module, which is used to implement the steps of the data packet deduplication method provided in the first aspect.

Fourthly, a data packet deduplication device is provided, which has the function of implementing the data packet deduplication method described in the second aspect above. The data packet deduplication device includes at least one module, which is used to implement the steps of the data packet deduplication method provided in the second aspect above.

Fifthly, a computer device is provided, comprising a processor and a memory, the memory being used to store a computer program for executing the data packet deduplication method provided in the first aspect, or a computer program for executing the data packet deduplication method provided in the second aspect. The processor is configured to execute the computer program stored in the memory to implement the data packet deduplication method described in the first aspect, or to implement the data packet deduplication method described in the second aspect.

In one possible implementation, the computer device may further include a communication bus for establishing a connection between the processor and the memory.

In a sixth aspect, a computer-readable storage medium is provided, wherein instructions are stored therein, which, when executed on a computer, cause the computer to perform the steps of the data packet deduplication method described in the first aspect, or to perform the steps of the data packet deduplication method described in the second aspect.

In a seventh aspect, a computer program product containing instructions is provided, which, when executed on a computer, cause the computer to perform the steps of the data packet deduplication method described in the first aspect, or to perform the steps of the data packet deduplication method described in the second aspect.

Alternatively, a computer program is provided that, when run on a computer, causes the computer to perform the steps of the data packet deduplication method described in the first aspect, or to perform the steps of the data packet deduplication method described in the second aspect.

The technical effects achieved by the second to seventh aspects mentioned above are similar to those achieved by the corresponding technical means in the first aspect, and will not be repeated here.

Attached Figure Description

Figure 1 is a schematic diagram of a link layer data packet sending and receiving process provided in an embodiment of this application;

Figure 2 is a schematic diagram of data packet forwarding in the case of link failure in the first network topology provided in the embodiment of this application;

Figure 3 is a schematic diagram of data packet forwarding in the case of link failure in the second network topology provided in the embodiments of this application;

Figure 4 is a schematic diagram of the hierarchy of a UB-G protocol stack provided in an embodiment of this application;

Figure 5 is a schematic diagram of a process for detecting duplicate packets using packet serial numbers in a related technology provided in an embodiment of this application;

Figure 6 is a schematic diagram of the hierarchy of a UB-C protocol stack provided in an embodiment of this application;

Figure 7 is a schematic diagram of a data center architecture provided in an embodiment of this application;

Figure 8 is a schematic diagram of an application scenario of a data packet deduplication method provided in an embodiment of this application;

Figure 9 is a schematic diagram of the structure of a network device provided in an embodiment of this application;

Figure 10 is a schematic diagram of the structure of a computer device provided in an embodiment of this application;

Figure 11 is a flowchart of a data packet deduplication method provided in an embodiment of this application;

Figure 12 is a schematic diagram of retransmitting data packets via an alternative path according to an embodiment of this application;

Figure 13 is a schematic diagram of the format of a first message provided in an embodiment of this application;

Figure 14 is a schematic diagram of the sending path of a first message based on a first network topology provided in an embodiment of this application;

Figure 15 is a schematic diagram of another first message transmission path based on a first network topology provided in an embodiment of this application;

Figure 16 is a schematic diagram of the sending path of a first message based on a second network topology provided in an embodiment of this application;

Figure 17 is a schematic diagram of another first message transmission path based on a second network topology provided in an embodiment of this application;

Figure 18 is a schematic diagram of another format of the first message provided in an embodiment of this application;

Figure 19 is a schematic diagram of another format of the first message provided in an embodiment of this application;

Figure 20 is a schematic diagram of a data packet header structure provided in an embodiment of this application;

Figure 21 is a logical schematic diagram of a receiving device processing data packets according to an embodiment of this application;

Figure 22 is a schematic diagram of the internal data packet processing flow of a switch according to an embodiment of this application;

Figure 23 is a schematic diagram of the interval release logic of a retransmission buffer provided in an embodiment of this application;

Figure 24 is a schematic diagram of the process of a receiving device responding to a deduplication trigger message according to an embodiment of this application;

Figure 25 is a schematic diagram of the process of deduplication of data packets by a receiving device according to an embodiment of this application;

Figure 26 is a schematic diagram of a data packet deduplication device provided in an embodiment of this application;

Figure 27 is a schematic diagram of another data packet deduplication device provided in an embodiment of this application.

Detailed Implementation

To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the implementation methods of this application will be further described in detail below with reference to the accompanying drawings.

To facilitate understanding, before explaining the data packet deduplication method provided in the embodiments of this application, the application background and implementation environment involved in the embodiments of this application will be introduced first.

First, the relevant background of the embodiments of this application will be introduced.

Referring to Figure 1, in the Unified Bus (UB) protocol, the data link layer supports sending and receiving service messages and link layer control messages. The smallest unit of data link layer transmission is a Flit, with a fixed size of 20 bytes. As shown in Figure 1, after the first forwarding device forwards a Flit of a data packet from a certain port, it backs up the sent Flit to its local retransmission buffer (also known as a Retry-Buff). Upon receiving this Flit, the second forwarding device in the next hop of the network performs a Cyclic Redundancy Check (CRC) check or decodes it. If the CRC check is correct, or the Forward Error Correction (FEC) decoding is successful, it returns an ACK message to the first forwarding device to notify it to release the retransmission buffer and continue forwarding the Flit to the receiving device. Furthermore, after receiving the ACK message from the second forwarding device, the first forwarding device deletes the Flit backup from the retransmission buffer to free up space resources in the retransmission buffer.

CRC checksum is a commonly used error detection method used to check whether errors occurred during data transmission. If the CRC checksum is correct, it means that no errors occurred during Flit transmission. FEC decoding is a more advanced error correction method; it can not only detect errors but also correct them in some cases. If FEC decoding is successful, it means that even if Flit had errors, they were successfully corrected.

In one possible implementation, the first forwarding device and the second forwarding device can be a switch, a router, or other network devices capable of forwarding data packets, such as a switching module on a computing chip. This application does not limit this.

In one possible implementation, the ACK message can be carried in the header of the link layer packet, such as the ACK field of the link layer packet header (LPH) or the link layer block header (LBH); or, the ACK message can also be sent through a special Flit, such as through a CRD (credit, a flow control mechanism) message, to inform the first forwarding device that the data packet Flit has been successfully received.

It should be understood that the LPH (Link Layer Header), as the header of a link-layer data packet, contains various information about the packet, such as source address, destination address, data length, and protocol type. During data transmission, the LPH helps network devices identify and process data packets. When a large data packet (e.g., longer than 32 folds) is segmented into multiple blocks for transmission, the LBH (Block Header) refers to the header of a block. Similar to the LPH, the LBH also contains information about the block, possibly including its length, type, and checksum. In communication protocols, CRD (Flow Control Token) is typically used for flow control, indicating the amount of data the receiver can receive. Simply put, a CRD is a counter or token system used to prevent the sender from sending too much data and causing the receiver's buffer to overflow. Therefore, when the receiver has enough buffer space to receive new data, it sends a CRD message to the sender, informing the sender how much more data it can continue sending.

During data link layer packet transmission, if the second forwarding device encounters a CRC check error or FEC decoding failure after receiving the data packet Flit, it will request the first forwarding device to retransmit the Flit. In response to this retransmission request, the first forwarding device will resend the backup data packet Flit from its retransmission buffer to the second forwarding device.

Similarly, for a retransmitted data packet Flit, if the CRC check is correct or the forward error correction coding (FEC) is successfully decoded, the second forwarding device will return an ACK message to the first forwarding device to notify the first forwarding device to release the retransmission buffer and continue to forward the Flit to the receiving device.

In practical applications, bus protocols are mostly used within the chassis, such as the Peripheral Component Interconnect Express (PCI-E). Within this chassis, devices are connected via cables, ensuring very high communication reliability. However, as bus networks grow larger, optical modules are needed to achieve cross-rack connections. However, the native reliability of optical ports is more than an order of magnitude lower than that of cables. Therefore, if a large number of optical ports are used in a network system, intermittent signal interruptions (i.e., optical signal outages) may occur.

As an example, in a network containing 12 blocks, with each block containing approximately 32 servers, and each server having 40 upstream ports (which could be abstractions of resources such as network interface cards or CPU cores), the entire network would contain approximately 12 * 32 * 40 = 15360 servers. For this network, the probability of a port status change (e.g., port up and down, possibly due to network failures, server restarts, etc.) occurring 3 times per hour is 15360. This means that on average, three ports will experience status changes per hour, regardless of whether it's due to intermittent interruptions or server power cycles. Additionally, the probability of an optical module experiencing an intermittent interruption is 2.1%, typically caused by fiber optic cable breaks, optical module malfunctions, dust contamination, etc. Based on this, in a 1024P (with 1024 processing units or ports) supernode, the number of optical modules can reach 8000. This means that in a large supernode, a large number of optical modules are used for data communication. According to the aforementioned optical module outage probability, the expected number of optical module outages per day in a 1024P supernode is 8000 * 2.1% / 365 = 0.46 times. Therefore, it is evident that in network systems using optical ports, a certain number of outage events may occur daily.

As an example, referring to Figure 2, in the first network topology (2D Torus network architecture), node 8 is communicating with node 7 via the primary path. After a link failure occurs between node 9 and node 10, node 10 sends all the collected data packets to node 7. After detecting the link failure, node 9 determines an alternative path through fast reroute (FRR) and retransmits the data packets in the retransmission buffer to node 7 along the alternative path.

When transmitting data packets between node 8 and node 7, the primary communication path can be node 8-node 9-node 10-node 11-node 7, and the backup communication path can be node 8-node 9-node 5-node 6-node 7.

However, when Node 9 retransmits data packets in the retransmission buffer, these packets may include data packets that could not receive the ACK message from Node 10 due to link failure. In other words, these data packets have already been sent to Node 7 by Node 10. At this time, Node 7 may receive duplicate data packets. These duplicate data packets may compromise data consistency, atomicity, concurrency control, and cause resource waste, affecting the correctness of write and atomic operations. It cannot be guaranteed that Node 7 can process network-forwarded data packets exactly once.

As another example, referring to Figure 3, in the second network topology (Clos networking architecture), node A in the processor unit is communicating with node H via the main path. After a link failure occurs between node Leaf 1 in the access layer and node Spine 1 in the core layer, node Spine 1 sends all the collected data packets to node H. After detecting the link failure, node Leaf 1 determines an alternative path through FRR and retransmits the data packets in the retransmission buffer to node H along the alternative path.

When transmitting data packets between node A and node H, the primary communication path can be node A - node Leaf 1 - node Spine 1 - node Leaf n - node H, and the alternative communication path can be node A - node Leaf 1 - node Spine 16 - node Leaf n - node H.

However, when Leaf 1 retransmits data packets in its retransmission buffer, these packets may include those that cannot receive ACK messages from Spine 1 due to link failures. In other words, these packets have already been sent to node H by Spine 1. At this point, node H may receive duplicate data packets. These duplicate data packets may compromise data consistency, atomicity, concurrency control, and cause resource waste, affecting the correctness of write and atomic operations. It cannot be guaranteed that node H can process network-forwarded data packets exactly once.

In summary, when multiple nodes in a UB system communicate through a network, ensuring that the receiving device can process the network-forwarded data packets exactly once is a pressing technical problem that needs to be solved.

In view of the above-mentioned technical problems, corresponding solutions have been provided in related technologies. To facilitate comparison and explanation of the solutions provided in the embodiments of this application, the solutions provided in related technologies will be briefly introduced here first.

In one related technology, please refer to Figure 4, which shows a schematic diagram of the UB-G domain (UB-Global, abbreviated as UB-G) protocol stack. The complete transport layer in UB-G is used to provide both reliable and unreliable transport services. A UB-G domain refers to a specific functional domain or system scope built based on UB-G technology or standards. In distributed systems, a domain typically refers to a collection of logically related system components or services that share common functional characteristics.

UB's reliable transport service detects duplicate packets and network packet loss using Packet Sequence Numbers (PSNs) to achieve reliable data packet transmission. The PSN is carried in the transport packet header (TPH). The receiving device increments the PSN value by 1 for each ACK message (also called a TPACK message) returned for a transport (TP) connection. In practice, the sending and receiving devices negotiate an initial PSN value when establishing a TP connection; this initial value is randomized between 0 and 16M-1. The initial PSN value is the same as the PSN value carried in the first data packet.

In one implementation, when the receiving device returns a TP ACK message, the TP ACK message can carry the accumulated PSN value to indicate that all packets with the accumulated PSN have been correctly received. For example, if the receiving device receives 10 packets from the sending device, and the PSN values of these 10 packets range from 1 to 10, the receiving device can aggregate all the ACK messages into one message, namely a TP ACK message with a PSN value of 10, and send this TP ACK message to the sending device to inform the sending device that the packet with PSN 10, as well as all previous packets, have been correctly received.

It should be noted that the PSN value only applies within a single TP connection. The PSN values of different TP connections are completely independent and have no correlation.

As an example, referring to Figure 5, the PSN value has a bit width of 24 bits, representing a range of 0 to 16M-1. The maximum number of outstanding requests for PSN values sent by the sending device is 8M, meaning that the maximum PSN value sent minus the minimum PSN value awaiting acknowledgment does not exceed 8M-1. Here, the outstanding request from the sending device refers to the number of incomplete requests that can be submitted simultaneously to the memory or other target devices within a given time period. These requests may include data packets, read/write operations, etc., which have been sent but have not yet received acknowledgment or completion.

As shown in Figure 5, if the minimum PSN value waiting for acknowledgment is 0, then the maximum PSN value allowed to be sent is 8M-1 (i.e., 8388607); if the minimum PSN value waiting for acknowledgment is 8M (i.e., 8388608), then the maximum PSN value allowed to be sent is 16M-1 (i.e., 16777215); if the minimum PSN value waiting for acknowledgment is 10M (i.e., 1048576), then the maximum PSN value allowed to be sent is 18M-1 (i.e., 18874367). Since 18M-1 exceeds the 24-bit representation range, there is a high-bit overflow, that is, the maximum PSN allowed to be sent is 2M-1 (i.e., 2097151).

For the receiving device, the expected PSN (EPSN) refers to the expected forward-order PSN. For example, if a data packet with PSN value = 5 has been successfully received by the receiving device, and the data packet with PSN value = 5 is not out-of-order, then the receiving device's EPSN value is 6.

In addition, the sending device will not send outstanding requests with PSN values exceeding 8M. Therefore, the range [EPSN-8M, EPSN-1] corresponding to EPSN values is the duplicate packet range.

Based on this, the receiving device's process for ensuring exactly once can be as follows: After receiving a data packet, the receiving device first determines whether the data packet falls within a duplicate range, an out-of-order range, or an invalid range based on the data packet's PSN value. If the data packet's PSN value falls within a duplicate range, the receiving device discards the data packet and returns an ACK message to the sending device. The PSN value carried in this ACK message is equal to the receiving device's EPSN value minus 1. If the data packet's PSN value is equal to the receiving device's EPSN value, the receiving device receives the data packet and returns an ACK message to the sending device. The PSN value carried in this ACK message is equal to the receiving device's EPSN value. If the data packet's PSN value falls within the maximum allowed out-of-order range, the receiving device receives the data packet and returns a selective ACK (SACK) message to the sending device. If the data packet's PSN value falls within an invalid range, the data packet is discarded directly.

However, in the above solution, all data packets received by the receiving device must pass through the UB-G transport layer. The UB-G transport layer occupies 3 square millimeters of the receiving device's resource area and has a processing speed of 200 Mpps. Here, pps stands for packets per second, referring to the number of data packets sent per second. Therefore, to achieve the 1 Gpps throughput requirement in the UB-C domain (UB-Clan, or UB-C for short), the above solution requires an additional 12 square millimeters of resource area in the receiving device. This resource overhead is too large, making it difficult for the receiving device to handle the load.

Similarly, UB-C here refers to a specific set or domain based on UB bus technology.

In another related technology, please refer to the UB-C protocol stack diagram shown in Figure 6. In the UB-C protocol stack, the lightweight transport layer does not have a TP connection, and its reliable transmission service relies on the retransmission mechanism of the UB link layer to ensure that there is no packet loss when transmitting data packets end-to-end.

However, the lightweight transport layer in the UB-C protocol can only solve the packet loss detection problem, but cannot solve the problem of the receiving device receiving duplicate packets, that is, it cannot guarantee that the receiving device can process the network forwarded data packets exactly once.

Based on the two solutions described above, this application provides a data packet deduplication method that can be applied to the lightweight transport layer in UB-C. In the data packet deduplication method provided in this application, after a forwarding device in the network detects a fault in the target communication path of the currently transmitted data packet and performs a First Retransmission (FRR), it marks the subsequent data packets retransmitted along the backup path with a retransmission flag to indicate to the receiving device that the data packet is a retransmitted data packet. Based on this, after receiving the data packet, the receiving device parses the data packet to determine whether it carries a retransmission flag. If the data packet carries a retransmission flag, it performs deduplication processing on the data packet according to a deduplication matching table to ensure that there are no duplicate data packets in the receiving device. The deduplication matching table stores information about the data packets already received by the receiving device. Therefore, this application only performs deduplication processing on a small number of data packets carrying retransmission flags. Compared to the UB-G scheme that judges and processes each data packet based on the PSN value, it achieves lightweight deduplication and ensures that the receiving device can process network-forwarded data packets exactly once.

Next, the application scenarios and implementation environments involved in the embodiments of this application will be introduced.

The data packet deduplication method provided in this application embodiment can be applied in data center scenarios. Referring to Figure 7, the data center can be a tightly coupled UB-C system (such as a supernode) or a loosely coupled UB-G system. Here, a tightly coupled UB-C system refers to a RACK or supernode, where different computing nodes and host devices are connected together via a high-speed network. UB-G is a loosely coupled distributed system linked by UBs, supporting peer-to-peer interconnection between distributed system nodes and interoperable with existing Internet Protocol (IP) and Ethernet (Eth) networks.

Please refer to Figure 8. When implementing the above data packet deduplication method, the hardware devices involved in this application embodiment include forwarding devices (such as switches, routers, etc.) and receiving devices in the network. The receiving device can be an independent computing node or a network device; this application embodiment does not impose any restrictions on this.

It should be noted that for the forwarding device located between the sending device and the receiving device, each communication path may include one forwarding device or multiple forwarding devices. That is, the forwarding device can be directly connected to the receiving device, or it can be connected to the receiving device through at least one other forwarding device. This application embodiment does not impose any restrictions on this.

Please refer to Figure 9, which is a schematic diagram of a network device provided in an embodiment of this application. This network device can be a switch, router, or other device with packet forwarding capabilities. In this embodiment, the network device acts as a packet forwarding device to perform the corresponding steps in the packet deduplication method.

As shown in Figure 9, in this embodiment of the application, the network device includes: a main control board 910, an interface board 930, and an interface board 940. In the case of multiple interface boards, a switching network board (not shown in Figure 9) may be included, which is used to complete data exchange between the interface boards (interface boards are also called line cards or service boards).

The main control board 910 performs functions such as system management, equipment maintenance, and protocol processing. Interface boards 930 and 940 provide various service interfaces (e.g., POS interface, GE interface, ATM interface, etc.) and forward data streams. The main control board 910 primarily has three types of functional units: a system management control unit, a system clock unit, and a system maintenance unit. The main control board 910, interface boards 930 and 940 communicate with each other via a system bus connected to the system backplane. Interface board 930 includes one or more processors 931. Processors 931 control and manage the interface boards, communicate with the central processing unit on the main control board, and handle data stream forwarding. The memory 932 on interface board 930 stores forwarding table entries; processors 931 forward data streams by searching the forwarding table entries stored in memory 932.

The interface board 930 includes one or more network interfaces 933 for receiving data streams or other information sent by other devices, and processing these data streams or information according to the instructions of the processor 931. The specific implementation process will not be described in detail here.

As shown in Figure 9, this embodiment includes multiple interface boards and employs a distributed forwarding mechanism. Under this mechanism, the operations on interface board 940 are essentially similar to those on interface board 930, and will not be elaborated further for simplicity. Furthermore, it is understood that the processors 931 and/or 941 in interface board 930 of Figure 9 can be dedicated hardware or chips, such as network processors or application-specific integrated circuits (ASICs), to implement the aforementioned functions. This implementation method is commonly referred to as using dedicated hardware or chips for the forwarding plane. Of course, processors 931 and/or 941 can also use general-purpose processors, such as general-purpose CPUs, to implement the functions described above.

Furthermore, it should be noted that there may be one or more main control boards, including a primary main control board and a backup main control board. There may also be one or more interface boards; the stronger the data processing capability of the network device, the more interface boards it provides. With multiple interface boards, these boards can communicate through one or more switching network boards, enabling load sharing and redundancy backup. In a centralized forwarding architecture, the network device may not require a switching network board; the interface boards handle the processing of the entire system's business data. In a distributed forwarding architecture, the network device includes multiple interface boards, which can exchange data with each other through a switching network board, providing high-capacity data exchange and processing capabilities. Therefore, the data access and processing capabilities of a distributed architecture network device are greater than those of a centralized architecture network device. The specific architecture adopted depends on the specific network deployment scenario, and no limitations are made here.

In some embodiments, memory 932 may be read-only memory (ROM), random access memory (RAM), electrically erasable programmable read-only memory (EEPROM), optical discs (including compact disc read-only memory (CD-ROM), compressed optical discs, laser discs, digital versatile optical discs, Blu-ray discs, etc.), magnetic disk storage media, or other magnetic storage devices, or any other medium capable of carrying or storing desired program code having an instruction or data structure form and accessible by a computer, but not limited thereto. Memory 932 may exist independently and be connected to processor 931 via a communication bus. Memory 932 may also be integrated with processor 931.

In some embodiments, network interface 933 can be a transceiver-like device used to communicate with other devices or communication networks, such as Ethernet, radio access network (RAN), wireless local area network (WLAN), etc. Network interface 933 includes a wired network interface and may also include a wireless network interface. The wired network interface can be, for example, an Ethernet interface. The Ethernet interface can be an optical interface, an electrical interface, or a combination thereof. The wireless network interface can be a WLAN interface, a cellular network communication interface, or a combination thereof. When the network device acts as any network device within a domain, network interface 933 is used to forward data packets to other network devices. When the network device acts as a head node within a domain, network interface 933 can also be used to communicate with receiving devices, for example, sending data packets and deduplication trigger messages to the receiving devices.

In some embodiments, a network device may include multiple processors, each of which may be a single-core processor or a multi-core processor. Here, a processor may refer to one or more devices, circuits, and/or processing cores used to process data (such as computer program instructions).

In some embodiments, the memory 932 is used to store a computer program that executes the scheme of this application. The processor 931 can execute the computer program stored in the memory 932 to cause the network device to perform the processing steps of the forwarding device in the method embodiments below. For specific implementation, please refer to the detailed description in the method embodiments, which will not be repeated here.

Please refer to Figure 10, which is a schematic diagram of a computer device provided in an embodiment of this application. This computer device can be a terminal device or a server. In this embodiment, the computer device acts as a receiving device to execute the corresponding steps in the data packet deduplication method. As shown in Figure 10, in this embodiment, the computer device includes at least one processor 101, a communication bus 102, a memory 103, and at least one communication interface 104.

Processor 101 can be a general-purpose central processing unit (CPU), a network processor (NP), a microprocessor, or one or more integrated circuits for implementing the solutions of this application, such as application-specific integrated circuits (ASICs), programmable logic devices (PLDs), or combinations thereof. The aforementioned PLD can be a complex programmable logic device (CPLD), a field-programmable gate array (FPGA), a generic array logic (GAL), or any combination thereof.

The communication bus 102 is used to transmit information between the aforementioned components. The communication bus 102 can be divided into an address bus, a data bus, a control bus, etc. For ease of illustration, only one thick line is used to represent it in Figure 10, but this does not mean that there is only one bus or one type of bus.

The memory 103 may be a read-only memory (ROM), a random access memory (RAM), an electrically erasable programmable read-only memory (EEPROM), an optical disc (including a compact disc read-only memory (CD-ROM), a compressed optical disc, a laser disc, a digital versatile optical disc, a Blu-ray disc, etc.), a magnetic disk storage medium, or other magnetic storage device, or any other medium capable of carrying or storing desired program code in the form of instructions or data structures that can be accessed by a computer, but not limited thereto. The memory 103 may exist independently and be connected to the processor 101 via the communication bus 102. The memory 103 may also be integrated with the processor 101.

Communication interface 104 uses any transceiver-like device for communicating with other devices or communication networks. Communication interface 104 includes a wired communication interface and may also include a wireless communication interface. The wired communication interface may be, for example, an Ethernet interface. The Ethernet interface may be an optical interface, an electrical interface, or a combination thereof. The wireless communication interface may be a wireless local area network (WLAN) interface, a cellular network communication interface, or a combination thereof, etc.

As an example, processor 101 may include one or more CPUs, such as CPU0 and CPU1 as shown in FIG10.

As an example, a computer device may include multiple processors, such as processor 101 and processor 105 as shown in Figure 10. Each of these processors may be a single-core processor or a multi-core processor. Here, "processor" may refer to one or more devices, circuits, and/or processing cores used to process data (such as computer program instructions).

In some embodiments, the computer device may further include output devices and input devices. The output device communicates with the processor 101 and can display information in various ways. For example, the output device may be a liquid crystal display (LCD), a light-emitting diode (LED) display device, a cathode ray tube (CRT) display device, or a projector, etc. The input device communicates with the processor 101 and can receive user input in various ways. For example, the input device may be a mouse, keyboard, touchscreen device, or sensing device, etc.

In some embodiments, memory 103 is used to store program code 110 for executing the scheme of this application, and processor 101 can execute the program code 110 stored in memory 103. The program code 110 may include one or more software modules. The computer device can implement the processing steps of the receiving device in the following method embodiments through processor 101 and program code 110 in memory 103. For specific implementation, please refer to the detailed description in the following method embodiments, which will not be repeated here.

It should be noted that in some communication network architectures (such as Torus networking architecture), computer devices or computing nodes themselves can also act as forwarding devices to forward data packets. In this case, the sending device, forwarding device, and receiving device can all be independent computer devices, such as servers, terminals, processor units, processor chips, etc. This application does not impose any restrictions on this.

It should be understood that the network architecture, device architecture, and business scenarios described in the embodiments of this application are for the purpose of more clearly illustrating the technical solutions of the embodiments of this application, and do not constitute a limitation on the technical solutions provided in the embodiments of this application. As those skilled in the art will know, with the evolution of network architecture and device architecture and the emergence of new business scenarios, the technical solutions provided in the embodiments of this application are also applicable to similar technical problems.

Next, the data packet deduplication method provided in the embodiments of this application will be explained in detail with reference to the accompanying drawings.

Please refer to Figure 11, which is a flowchart of a data packet deduplication method provided in an embodiment of this application. The method is applied to a receiving device and includes the following steps.

Step 1101: Receive the first data packet.

For a data packet sent from the sending device to the receiving device, after the sending device sends the data packet out of the network, the data packet may be forwarded by multiple forwarding devices in sequence before finally being forwarded to the receiving device.

In a network, during the forwarding of data packets by multiple forwarding devices, some network devices may malfunction, or the links between network devices may fail. In such cases, to ensure the receiving device can receive the corresponding data packets, the first forwarding device that forwarded the data packets will, based on the backup data packets in its retransmission buffer, determine an alternative path using FRR (Fault Reset) and retransmit the data packets to the receiving device along the alternative path. In this scenario, the first data packet received by the receiving device may be the first data packet sent after the failure, or it may be a retransmitted data packet.

In some embodiments, a forwarding device in the network may add a retransmission identifier to a retransmission packet for the packet it retransmits, in order to indicate to the receiving device that the packet is a retransmission.

As an example, referring to Figure 12, within the UB-C domain, when the S1 switch detects a failure in the communication path (e.g., a failure in the path segment between the S1 switch and the S3 switch), it determines an alternative path through FRR, namely S1 switch - S2 switch - S3 switch - S4 switch - receiving device, and retransmits the data packets in the retransmission buffer so that the data packets can be forwarded to the receiving device along the alternative path.

In the transaction layer of this data packet, the configuration (CFG) field value in the UB protocol is 6, indicating that the data packet is a transmission message within a certain family domain of the data center shown in Figure 7.

Furthermore, before retransmitting a data packet, the S1 switch adds a retransmission flag "R" to the packet header, sets its value to 1, and then sends the data packet to the receiving end via the backup path. After receiving the data packet, the receiving end parses the packet header. If the value of "R" is 0, it determines that the data packet is the original packet; if the value of "R" is 1, it means that the data packet is being retransmitted.

In some embodiments, when a forwarding device in the network detects a failure in the target communication path for forwarding the first data packet, it may also construct a first message and send the first message to the receiving device to indicate that the data packets subsequently sent by the receiving device include retransmission data packets.

The first message includes a first field, which carries a message type. The message type indicates that the first message is a deduplication trigger message. The deduplication trigger message is used to instruct the receiving device to perform deduplication processing on the data packets received after the first message.

As an example, as shown in Figure 13, in the format of the fully compressed network header under the UB compressed message format, the first field can be the next layer protocol (NLP) field. The specific value of this NLP field is used to indicate the message type. Depending on the different NLP fields in the network layer header (NTH), there are three possible combinations:

The first type: NTH.NLP=000, with a transport packet header (TPH). This TPH.NLP is used to indicate whether confidentiality and integrity protection (CIP) is included.

CIP.NLP indicates whether it includes a UB partition identifier (UPI)/entity ID (EID, which indicates the unique identifier of a UB entity within the UB family domain) and its bit width.

The second type: NTH.NLP=001, without TPH, CIP, EID and transaction header (TAH), with 16-bit UPI to identify that the message is used for the enumeration management protocol.

The third type: NTH.NLP=110, without TPH, CIP, UPI and TAH, with EID, used to identify the message as a deduplication trigger message.

The EID carried can be the EID of the neighboring node to which the faulty node is connected.

In one possible implementation, the first message may carry the network address (CNA) of the sending device in the UB-C domain in the header as the source address information (denoted as SCNA), and at the same time carry the CNA of the receiving device as the destination address information (denoted as DCNA).

In one possible implementation, the service level (SL) of the deduplication trigger message is set to the highest priority, so that the deduplication trigger message can enter the highest priority transmission queue and be sent to the receiving device as soon as possible.

It should be understood that in the complete compressed network header shown in Figure 13, CRD (i.e., credit) indicates whether the message returns a credit, ACK indicates whether the message releases the retransmission buffer space, and CRD_VL indicates the virtual link (VL) number corresponding to the credit returned by the CRD segment. Regarding the CFG segment, when CFG=0, it indicates a data link layer control message; CFG=1-2, reserved; CFG=3, indicates a Global_Domain-IPv4 message; CFG=4, indicates a Global_Domain-IPv6 message; CFG=5, indicates a network control message; CFG=6, indicates a message within a family domain; CFG=7-15, reserved. Furthermore, RM is the UB message routing mode indicator segment. In addition, Plength[13:10] indicates the number of message blocks, with 0 to 15 indicating 1 to 16 blocks respectively. Plength[9:5] indicates the number of folds in the last block, with 0 to 31 indicating 1 to 32 folds respectively. Plength[4:0] indicates the number of bytes in the packet tail payload (excluding LPH/LBH and BCRC fields). When Plength[4:0] is 0-15, it indicates that the number of payload bytes in the last Flit within the packet is Plength[4:0]+1; when Plength[4:0] is 16-19, it indicates that the number of payload bytes in the second-to-last Flit within the packet is Plength[4:0]+1; when Plength[4:0] is 20-27, it indicates that the number of payload bytes in the second-to-last Flit within the packet is Plength[4:0]-11; when Plength[4:0] is 28-31, it indicates that the number of payload bytes in the last Flit within the packet is Plength[4:0]-11. This application embodiment does not provide a specific explanation of this; for specific meanings, please refer to the relevant descriptions of the complete compressed network header under the UB compressed message format. This application embodiment will not elaborate further here.

Based on this, when the receiving device executes the data packet deduplication method provided in the embodiments of this application, the receiving device also needs to perform the following steps:

(1) Receive the first message.

The first message carries information about the forwarded data packets, which are data packets forwarded by the target forwarding device to the receiving device within the target fault detection period. The first message is used to indicate that a fault was detected in the target communication path within the target fault detection period. The target communication path is the communication path between the sending device and the receiving device.

The fault detection period is a preset value. The basis for setting it in this application embodiment is not limited. For example, the fault detection period value can be flexibly set and adjusted based on the time (nanosecond level) when the fault is detected in the network detection.

In one possible implementation, the target fault detection period is the period during which the receiving device receives the first data packet.

In some embodiments, the target communication path is a communication path between a sending device and a receiving device, and the target forwarding device can be at least one forwarding device on that communication path. That is, the target forwarding device can be a single forwarding device that, upon detecting a fault in the path segment connected to it on the target communication path, constructs a first message and sends the first message to the receiving device. Alternatively, the target forwarding device can be multiple forwarding devices that, upon detecting a fault in the path segment connected to them on the target communication path, construct a first message and send the first message to the receiving device. This application does not impose any limitations on this.

It should be noted that when a target forwarding device detects a fault in a path segment connected to it within the target communication path, the fault could be due to a problem with the link corresponding to that path segment, or it could be due to a fault with another forwarding device connected to that path segment. When a link corresponding to a path segment is faulty, both forwarding devices connected to that link can detect the link fault, and both can construct the first message. Conversely, when a forwarding device connected to a path segment is faulty, other forwarding devices connected to that device can detect the device fault, and one or more of these devices can construct the first message.

It should be understood that although the first message is constructed by the target forwarding device that detected the fault, if there are other forwarding devices between the target forwarding device and the receiving device, the target forwarding device can send the first message to the receiving device through these other forwarding devices. Of course, if the target forwarding device and the receiving device are directly connected, the target forwarding device can also directly send the first message to the receiving device. This application embodiment does not limit the number of other forwarding devices that exist between the target forwarding device and the receiving device.

In one possible implementation, the target forwarding device is located on the target communication path, and the target forwarding device includes at least one forwarding device that detects a fault in the target device or the target path segment on the target communication path. If the target device is faulty, the target forwarding device includes at least one forwarding device connected to the target device; if the target path segment is faulty, the target forwarding device includes a first forwarding device and/or a second forwarding device, the second forwarding device being a next-hop forwarding device communicatively connected to the first forwarding device, and the target path segment being the path segment between the first forwarding device and the second forwarding device.

The target forwarding device sends the first message to the receiving device in the following two ways:

In the first sending method, the target forwarding device directly sends the first message to a specific receiving device.

To facilitate understanding, the following example illustrates the process of the target forwarding device sending the first message, using the two network topologies shown above.

Referring to Figure 14, in the first network topology (2D Torus networking architecture), node 8 sends data packets to node 7 along the main path. Node 8 is the sending device and node 7 is the receiving device. During the data packet sending process, if there is a fault in the path segment between the sending device and the receiving device, such as a fault in the link between node 9 and node 10, then after detecting the link fault, nodes 9 and 10 can construct a first message and send the first message to node 7 along the determined backup path.

The data packet transmission path between node 8 and node 7 is node 8-node 9-node 10-node 11-node 7. When sending the first message, node 9 uses the transmission path node 9-node 5-node 6-node 7, and node 10 uses the transmission path node 10-node 11-node 7.

Of course, based on the actual network topology, nodes 8 and 7 can also use other paths to transmit data packets, and nodes 9 and 10 can also use other paths to send the first message. This application embodiment does not limit this.

Therefore, when a link corresponding to a path segment fails, the target forwarding device connected to that path segment can send the first message. The target forwarding device can be the preceding forwarding device connected to that path segment, the following forwarding device connected to that path segment, or both forwarding devices connected to that path segment.

Referring to Figure 15, in the first network topology (2D Torus networking architecture), node 8 sends data packets to node 7 along the main path. Node 8 is the sending device and node 7 is the receiving device. During the data packet sending process, if there is a fault in the forwarding device between the sending device and the receiving device, such as a fault in node 9, then after detecting the fault in node 9, nodes 5, 8, 10, and 13 can construct a first message and send the first message to node 7 along the determined backup path.

The data packet transmission path between nodes 8 and 7 is node 8-node 9-node 10-node 11-node 7. When sending the first message, the transmission path used by node 5 is node 5-node 6-node 7, the transmission path used by node 8 is node 8-node 4-node 7, the transmission path used by node 10 is node 10-node 11-node 7, and the transmission path used by node 13 is node 13-node 14-node 15-node 11-node 7.

Of course, based on the actual network topology, nodes 8 and 7 can also use other paths to transmit data packets, and nodes 5, 8, 10, and 13 can also use other paths to send the first message. This application embodiment does not limit this.

Therefore, when a forwarding device connected to one end of a path segment fails, the first message can be sent by another forwarding device connected to that forwarding device.

Referring to Figure 16, in the second network topology (Clos networking architecture), node A sends data packets to node H along the main path. Node A is the sending device and node H is the receiving device. During the data packet transmission process, if there is a fault in the path segment between the sending device and the receiving device, such as a fault in the link between node Leaf 1 and node Spine 1, then after detecting the link fault, nodes Leaf 1 and Spine 1 can construct a first message and send the first message to node H along the determined backup path.

The data packet transmission path between node A and node H is node A - node Leaf 1 - node Spine 1 - node Leaf n - node H. When sending the first message, node Leaf 1 uses the transmission path of node Leaf 1 - node Spine 16 - node Leaf n - node H, and node Spine 1 uses the transmission path of node Spine 1 - node Leaf n - node H.

Of course, based on the actual network topology, nodes A and H can also use other paths to transmit data packets, and nodes Leaf 1 and Spine 1 can also use other paths to send the first message. This application embodiment does not limit this.

Therefore, when a link corresponding to a path segment fails, the target forwarding device connected to that path segment can send the first message. The target forwarding device can be the preceding forwarding device connected to that path segment, the following forwarding device connected to that path segment, or both forwarding devices connected to that path segment.

Referring to Figure 17, in the second network topology (Clos networking architecture), node A sends data packets to node H along the main path. Node A is the sending device and node H is the receiving device. During the data packet transmission process, if there is a fault in the forwarding device between the sending device and the receiving device, such as a fault in node Spine 1, then after detecting the link fault, nodes Leaf 1 and Leaf n can construct a first message and send the first message to node H along the determined backup path.

The data packet transmission path between node A and node H is node A - node Leaf 1 - node Spine 1 - node Leaf n - node H. When sending the first message, node Leaf 1 uses the transmission path of node Leaf 1 - node Spine 16 - node Leaf n - node H, and node Leaf n uses the transmission path of node Leaf n - node H.

Of course, based on the actual network topology, nodes A and H can also use other paths to transmit data packets, and node Leaf 1 can also use other paths to send the first message. This application embodiment does not limit this.

Therefore, when a forwarding device connected to one end of a path segment fails, the first message can be sent by another forwarding device connected to that forwarding device.

It should be noted that when the target forwarding device sends the first message to the designated receiving device, this first message needs to carry the receiving device's device information to facilitate message forwarding by the next-hop forwarding device. This device information can include unique identifiers such as the receiving device's device type, device number, and device address.

In one possible implementation, the target forwarding device can maintain the DCNA information of the data packets forwarded during time period T, and carry the DCNA information of the receiving device in the first message when sending the first message to the designated receiving device.

The value of T can be equal to or greater than the value of the fault detection period mentioned above. This application does not impose any restrictions on this.

In the second transmission method, the target forwarding device broadcasts the first message to a specific receiving device.

Similarly, for ease of understanding, the following example illustrates the process of the target forwarding device sending the first message, using the two network topologies shown above.

Referring again to Figure 14, if node 10 detects a link failure, node 10 constructs a first message and broadcasts it to node 7. Node 10 can broadcast the first message to node 7 via at least two hops and to other nodes in this first network topology via at least one hop.

For example, node 10 sends the first message to nodes 11, 6 and 14 via a one-hop broadcast, to nodes 13, 14, 15, 8, 7, 5 and 2 via a two-hop broadcast, and to nodes 12, 4, 1, 0 and 3 via a three-hop broadcast.

Of course, after detecting a link failure, node 9 can also broadcast the first message to other nodes according to the above logic. This application embodiment does not limit this.

Referring again to Figure 15, if nodes 8 and 10 detect a fault in node 9, then nodes 8 and 9 construct a first message and broadcast it to node 7. Nodes 8 and 10 can broadcast the first message to node 7 via at least two hops and to other nodes in this first network topology via at least one hop.

For example, node 10 sends the first message to nodes 11, 6, and 14 via a one-hop broadcast, to nodes 13, 14, 15, 8, 7, 5, and 2 via a two-hop broadcast, and to nodes 12, 4, 1, 0, and 3 via a three-hop broadcast. Node 8 sends the first message to nodes 12, 4, and 11 via a one-hop broadcast, to nodes 13, 5, 0, and 12 via a two-hop broadcast, and to nodes 1 and 3 via a three-hop broadcast.

Of course, after detecting a fault in node 9, nodes 5 and 13 can also broadcast the first message to other nodes according to the above logic. This application embodiment does not limit this.

Referring again to Figure 16, if node Spine 1 detects a link failure, it constructs a first message and broadcasts it to node H. Node Spine 1 can broadcast the first message to node H via at least two hops and to other nodes in this second network topology via at least one hop.

For example, node Spine 1 sends the first message to nodes Leaf 2, Leaf n-1, and Leaf n via a one-hop broadcast, and then sends the first message to nodes C, D, E, F, G, and H via a two-hop broadcast.

Of course, after detecting a link failure, node Leaf 1 can also broadcast the first message to other nodes (such as node A and node B) according to the above logic. This application embodiment does not limit this.

Referring again to Figure 17, if node Leaf 1 detects a failure in node Spine 1, then node Leaf 1 constructs a first message and broadcasts it to node H. Node Leaf 1 can broadcast the first message to node H via at least three hops and to other nodes in this second network topology via at least one hop.

For example, node Leaf 1 sends the first message to nodes Spine 16, A, and B via a one-hop broadcast, to Leaf 2, Leaf n-1, and Leaf n via a two-hop broadcast, and to nodes C, D, E, F, G, and H via a three-hop broadcast.

Of course, nodes Leaf 1, Leaf 2, Leaf n-1, and Leaf n can also broadcast the first message to other nodes according to the above logic after detecting a fault in node Spine 1. This application embodiment does not limit this.

In some embodiments, regardless of the sending method used to send the first message, the target forwarding device may also set the SL of the message to the highest priority before sending it to the receiving device.

In summary, after introducing the method by which the target forwarding device sends the first message in conjunction with Figures 14-17, the information carried by the first message will be explained in detail below.

As explained above, the first message sent to the receiving device carries information about the forwarded data packets, which are data packets forwarded by the target forwarding device to the receiving device within the target fault detection period.

In some embodiments, the information of forwarded data packets includes identification information of the forwarded data packets. This identification information includes the transaction layer segment number (TASSN) of the forwarded data packet, or the packet sequence number (PSN) information of the forwarded data packet.

Based on this, the first message includes a first field, which is used to carry identification information of the forwarded data packet.

In one possible implementation, the information in the forwarded data packet also includes device address information, which at least indicates the address of the sending device.

Based on this, the first message includes a first field and a second field. The first field is used to carry the identification information of the forwarded data packet, and the second field is used to carry the device address information.

In one possible implementation, the first message may also carry other information about the data packet, such as SDNA, DCNA, TPH, TAH, etc.

As an example, as shown in Figure 18, the first message carries the SCNA and TASSN corresponding to the data packets forwarded by the target forwarding device to the receiving device within the target fault detection period.

As another example, as shown in Figure 19, the first message carries the SCNA and PSN corresponding to the data packets forwarded by the target forwarding device to the receiving device within the target fault detection period.

Here, SCNA represents the address information of the sending device within the UB-C domain. Thus, the receiving device can determine which sending device the data packet originated from based on the SCNA.

It should be noted that the meanings of the other fields in the first message, as well as the content they carry, can be found in the relevant descriptions of the complete compressed network header in the UB compressed message format. This application embodiment will not elaborate on these details here.

(2) Update the deduplication matching table based on the data packet information.

In other words, based on the first message, the information of the data packets already forwarded by the target forwarding device carried in it is added to the deduplication matching table, thus realizing the update of the deduplication matching table.

Based on the two types of information carried in the first message mentioned above, the deduplication matching table maintained by the receiving device in this embodiment of the application also includes the following two cases:

In the first scenario, as shown in Table 1 below, based on the SCNA and TASSN carried in the first message, the deduplication matching table maintained and managed by the receiving device stores the SCNA and TASSN corresponding to the data packets forwarded by the target forwarding device to the receiving device. In other words, the deduplication matching table records the correspondence between the SCNA and TASSN of the data packets.

Table 1

In one possible implementation, the deduplication matching table can also store the correspondence between the hash values of SCNA and TASSN.

In the second scenario, as shown in Table 2 below, based on the SCNA and PSN carried in the first message, the deduplication matching table maintained and managed by the receiving device stores information including: the SCNA, EPSN, and maximum out-of-order space corresponding to the data packets forwarded by the target forwarding device to the receiving device. In other words, the deduplication matching table records the correspondence between the SCNA, EPSN, and maximum out-of-order space of the data packets.

Table 2

It should be noted that the maximum allowed out-of-order space indicates the number of out-of-order packets that are allowed to be received before the expected ordered packets, which is not shown in Table 2 above.

Step 1102: If the first data packet carries a retransmission flag, then the first data packet is deduplicated according to the deduplication matching table so that there is a first data packet in the receiving device. The deduplication matching table stores information about the data packets that the receiving device has received.

In some embodiments, when a forwarding device needs to mark retransmitted data packets, it can carry a retransmission identifier in a reserved space in the data packet header.

As an example, as shown in Figure 20, the reserved bit "4" of Byte 1 in the data packet header is defined as a retransmission flag (denoted as R) to indicate whether the data packet is a retransmission. When the retransmission flag value is R=1, it indicates that the first data packet is a data packet that the target forwarding device retrieves from the retransmission buffer and retransmits.

In some embodiments, the first data packet also carries an end-of-retransmission flag, which indicates that the first data packet is the last data packet to undergo deduplication.

As an example, referring to Figure 20, the reserved "1" bit in Byte0 of the data packet header can be defined as the end retransmission flag (denoted as LR), and the information of this flag bit can be used to indicate whether the first data packet is the last retransmitted data packet. When the retransmission flag bit is R=1 and the end retransmission flag bit is LR=1, it means that the first data packet is the last retransmitted data packet sent from the retransmission buffer.

In one possible implementation, step 1102 above can be implemented as follows: if the first data packet is determined to be a duplicate data packet according to the deduplication matching table, the first data packet is discarded; if the first data packet is determined to be a non-duplicate data packet according to the deduplication matching table, the first data packet is retained.

Furthermore, if the first data packet is a non-duplicate data packet, after retaining the first data packet, it is also necessary to record the information of the first data packet in the deduplication matching table in order to update the deduplication matching table.

In one possible implementation, if the first data packet does not carry a retransmission flag, then no deduplication processing is performed on the first data packet. That is, if the first data packet does not carry a retransmission flag, the first data packet is processed using normal data processing logic, such as uploading the first data packet or storing the first data packet. This application embodiment does not impose any limitations on this.

As an example, referring to Figure 21, when data packet pkt1 does not carry a retransmission flag, it is directly uploaded. For data packet pkt2, if the deduplication trigger message carries information about data packet pkt2, upon receiving data packet pkt2, the deduplication matching table is queried based on this information to determine whether data packet pkt2 is a duplicate data packet. If data packet pkt2 is not a duplicate data packet, it is uploaded, and the deduplication matching table is updated based on its information. For the received data packet pkt3, if data packet pkt3 carries a retransmission flag, the deduplication matching table is queried based on this data packet pkt3 to determine whether it is a duplicate data packet. If data packet pkt3 is not a duplicate data packet, it is uploaded, and the deduplication matching table is updated based on its information.

In summary, in this embodiment, after receiving a data packet, if the data packet carries a retransmission flag, it indicates that the data packet is a retransmitted data packet in the network. Since the receiving device may have previously received the same retransmitted data packet, it needs to perform deduplication processing on the data packet when it carries a retransmission flag. This ensures that there are no duplicate data packets in the receiving device, allowing it to process network-forwarded data packets exactly once.

Moreover, compared to related technologies that rely on PSN to detect duplicate packets and require determining whether a packet is a duplicate based on the PSN carried by each packet, the receiving device in this embodiment can directly predict whether a packet is a duplicate based on whether it carries a retransmission flag during the packet reception process. Thus, it only needs to perform deduplication processing on a small number of packets marked with retransmission flags based on the deduplication matching table, without having to perform the operation of determining whether each packet is a duplicate. This greatly reduces the resource consumption of the receiving device and achieves lightweight deduplication of packets by the receiving device.

Based on the data packet deduplication method shown in the above embodiments, the data packet deduplication method provided in this application embodiment will be explained and described next through the interaction between the target forwarding device and the receiving device. The data packet deduplication method includes the following steps S1-S6:

Step S1: The target forwarding device sends the first data packet to the receiving device.

Step S2: The receiving device receives the first data packet.

Step S3: If the first data packet carries a retransmission flag, the receiving device performs deduplication processing on the first data packet according to the deduplication matching table so that there is a first data packet in the receiving device.

The deduplication matching table stores information about the data packets that the receiving device has received.

In one possible implementation, the process of deduplicating the first data packet according to the deduplication matching table can be as follows: if the first data packet is determined to be a duplicate data packet according to the deduplication matching table, the first data packet is discarded; if the first data packet is determined to be a non-duplicate data packet according to the deduplication matching table, the first data packet is retained.

Here, retaining the first data packet means storing the first data packet and recording its information in the deduplication matching table so as to update the deduplication matching table.

Step S4: If the first data packet does not carry a retransmission flag, the receiving device processes the first data packet according to the normal data processing logic.

It should be noted that the reason why the first data packet carries a retransmission identifier is because the target forwarding device detects a fault in the target communication path and performs FRR. In this case, in order to ensure the reliable transmission of data packets, the target forwarding device will mark the retransmitted first data packet, so that the first data packet carries a retransmission identifier.

The target communication path is the communication path between the sending device and the receiving device.

In one possible implementation, the first data packet also carries an end-of-retransmission flag, which indicates that the first data packet is the last data packet to undergo deduplication.

In other words, when the target forwarding device needs to retransmit multiple data packets to the receiving device, the target forwarding device can mark all of these multiple data packets with a retransmission flag to indicate to the receiving device that these multiple data packets are retransmitted data packets; at the same time, it can mark the last data packet among the multiple data packets with an end retransmission flag to indicate to the receiving device that this data packet is the last retransmitted data packet.

In this embodiment of the application, the end retransmission flag can instruct the receiving device to end the deduplication operation, that is, not to perform deduplication on the data packets received after the data packet carrying the end retransmission flag.

Furthermore, if the target forwarding device detects a fault in the target communication path, it will also send a first message to the receiving device to indicate that there may be retransmission of data packets later. Based on this, the data packet deduplication method provided in this application embodiment further includes the following steps:

Step S5: The target forwarding device sends a first message to the receiving device. The first message carries information about the forwarded data packets. The forwarded data packets are the data packets that the target forwarding device forwards to the receiving device within the target fault detection period.

The first message includes a first field, which carries a message type. The message type indicates that the first message is a deduplication trigger message. The deduplication trigger message is used to instruct the receiving device to perform deduplication processing on the data packets received after the first message.

In practice, the first message is used to indicate that a fault is detected in the target communication path during the target fault detection period. The target communication path is the communication path between the sending device and the receiving device.

In this scenario, the target forwarding device is located on the target communication path, and the target forwarding device includes at least one forwarding device that detects a fault in the target device or the target path segment on the target communication path. If the target device is faulty, the target forwarding device includes at least one forwarding device connected to the target device; if the target path segment is faulty, the target forwarding device includes a first forwarding device and/or a second forwarding device, where the second forwarding device is a next-hop forwarding device in the target communication path that is communicatively connected to the first forwarding device, and the target path segment is the path segment between the first forwarding device and the second forwarding device.

In one possible implementation, the information of the forwarded data packet includes identification information of the forwarded data packet, which may include the transaction layer segment number of the forwarded data packet or the packet sequence number information of the forwarded data packet.

In one possible implementation, the information in the forwarded data packet also includes device address information, which at least indicates the address of the sending device.

Step S6: The receiving device receives the first message and updates the deduplication matching table based on the information of the forwarded data packets.

The following description, in conjunction with the accompanying drawings, provides an exemplary illustration of the process by which the target forwarding device marks data packets as retransmission targets, sends the first message to the receiving device, and the receiving device implements lightweight deduplication, in order to further illustrate its specific implementation process.

Referring back to Figure 12, if the sending device sends data packets to the receiving device via the main path, i.e., S1 switch - S3 switch - S4 switch - receiving device, and the S1 switch and S3 switch detect a link failure, they can construct a first message immediately, set the SL level of the message to the highest priority, and send the first message to the receiving device.

Taking an S3 switch as an example, an S3 switch sends out a retriggered message in two ways:

(1) The S3 switch maintains the DCNA information of the data packets forwarded within the T time period. At this time, the first message can be sent directly to the receiving device through the backup path based on the DCNA information.

(2) The S3 switch sends the first message via broadcast.

The backup path is S1 switch - S2 switch - S3 switch - S4 switch - receiving device.

Please refer to Figure 22. For the S2 switch, when it receives a data packet, its internal logic for processing the data packet is as follows: After receiving the block/flit of the data packet from a certain port, if the data packet is verified to be correct or successfully decoded, it is sent to the service support unit (SSU). While waiting for all blocks/flits of a certain data packet to be received and the data packet to be assembled, the SSU searches the data packet forwarding table based on the DCNA information in LPH. After the data packet is assembled and the forwarding table result is found, the data packet is forwarded out from a certain port, and a backup of the data packet is saved to the retransmission buffer.

For the S2 switch, after receiving the ACK message sent by the next-hop S3 switch via LPH/LBH, it releases the corresponding space in the retransmission buffer. Furthermore, it saves information about the data packet (such as some or all packet header information, or hash values) to the buffer to facilitate the construction of the first message.

In this embodiment of the application, for switches in the network, the retransmission buffer needs to be managed at the packet level. If a portion of a packet's block/flit does not receive an ACK message, the corresponding range of resources in the retransmission buffer will not be released temporarily.

As shown in Figure 23, a packet (pkt) consists of four blocks/flits, from Addr3 to Addr6. If Addr5 does not receive an ACK message, the switch will resend Addr3-Addr6 via the backup path after a Free Response (FRR), and will not release the interval resources corresponding to that packet temporarily.

Furthermore, after receiving a data packet, the receiving device parses the packet header. As shown in Figure 24, if the receiving device parses the packet header and determines that the message type carried in the first field of the data packet is a deduplication trigger message, it starts its own deduplication module and updates the deduplication matching table based on the information carried in the first message. As shown in Figure 25, if the receiving device determines that the data packet carries a retransmission identifier after parsing the packet header, it needs to query the deduplication matching table based on the information of the data packet to detect whether the data packet is a duplicate data packet. If the data packet is a duplicate data packet, it discards the data packet; if it is not a duplicate data packet, it updates the deduplication matching table based on the information of the data packet and uploads the data packet to the upper-layer module.

Therefore, for data packets forwarded over a network, if the data packet is a retransmitted data packet taken from the retransmission buffer, the data packet will carry a retransmission identifier. In other words, for the forwarding device in the network, when forwarding data packets to the receiving device, it marks a small number of retransmitted data packets to carry a retransmission identifier.

For the receiving device, when it receives a data packet, if the packet carries a retransmission flag, it indicates that the packet is a retransmitted data packet in the network. Since the receiving device may have already received the same retransmitted data packet before, it needs to perform deduplication processing on the packet. This ensures that there are no duplicate data packets in the receiving device, allowing it to process network-forwarded data packets exactly once.

Moreover, compared to related technologies that rely on PSN to detect duplicate packets and require determining whether a packet is a duplicate based on the PSN carried by each packet, the receiving device in this embodiment can directly predict whether a packet is a duplicate based on whether it carries a retransmission flag during the packet reception process. Thus, it only needs to perform deduplication processing on a small number of packets marked with retransmission flags based on the deduplication matching table, without having to perform the operation of determining whether each packet is a duplicate. This greatly reduces the resource consumption of the receiving device and achieves lightweight deduplication of packets by the receiving device.

It should be noted that the relevant content of Figures 22-25 can also be referred to the explanation in the method embodiment of Figure 11 above, and will not be repeated here.

Figure 26 is a schematic diagram of a data packet deduplication device provided in an embodiment of this application. The data packet deduplication device can be implemented by software, hardware, or a combination of both as part or all of the receiving end device. Referring to Figure 26, the data packet deduplication device 2600 includes: a first receiving module 2601 and a deduplication module 2602.

The first receiving module 2601 is used to receive the first data packet;

The deduplication module 2602 is used to perform deduplication processing on the first data packet according to the deduplication matching table if the first data packet carries a retransmission identifier, so that there is a first data packet in the receiving device. The deduplication matching table stores information about the data packets that the receiving device has received.

In one possible implementation, the deduplication module 2602 is specifically used for:

If the first data packet is determined to be a duplicate data packet according to the deduplication matching table, then the first data packet is discarded;

If the first data packet is determined to be a non-duplicate data packet according to the deduplication matching table, then the first data packet is retained.

In one possible implementation, the data packet deduplication device 2600 further includes:

The update module is used to store the first data packet and record the information of the first data packet in the deduplication matching table in order to update the deduplication matching table.

In one possible implementation, the data packet deduplication device 2600 further includes:

The second receiving module is also used to receive a first message, which carries information about a forwarded data packet. The forwarded data packet is a data packet that the target forwarding device forwards to the receiving device within the target fault detection period. The first message is used to indicate that a fault is detected in the target communication path within the target fault detection period. The target communication path is the communication path between the sending device and the receiving device.

The update module is used to update the deduplication matching table based on information from forwarded data packets.

In one possible implementation, the first message includes a first field carrying a message type. The message type indicates that the first message is a deduplication trigger message, which is used to instruct the receiving device to perform deduplication processing on data packets received after the first message.

In one possible implementation, the target forwarding device is located on the target communication path, and the target forwarding device includes at least one forwarding device that detects a fault in the target device or the target path segment on the target communication path.

If the target device is faulty, the target forwarding device includes at least one forwarding device connected to the target device;

If the target path segment is faulty, the target forwarding device includes a first forwarding device and/or a second forwarding device. The second forwarding device is the next-hop forwarding device in the target communication path that is connected to the first forwarding device. The target path segment is the path segment between the first forwarding device and the second forwarding device.

In one possible implementation, the information of the forwarded data packet includes the identification information of the forwarded data packet;

The identification information includes the transaction layer segment number of the forwarded data packet or the packet sequence number of the forwarded data packet.

In one possible implementation, the deduplication trigger message includes a first field, which carries identification information of the forwarded data packet.

In one possible implementation, the information in the forwarded data packet also includes device address information, which at least indicates the address of the sending device.

In one possible implementation, the deduplication trigger message includes a first field and a second field. The first field is used to carry the identification information of the forwarded data packet, and the second field is used to carry the device address information.

In one possible implementation, the first data packet also carries an end-of-retransmission flag, which indicates that the first data packet is the last data packet to undergo deduplication.

In this embodiment, if a received data packet carries a retransmission flag, it indicates that the data packet is a retransmitted data packet in the network. Since the receiving device may have previously received the same retransmitted data packet, it needs to perform deduplication processing on the packet. This ensures that there are no duplicate data packets in the receiving device, allowing it to process network-forwarded data packets exactly once.

Moreover, compared to related technologies that rely on PSN to detect duplicate packets and require determining whether a packet is a duplicate based on the PSN carried by each packet, the receiving device in this embodiment can directly predict whether a packet is a duplicate based on whether it carries a retransmission flag during the packet reception process. Thus, it only needs to perform deduplication processing on a small number of packets marked with retransmission flags based on the deduplication matching table, without having to perform the operation of determining whether each packet is a duplicate. This greatly reduces the resource consumption of the receiving device and achieves lightweight deduplication of packets by the receiving device.

It should be noted that the data packet deduplication device 2600 provided in the above embodiments is only illustrated by the division of the above functional modules when performing deduplication processing on received data packets. In practical applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above. In addition, the data packet deduplication device 2600 provided in the above embodiments and the data packet deduplication method embodiments belong to the same concept, and the specific implementation process can be found in the method embodiments, which will not be repeated here.

Figure 27 is a schematic diagram of another data packet deduplication device provided in an embodiment of this application. This data packet deduplication device can be implemented by software, hardware, or a combination of both as part or all of the target forwarding device. Referring to Figure 27, the data packet deduplication device 2700 includes: a first sending module 2701.

The first sending module 2701 is used to send a first data packet to the receiving device to instruct the receiving device to perform deduplication processing on the first data packet according to the deduplication matching table, so that there is a first data packet in the receiving device; wherein, the first data packet carries a retransmission identifier, and the deduplication matching table stores information on the data packets that the receiving device has received.

In one possible implementation, the data packet deduplication device 2700 further includes:

The second sending module is used to send a first message to the receiving device to instruct the receiving device to update the deduplication matching table based on the information of the forwarded data packets carried in the first message.

Among them, the forwarded data packet is the data packet forwarded by the target forwarding device to the receiving device within the target fault detection period, and the first message is used to indicate that a fault is detected in the target communication path within the target fault detection period. The target communication path is the communication path between the sending device and the receiving device.

In one possible implementation, the first message includes a first field carrying a message type. The message type indicates that the first message is a deduplication trigger message, which is used to instruct the receiving device to perform deduplication processing on data packets received after the first message.

In one possible implementation, the target forwarding device is located on the target communication path, and the target forwarding device includes at least one forwarding device that detects a fault in the target device or the target path segment on the target communication path.

If the target device is faulty, the target forwarding device includes at least one forwarding device connected to the target device;

If the target path segment is faulty, the target forwarding device includes a first forwarding device and/or a second forwarding device. The second forwarding device is the next-hop forwarding device in the target communication path that is connected to the first forwarding device. The target path segment is the path segment between the first forwarding device and the second forwarding device.

In one possible implementation, the information of the forwarded data packet includes identification information of the forwarded data packet, which may include the transaction layer segment number of the forwarded data packet or the packet sequence number information of the forwarded data packet.

In one possible implementation, the information in the forwarded data packet also includes device address information, which at least indicates the address of the sending device.

In one possible implementation, the first data packet also carries an end-of-retransmission flag, which indicates that the first data packet is the last data packet to undergo deduplication.

In this embodiment of the application, for data packets forwarded by the network, if the data packet is a retransmitted data packet taken from the retransmission buffer, the data packet will carry a retransmission identifier. In other words, for the forwarding device in the network, when forwarding data packets to the receiving device, it marks a small number of retransmitted data packets to carry a retransmission identifier.

Moreover, compared to related technologies that rely on PSN to detect duplicate packets, requiring each packet to carry a PSN to determine if it is a duplicate, the forwarding device in this embodiment can tag a small number of retransmitted packets. This allows the receiving device to directly predict whether a packet is a retransmitted packet based on whether it carries a retransmission flag. Consequently, only a small number of packets with retransmission flags need to be deduplicated based on the deduplication matching table. This eliminates the need to perform a deduplication check on each packet individually, significantly reducing the resource consumption of the receiving device and achieving lightweight deduplication of packets.

It should be noted that the data packet deduplication device 2700 provided in the above embodiments, when marking retransmitted data packets and sending the first message, is only illustrated by the division of the above functional modules. In practical applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above. In addition, the data packet deduplication device 2700 provided in the above embodiments and the data packet deduplication method embodiments belong to the same concept, and its specific implementation process is detailed in the method embodiments, which will not be repeated here.

This application also provides a computer-readable storage medium storing a computer program that, when run on a computer or processor, causes the computer or processor to perform the steps of the data packet deduplication method shown in the above embodiments.

This application also provides a computer program product comprising computer instructions that, when executed by a computer or processor, cause the computer or processor to perform the steps of the data packet deduplication method described in the above embodiments. Alternatively, a computer program is provided that, when run on a computer or processor, causes the computer or processor to perform the steps of the data packet deduplication method described in the above embodiments.

In the above embodiments, implementation can be achieved, in whole or in part, through software, hardware, firmware, or any combination thereof. When implemented in software, it can be implemented, in whole or in part, as a computer program product. The computer program product includes one or more computer instructions. When the computer instructions are loaded and executed on a computer, all or part of the processes or functions described in the embodiments of this application are generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions can be transmitted from one website, computer, server, or data center to another via wired (e.g., coaxial cable, fiber optic, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium can be any available medium accessible to a computer, or a data storage device such as a server or data center that integrates one or more available media. The available medium can be a magnetic medium (e.g., floppy disk, hard disk, magnetic tape), an optical medium (e.g., digital versatile disc (DVD)), or a semiconductor medium (e.g., solid-state disk (SSD)). It is worth noting that the computer-readable storage medium mentioned in the embodiments of this application can be a non-volatile storage medium; in other words, it can be a non-transient storage medium.

It should be understood that "multiple" as mentioned herein refers to two or more. In the description of the embodiments of this application, unless otherwise stated, "/" means "or," for example, A/B can mean A or B; "and/or" in this document is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and/or B can represent: A existing alone, A and B existing simultaneously, and B existing alone. In addition, to facilitate a clear description of the technical solutions of the embodiments of this application, the terms "first," "second," etc., are used in the embodiments of this application to distinguish identical or similar items with substantially the same function and effect. Those skilled in the art will understand that the terms "first," "second," etc., do not limit the quantity or execution order, and the terms "first," "second," etc., do not necessarily imply that they are different.

It should be noted that the information (including but not limited to user device information, user personal information, etc.), data (including but not limited to data used for analysis, data stored, data displayed, etc.) and signals involved in the embodiments of this application are all authorized by the user or fully authorized by all parties, and the collection, use and processing of related data must comply with the relevant laws, regulations and standards of the relevant countries and regions.

The above descriptions are embodiments provided in this application and are not intended to limit this application. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the protection scope of this application.

Claims (21)

A data packet deduplication method, characterized in that it is applied to a receiving device, the method comprising: Receive the first data packet; If the first data packet carries a retransmission flag, the first data packet is deduplicated according to the deduplication matching table so that there is a first data packet in the receiving device. The deduplication matching table stores information about the data packets that the receiving device has received. The method as described in claim 1, wherein the step of deduplicating the first data packet according to the deduplication matching table includes: If the first data packet is determined to be a duplicate data packet according to the deduplication matching table, then the first data packet is discarded; If the first data packet is determined to be a non-duplicate data packet according to the deduplication matching table, then the first data packet is retained. The method as described in claim 2, wherein retaining the first data packet includes: The first data packet is stored, and its information is recorded in the deduplication matching table to update the deduplication matching table. The method according to any one of claims 1-3, characterized in that the method further comprises: A first message is received, the first message carrying information about a forwarded data packet, the forwarded data packet being a data packet forwarded by the target forwarding device to the receiving device within the target fault detection period, the first message being used to indicate that a fault is detected in the target communication path within the target fault detection period, the target communication path being the communication path between the sending device and the receiving device; The deduplication matching table is updated based on the information of the forwarded data packets. The method as described in claim 4, wherein the first message includes a first field, the first field carries a message type, the message type indicates that the first message is a deduplication trigger message, and the deduplication trigger message is used to instruct the receiving device to perform deduplication processing on data packets received after the first message. The method as described in claim 4 or 5, wherein the target forwarding device is located on the target communication path, and the target forwarding device includes at least one forwarding device that detects a fault in the target device or the target path segment on the target communication path; If the target device is faulty, the target forwarding device includes at least one forwarding device connected to the target device; If the target path segment is faulty, the target forwarding device includes a first forwarding device and/or a second forwarding device, wherein the second forwarding device is the next-hop forwarding device in the target communication path that is connected to the first forwarding device, and the target path segment is the path segment between the first forwarding device and the second forwarding device. The method according to any one of claims 4-6, wherein the information of the forwarded data packet includes the identification information of the forwarded data packet, the identification information including the transaction layer segment number of the forwarded data packet or the packet sequence number information of the forwarded data packet. The method as described in claim 7, wherein the information of the forwarded data packet further includes device address information, the device address information indicating at least the address of the sending device. The method according to any one of claims 1-8, wherein the first data packet further carries an end-of-retransmission identifier, the end-of-retransmission identifier indicating that the first data packet is the last data packet to undergo deduplication. A data packet deduplication method, characterized in that it is applied to a target forwarding device, the method comprising: A first data packet is sent to the receiving device to instruct the receiving device to perform deduplication processing on the first data packet according to the deduplication matching table, so that there is a first data packet in the receiving device; wherein, the first data packet carries a retransmission identifier, and the deduplication matching table stores information on data packets that the receiving device has received. The method as described in claim 10, characterized in that the method further comprises: Send a first message to the receiving device to instruct the receiving device to update the deduplication matching table based on the information of the forwarded data packets carried in the first message; Wherein, the forwarded data packet is the data packet forwarded by the target forwarding device to the receiving device within the target fault detection period, and the first message is used to indicate that a fault is detected in the target communication path within the target fault detection period, and the target communication path is the communication path between the sending device and the receiving device. The method as described in claim 11, wherein the first message includes a first field, the first field carries a message type, the message type indicating that the first message is a deduplication trigger message, and the deduplication trigger message is used to instruct the receiving device to perform deduplication processing on data packets received after the first message. The method according to any one of claims 10-12, wherein the target forwarding device is located on the target communication path, and the target forwarding device includes at least one forwarding device that detects a fault in the target device or the target path segment on the target communication path; If the target device is faulty, the target forwarding device includes at least one forwarding device connected to the target device; If the target path segment is faulty, the target forwarding device includes a first forwarding device and/or a second forwarding device, wherein the second forwarding device is the next-hop forwarding device in the target communication path that is connected to the first forwarding device, and the target path segment is the path segment between the first forwarding device and the second forwarding device. The method as described in claim 11, wherein the information of the forwarded data packet includes the identification information of the forwarded data packet, the identification information including the transaction layer segment number of the forwarded data packet or the packet sequence number information of the forwarded data packet. The method as described in claim 14, wherein the information of the forwarded data packet further includes device address information, the device address information indicating at least the address of the sending device. The method as described in any one of claims 10-15, wherein the first data packet further carries an end-of-retransmission identifier, the end-of-retransmission identifier indicating that the first data packet is the last data packet to undergo deduplication. A data packet deduplication device, characterized in that it is included in a receiving end device, the device comprising: The first receiving module is used to receive the first data packet; The deduplication module is used to perform deduplication processing on the first data packet according to the deduplication matching table if the first data packet carries a retransmission identifier, so that there is a first data packet in the receiving device. The deduplication matching table stores information about the data packets that the receiving device has received. A data packet deduplication device, characterized in that it is included in a target forwarding device, the device comprising: The sending module is configured to send a first data packet to the receiving device to instruct the receiving device to perform deduplication processing on the first data packet according to the deduplication matching table, so that there is a first data packet in the receiving device; wherein, the first data packet carries a retransmission identifier, and the deduplication matching table stores information on data packets that the receiving device has received. A computer device, characterized in that the computer device includes a processor and a memory; The memory is used to store computer programs; The processor is configured to execute the computer program to implement the steps of the method according to any one of claims 1-9, or to implement the steps of the method according to any one of claims 10-16. A computer-readable storage medium, characterized in that the storage medium stores a computer program, which, when executed by a processor, implements the steps of the method according to any one of claims 1-9, or implements the steps of the method according to any one of claims 10-16. A computer program product, characterized in that the computer program product stores computer instructions, which, when executed by a processor, implement the steps of the method according to any one of claims 1-9, or implement the steps of the method according to any one of claims 10-16.