[go: up one dir, main page]

CN119232639A - Method and device for determining path control information, storage medium and electronic device - Google Patents

Method and device for determining path control information, storage medium and electronic device Download PDF

Info

Publication number
CN119232639A
CN119232639A CN202310794561.2A CN202310794561A CN119232639A CN 119232639 A CN119232639 A CN 119232639A CN 202310794561 A CN202310794561 A CN 202310794561A CN 119232639 A CN119232639 A CN 119232639A
Authority
CN
China
Prior art keywords
path
control information
path control
information
determining
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN202310794561.2A
Other languages
Chinese (zh)
Inventor
李和松
段威
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
ZTE Corp
Original Assignee
ZTE Corp
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by ZTE Corp filed Critical ZTE Corp
Priority to CN202310794561.2A priority Critical patent/CN119232639A/en
Priority to PCT/CN2024/079407 priority patent/WO2025001256A1/en
Publication of CN119232639A publication Critical patent/CN119232639A/en
Pending legal-status Critical Current

Links

Classifications

    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04LTRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L45/00Routing or path finding of packets in data switching networks
    • H04L45/24Multipath
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04LTRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L45/00Routing or path finding of packets in data switching networks
    • H04L45/02Topology update or discovery

Landscapes

  • Engineering & Computer Science (AREA)
  • Computer Networks & Wireless Communication (AREA)
  • Signal Processing (AREA)
  • Data Exchanges In Wide-Area Networks (AREA)

Abstract

本申请实施例提出了一种路径控制信息的确定方法、装置、存储介质及电子装置,该方法包括:根据客户端输入的路径约束信息在拓扑数据库中查询所述路径约束信息对应的可用路径;通过南向接口适配器确定所述路径约束信息所指示的物理网络中所述可用路径的路径控制信息;将所述路径控制信息保存在目标数据库中,其中,所述目标数据库用于向所述客户端提供所述路径控制信息。采用上述技术方案,解决了相关技术中现有的数据中心网络无法实现全量的路径控制信息发现,进而进行绝对路径控制的问题。

The embodiment of the present application proposes a method, device, storage medium and electronic device for determining path control information, the method comprising: querying the available path corresponding to the path constraint information in the topology database according to the path constraint information input by the client; determining the path control information of the available path in the physical network indicated by the path constraint information through the southbound interface adapter; storing the path control information in the target database, wherein the target database is used to provide the path control information to the client. The above technical solution solves the problem that the existing data center network in the related technology cannot realize the full amount of path control information discovery, and then perform absolute path control.

Description

Method and device for determining path control information, storage medium and electronic device
Technical Field
The present application relates to the field of communications, and in particular, to a method and apparatus for determining path control information, a storage medium, and an electronic device.
Background
In a typical data center network there are often very many physical network paths between communication nodes, on which network devices run routing protocols and perform IP-based efficient traffic forwarding. The redundant design of the multiple physical links is an important means for guaranteeing the east-west traffic throughput performance and reliability of the common data center, but also adds a great technical obstacle to the discovery of network paths and the control of forwarding paths.
Specifically, as shown in fig. 1, in a typical data center network, a multi-level CLOS (CLOS topology, a non-blocking networking technology) architecture is generally adopted to implement a networking architecture, two communication nodes (S1, sn) often have a very large number of physical redundant paths, the network performs IP-based traffic forwarding, and a Hash algorithm (Hash Function, abbreviated as Hash) is performed through a five-tuple of a message to implement load balancing of equivalent multipaths. Generally, the network has good expansibility and stability, and has no strong demand for traffic engineering, so that the network can well meet the communication demand of the service by carrying out load balancing through a HASH algorithm. However, with the rise of distributed storage, high-performance computing and AI distributed machine learning scenarios, traffic flows are extremely sensitive to delay, jitter and throughput performance indexes caused by network congestion, and a sender node is required to discover all available forwarding paths in the network and control the traffic to take specified deterministic paths according to the discovered paths, so as to realize traffic engineering dominated by the sender, thereby avoiding congestion as much as possible and achieving the purpose of optimizing traffic forwarding performance.
Aiming at the problems of path discovery and flow control, two main technical implementation schemes exist in the industry at present:
the scheme one is based on Traceroute technology (a general method for realizing path discovery by sending probe packets) path discovery and flow control;
among the related technical solutions disclosed, the method based on the Traceroute mechanism is most widely used and widely adopted by the industry, and the specific method can be seen in reference Clove:Congestion-Aware Load Balancing at the Virtual Edge. The method is generally that a sender node (usually a server or an intelligent network card) actively sends a UDP/TCP message, discovers a network path by changing a source port and a TTL value of the message, forms a path database of the sender, and then sends a data message by using the same five-tuple (source IP (IP Address), destination IP, protocol family, source port and destination port), thereby realizing forwarding path control of flow.
The method has the obvious defects that (1) due to the existence of hash collision, a Traceroute mechanism cannot ensure the discovery of 100% network paths, (2) when a network link or network equipment fails, is restarted or is configured to be changed, the discovered paths cannot be updated in time, and the information for flow forwarding path control can be invalid, (3) the path discovery is realized by depending on a CPU (Central processing Unit) or an intelligent network card at a transmitting end, so that the cost of computing resources and the complexity of software logic are increased;
path discovery and flow control based on MPLS/SR technology;
In some technical implementations, by introducing support of multiprotocol label switching (Multiprotocol Label Switching, abbreviated as MPLS) or SR (Segment Routing) features in a network, discovery of network paths is achieved through control plane protocols, and control of traffic forwarding paths is achieved through mechanisms of traffic engineering or source routing, as in typical application cases such as reference Predictable vFabric on Informative DATA PLANE. While MPLS/SR technology has matured in its application to wide area networks, it has not been widely deployed in data centers, allowing for a full range of network path discovery and absolute path control. If the support of MPLS/SR technology is to be introduced in a data center network, the hardware and software of existing data center switches and routers need to be updated, and in particular, the updating of hardware brings about very expensive cost, so that the method is not applied to the existing network on a large scale.
In summary, implementing the discovery of the path and the control of the traffic forwarding path in the existing data center network is an explicit requirement of the current emerging service scenario, and the existing technical solution has a series of problems of insufficient coverage, untimely information, high deployment cost, and the like, and a new method is needed to solve the problems.
Aiming at the problem that the existing data center network cannot realize the discovery of the full amount of path control information and further carry out absolute path control in the related technology, no effective solution is proposed at present.
Accordingly, there is a need for improvements in the related art to overcome the drawbacks of the related art.
Disclosure of Invention
The embodiment of the application provides a method and a device for determining path control information, a storage medium and an electronic device, which at least solve the problem that the existing data center network cannot realize the discovery of the full amount of path control information and further carry out absolute path control.
According to one aspect of the embodiment of the application, a method for determining path control information is provided, and the method is applied to network multipath service NMPS and comprises the steps of inquiring available paths corresponding to path constraint information in a topology database according to the path constraint information input by a client, determining the path control information of the available paths in a physical network indicated by the path constraint information through a southbound interface adapter, and storing the path control information in a target database, wherein the target database is used for providing the path control information for the client.
In an exemplary embodiment, the method comprises the steps of inquiring available paths corresponding to path constraint information in a topology database according to the path constraint information input by a client, wherein the path constraint condition in the path constraint information is determined, the path constraint condition comprises a source IP address, a destination IP address and a constraint condition, and the available paths corresponding to the path constraint information are inquired in the topology database according to the path constraint condition.
In one exemplary embodiment, determining, by the southbound interface adapter, path control information of the available path in the physical network indicated by the path constraint information includes determining a traversal range corresponding to a variable in the path constraint information, determining, by the southbound interface adapter, an actual forwarding path corresponding to target path control information within the traversal range, and determining, if the actual forwarding path corresponding to the target path control information is consistent with the available path, the target path control information as path control information of the available path.
In an exemplary embodiment, after determining, by the southbound interface adapter, path control information corresponding to each path in the traversal range, the method further includes continuing the traversal process until control information is found that the actual forwarding path is consistent with the available path, if the actual forwarding path corresponding to the target path control information is inconsistent with the available path.
In an exemplary embodiment, determining, by the southbound interface adapter, an actual forwarding path corresponding to the target path control information in the traversal range includes determining target path control information in the traversal range, determining, by the southbound interface adapter, outbound interface information of the target path control information, and concatenating the target path control information and the outbound interface information to obtain the actual forwarding path.
In an exemplary embodiment, before determining, by the southbound interface adapter, an actual forwarding path corresponding to the target path control information in the traversal range, the method further includes determining a path control information type corresponding to the path constraint information, where the path control information type includes, but is not limited to, one of a five-tuple and a four-tuple, the path constraint information further includes path quality description information, and the path quality description information includes at least one of a delay, a jitter, and a packet loss, and the target path control information is the same as the path control information type of the path constraint information.
In one exemplary embodiment, the method comprises the steps of determining an updated available path according to incremental change information and the available path when the incremental change information of the topology database is received, determining updated path control information corresponding to the updated available path in the traversing range of the physical network indicated by the path constraint information through a southbound interface adapter, and updating the path control information in the target database through the updated path control information.
In an exemplary embodiment, after updating the path control information in the target database with the updated path control information, the method further comprises pushing updated path control information to the client by determining path subscription information of the client, pushing the updated path control information to the client according to the subscription information, and returning the updated path control information to the client upon receiving a poll message of the client.
According to still another aspect of the embodiment of the present application, there is further provided a system for determining path control information, which includes a network multipath service NMPS configured to query, in a topology database, an available path corresponding to path constraint information according to the path constraint information input by a client, a southbound interface adapter configured to determine path control information of the available path in a physical network indicated by the path constraint information, and a target database configured to store the path control information and provide the path control information to the client.
In one exemplary embodiment, the system further includes a topology database for providing available path information to the network multipath service NMPS and, in the event of a network topology change, sending incremental change information to the network multipath service NMPS.
According to still another aspect of the embodiments of the present application, there is also provided a computer-readable storage medium having a computer program stored therein, wherein the computer program is configured to perform the above-described path control information determination method when run.
According to still another aspect of the embodiments of the present application, there is further provided an electronic device including a memory, a processor, and a computer program stored on the memory and executable on the processor, wherein the processor executes the method for determining path control information by using the computer program.
The method comprises the steps of inquiring available paths corresponding to path constraint information in a topology database according to the path constraint information input by a client, determining path control information of the available paths in a physical network indicated by the path constraint information through a southbound interface adapter, and storing the path control information in a target database, wherein the target database is used for providing the path control information for the client. That is, the network multipath service NMPS queries the available paths in the topology database according to the path constraint information input by the client, and queries the path control information corresponding to the available paths through the southbound interface adapter, so as to provide the path control information for the client, thereby solving the technical problem that the existing data center network cannot realize the discovery of the full amount of path control information and further perform absolute path control.
Drawings
The accompanying drawings, which are included to provide a further understanding of the application and are incorporated in and constitute a part of this specification, illustrate embodiments of the application and together with the description serve to explain the application and do not constitute a limitation on the application. In the drawings:
FIG. 1 is a schematic diagram of a typical data center network in the related art;
fig. 2 is a block diagram of a hardware configuration of a computer terminal of a method for determining path control information according to an embodiment of the present application;
Fig. 3 is a flowchart (one) of a method of determining path control information according to an embodiment of the present application;
FIG. 4 is a schematic diagram of a path control information determining method according to an embodiment of the present application;
FIG. 5 is an A1 interface logic diagram of path control information according to an embodiment of the present application;
FIG. 6 is an A2 interface logic diagram of path control information according to an embodiment of the present application;
FIG. 7 is an A3 interface logic diagram of path control information according to an embodiment of the present application;
FIG. 8 is an A4 interface logic diagram of path control information according to an embodiment of the present application;
fig. 9 is a flowchart (two) of a method of determining path control information according to an embodiment of the present application;
fig. 10 is a flowchart (iii) of a method of determining path control information according to an embodiment of the present application;
Fig. 11 is a path control information update flowchart of a path control information determination method according to an embodiment of the present application;
Fig. 12 is a block diagram of a configuration of a path control information determining apparatus according to an embodiment of the present application.
Detailed Description
In order that those skilled in the art will better understand the present application, a technical solution in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings in which it is apparent that the described embodiments are only some embodiments of the present application, not all embodiments. All other embodiments, which can be made by those skilled in the art based on the embodiments of the present application without making any inventive effort, shall fall within the scope of the present application.
It should be noted that the terms "first," "second," and the like in the description and the claims of the present application and the above figures are used for distinguishing between similar objects and not necessarily for describing a particular sequential or chronological order. It is to be understood that the data so used may be interchanged where appropriate such that the embodiments of the application described herein may be implemented in sequences other than those illustrated or otherwise described herein. Furthermore, the terms "comprises," "comprising," and "having," and any variations thereof, are intended to cover a non-exclusive inclusion, such that a process, method, system, article, or apparatus that comprises a list of steps or elements is not necessarily limited to those steps or elements expressly listed but may include other steps or elements not expressly listed or inherent to such process, method, article, or apparatus.
The method embodiments provided in the embodiments of the present application may be executed in a computer terminal or similar computing device. Taking the operation on a computer terminal as an example, fig. 2 is a block diagram of the hardware structure of the computer terminal of the path control information determining method according to the embodiment of the present application. As shown in fig. 2, the computer terminal may include one or more (only one is shown in fig. 2) processors 202 (the processor 202 may include, but is not limited to, a microprocessor (Microprocessor Unit, abbreviated MPU) or a programmable logic device (Programmable logic device, abbreviated PLD)) and a memory 204 for storing data, and in an exemplary embodiment, the computer terminal may further include a transmission device 206 for communication functions and an input-output device 208. It will be appreciated by those skilled in the art that the configuration shown in fig. 2 is merely illustrative and is not intended to limit the configuration of the computer terminal described above. For example, a computer terminal may also include more or fewer components than shown in FIG. 2, or have a different configuration than the equivalent functions shown in FIG. 2 or more than the functions shown in FIG. 2.
The memory 204 may be used to store a computer program, for example, a software program of application software and a module, such as a computer program corresponding to a method for determining path control information in an embodiment of the present application, and the processor 202 executes the computer program stored in the memory 204 to perform various functional applications and data processing, that is, to implement the above-described method. Memory 204 may include high-speed random access memory, and may also include non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some examples, memory 204 may further include memory located remotely from processor 202, which may be connected to the computer terminal via a network. Examples of such networks include, but are not limited to, the internet, intranets, local area networks, mobile communication networks, and combinations thereof.
The transmission device 206 is used to receive or transmit data via a network. Specific examples of the network described above may include a wireless network provided by a communication provider of a computer terminal. In one example, the transmission device 206 includes a network adapter (Network Interface Controller, simply referred to as a NIC) that can connect to other network devices through a base station to communicate with the internet. In one example, the transmission device 206 may be a Radio Frequency (RF) module, which is used to communicate with the internet wirelessly.
Fig. 3 is a flowchart (one) of a method for determining path control information according to an embodiment of the present application, which is applied to a network multipath service NMPS, as shown in fig. 3, and includes the steps of:
step S302, inquiring available paths corresponding to path constraint information in a topology database according to the path constraint information input by a client;
Step S304, determining the path control information of the available paths in the physical network indicated by the path constraint information through a southbound interface adapter;
And step S306, the path control information is stored in a target database, wherein the target database is used for providing the path control information for the client.
The embodiment of the application queries an available path corresponding to the path constraint information in a topology database according to the path constraint information input by a client, determines path control information of the available path in a physical network indicated by the path constraint information through a southbound interface adapter, and stores the path control information in a target database, wherein the target database is used for providing the path control information for the client. That is, the network multipath service NMPS queries the available paths in the topology database according to the path constraint information input by the client, and queries the path control information corresponding to the available paths through the southbound interface adapter, so as to provide the path control information for the client, thereby solving the technical problem that the existing data center network cannot realize the discovery of the full amount of path control information and further perform absolute path control. The method and the device realize timely and full-quantity discovery of available paths of the network and realize the technical effects of determining the path forwarding of network traffic on the premise of not changing the form of the network equipment of the existing data center and the networking technology.
In an exemplary embodiment, the method comprises the steps of inquiring available paths corresponding to path constraint information in a topology database according to the path constraint information input by a client, wherein the path constraint condition in the path constraint information is determined, the path constraint condition comprises a source IP address, a destination IP address and a constraint condition, and the available paths corresponding to the path constraint information are inquired in the topology database according to the path constraint condition.
It should be noted that, in the embodiment of the present application, a new NMPS (Network Multi-PATH SERVICE ) is designed based on the Network management platform, specifically, as shown in fig. 4, NMPS is connected with a client through an interface A1, connected with a topology database through an interface A2, and connected with a southbound interface adapter through an interface A3, so as to implement path discovery and flow control of the existing data center Network.
It can be understood that the path constraint information input by the client includes at least a source IP address, a destination IP address, a protocol number, and a variable. Wherein the variables may exist in different forms:
the first form is that the variable can be a source port number, and the corresponding path constraint information is in a five-tuple form, for example, h= { source S1, destination Sn, TCP, source port number, destination port 5000}, where the transmission control protocol (Transmission Control Protocol, abbreviated as TCP).
Where h represents path constraint information, source S1 represents a source IP address, destination Sn represents a destination IP address, destination port number is fixed to 5000, source port number is a variable, and the variable range may be [3000,10000], for example, and the variable in the path constraint information in the form of five-tuple is not limited to the source port number, and may be variable, for example, destination port number.
The second type is that the variable can be an extension field, and the corresponding path constraint information is in a four-tuple form, for example { source IP, destination IP, protocol number, extension field }, wherein the extension field can be a 2-byte reserved field of the head of a virtual extension local area network (Virtual Extensible LAN, simply called VXLAN), and the variable range of the extension field can be [0,65535].
It should be noted that, the path constraint information input by the client includes, but is not limited to, four-tuple and five-tuple forms, and the path constraint condition used for determining the available path in the input path constraint information generally includes a source IP address and a destination IP address, but may also include other constraint conditions, such as disjoint paths, if needed. By determining the path constraints in the path constraint information, the topology database will feed back NMPS all available paths that meet the path constraints. The available path form of the feedback is as follows:
Path-1, { [ Leaf-1, out interface 1], [ Spine-1, out interface 1. ]. The use of the device is described herein;
........
Path-N { [ Leaf-1, out interface 1], [ Spine-10, out interface 1. }.
In one exemplary embodiment, determining, by the southbound interface adapter, path control information of the available path in the physical network indicated by the path constraint information includes determining a traversal range corresponding to a variable in the path constraint information, determining, by the southbound interface adapter, an actual forwarding path corresponding to target path control information within the traversal range, and determining, if the actual forwarding path corresponding to the target path control information is consistent with the available path, the target path control information as path control information of the available path.
In an exemplary embodiment, after determining, by the southbound interface adapter, path control information corresponding to each path in the traversal range, the method further includes continuing the traversal process until control information is found that the actual forwarding path is consistent with the available path, if the actual forwarding path corresponding to the target path control information is inconsistent with the available path.
It will be appreciated that the variables in the path constraint information, i.e. the variables mentioned in the previous embodiments, may be a source port number range or a variable field range.
After determining the available paths from the topology database, for example, if the variable is the source port number range, and if the source port range is [3000,10000] from the path constraint information, determining, by the NMPS through the southbound interface adapter, that all the actual forwarding paths of the source port number [3000,10000] are the same paths as the available paths, and taking the path control information of the paths same as the available paths as the path constraint information of the available paths, where the specific traversal scheme is to traverse from the source port number 3000 (i.e., taking the source port number 3000 as the source port number in the target path control information), and if the actual forwarding paths corresponding to the source port number 3000 are different from any one of the available paths, continuing to take the source port number 3001 as the source port number of the target path control information, and further determining whether the actual forwarding paths corresponding to the target path control information are the same as the available paths until determining the actual forwarding paths corresponding to all the available paths.
For example, there are 10 available paths queried from the topology database:
path-1 { [ Leaf-1, out interface 1], [ Spine-1, out interface 1. ], etc.,
........
Path-10 { [ Leaf-1, out interface 1], [ Spine-10, out interface 1. }. Wherein the user datagram protocol (User Datagram Protocol, abbreviated as UDP).
After traversing through the southbound interface adapter, the path control information of the actual forwarding path identical to the 10 available paths is determined as follows:
< source S1, destination Sn, UDP,3005,8000> walks Path-1 according to the current network forwarding policy;
< source S1, destination Sn, UDP,4000,8000> walks Path-2 according to the current network forwarding policy;
.......
< source S1, destination Sn, UDP,4800,8000> walks Path-10 according to the current network forwarding policy.
It should be noted that, part of the path control information of the available paths is substantially queried in the topology database, and the query of the path control information by the southbound interface adapter is substantially more focused on determining a specific variable corresponding to the available paths in the traversal range of the variable, for example, in the above embodiment, the query of 10 available paths from the topology database is based on the southbound interface adapter traversing to the specific source port number corresponding to the 10 available paths in the source port number range [3000,10000], 3005,4000.
In an exemplary embodiment, determining, by the southbound interface adapter, an actual forwarding path corresponding to the target path control information in the traversal range includes determining target path control information in the traversal range, determining, by the southbound interface adapter, outbound interface information of the target path control information, and concatenating the target path control information and the outbound interface information to obtain the actual forwarding path.
The target path control information in the traversal process is a determined five-tuple, the southbound interface adapter can call interfaces of all devices along the path to obtain output interface information of the determined five-tuple, and the determined five-tuple and the output interface information are spliced to obtain an actual forwarding path. For example, when the variable is the source port number and the traversal range is [3000,10000], traversal is started from 3000, and then, when the specific five-tuple is < source S1, destination Sn, UDP,3000,8000>, the southbound interface adapter calls the interfaces of all devices along the way of the specific five-tuple to obtain the outbound interface information corresponding to < source S1, destination Sn, UDP,3000,8000>, and the actual forwarding path is obtained after the < source S1, destination Sn, UDP,3000,8000> and the outbound interface information are spliced.
In an exemplary embodiment, before determining, by the southbound interface adapter, an actual forwarding path corresponding to the target path control information in the traversal range, the method further includes determining a path control information type corresponding to the path constraint information, where the path control information type includes, but is not limited to, one of a five-tuple and a four-tuple, the path constraint information further includes path quality description information, and the path quality description information includes at least one of a delay, a jitter, and a packet loss, and the target path control information is the same as the path control information type of the path constraint information. That is, the path constraint information input by the client may be a path control information with traversable variables, the target path control information determined in the subsequent step and the path control information corresponding to each available path are fixed path control information, and the final obtained path control information should be the same as the path control information type of the path constraint information, if the path constraint information input by the client is a five-tuple with variable variables, the path control information corresponding to the available path determined based on the path constraint information should also be a five-tuple.
It should be noted that, actually stored in the target database is the available path and a specific quantity corresponding to the available path, where the specific quantity is a quantity in a variable, for example, when the variable is the source port number [3000,10000], a source port number corresponding to the available path 1 is found 3005 (that is, the source port number 3005 is adopted, the data packet may walk the available path 1 in the physical network), and then the found available path 1+source port number 3005 is finally stored in the target database. When the available paths are multiple, the source port numbers corresponding to the available paths 1 and the available paths 1 are stored in the target database, and the source port numbers corresponding to the available paths N and the available paths N are the same
In one exemplary embodiment, the method comprises the steps of determining an updated available path according to incremental change information and the available path when the incremental change information of the topology database is received, determining updated path control information corresponding to the updated available path in the traversing range of the physical network indicated by the path constraint information through a southbound interface adapter, and updating the path control information in the target database through the updated path control information.
The embodiment of the application can update the path control information in time under the condition of changing the network topology. Specifically, when a device or a link level fault occurs in the network or the forwarding behavior is changed due to the device restarting, the network device actively reports the state update information, and a specific reporting mechanism is not limited, and a simple network management protocol TRAP (Simple Network Management Protocol Trap, abbreviated as SNMP TRAP) may be selected. The topology database updates the full network topology database and informs NMPS of the relevant incremental change information upon receiving the network topology change notification. NMPS obtaining updated available paths according to the incremental change information and the determined available paths, re-traversing the traversing range of the variable based on the updated available paths NMPS to obtain updated path control information, and updating the path control information in the target database by the updated path control information.
In an exemplary embodiment, after updating the path control information in the target database with the updated path control information, the method further includes pushing updated path control information to the client by one of determining path subscription information of the client, pushing the updated path control information to the client according to the path subscription information, and returning the updated path control information to the client upon receiving a poll message of the client.
That is, when the path control information is updated, the updated path control information may be actively pushed to the client according to the path subscription information of the client, or the client may actively poll to obtain the updated path control information.
The method of determining the path control information will be further described with reference to the following embodiments.
Aiming at the defect that the existing data center network realizes real-time and full-quantity path discovery and flow control capability, the embodiment of the application starts from network equipment and a network management and control system and designs a set of solution for solving the problems in the prior art.
The technical architecture related to the embodiment of the application is briefly described as follows:
as shown in fig. 4, the present application designs a new Network Multi-path service (Network Multi-PATH SERVICE, abbreviated as NMPS) based on the Network management platform, for implementing path discovery and flow control of the existing data center Network. The overall technology architecture includes the following components NMPS, a topology database 42, a southbound interface adapter 43, and a DB (database) database 44.
Specifically, 1) a topology database, namely a general component of a general network management and control system, is real-time network topology information obtained through BGP-LS/LLDP (border gateway protocol-link state (Border Gateway Protocol-LINK STATE, abbreviated as BGP-LS), link layer discovery protocol (LINK LAYER Discovery Protocol, abbreviated as LLDP)) and other means, and is an external component depending on NMPS in the embodiment of the application. NMPS obtaining the total path information and real-time path change update of the network by means of the topology database, and 2) a southbound interface adapter, which is a general component of a general network management and control system, is used for realizing interaction between the management and control system and a physical network, realizing southbound interfaces according to the actual condition of the network and shielding the difference of the physical network. In the technical architecture of the embodiment of the present application, NMPS obtains an actual forwarding path of a flow under a specific constraint condition through an interface A3 provided by a southbound interface adapter, and the southbound interface adapter may use an interface A4 newly exposed by a device to obtain a forwarding path under a specified constraint condition in a current state of each device. 3) NMPS network multipath service the core component in the embodiment of the application provides complete network multipath discovery and flow control functions by realizing four types of different interfaces, and the four types of interfaces are briefly described as follows:
The northbound interface A1: NMPS is an externally exposed northbound interface, receives constraint conditions for discovering and controlling the path, returns corresponding multipath results, and when the path information is changed, actively polls by a client side calling the interface or actively pushes subscription-release logic realized by the interface A1, wherein the basic realization logic is shown in figure 5.
For example, the user provides input information h= { source S1, destination Sn, TCP, source port number, destination port 5000}, where variable information Φ = source port number, variable range is [3000,10000]. Input to the NMPS through the A1 interface, the NMPS returns available network paths and information for flow control (corresponding to path control information in the above embodiment) according to the path discovery logic:
[ Path-1, { Source S1, destination Sn, TCP, source Port number 3500, destination Port 5000} ],
........
[ Path-N, { Source S1, destination Sn, TCP, source Port number 8000, destination Port 5000} ];
The path information is stored in a database, and when the operating state of the physical network link or the equipment changes, the path information triggers updating, and then the service interface A1 informs the client to execute corresponding information updating.
The service interface A2 is that the network multipath service obtains the total network paths and real-time updating under the specified constraint condition from the existing topology database through the service interface A2, the construction mode of the topology database adopts the capability of the existing management and control platform, such as BGP-LS, PCE (wherein, path calculation element (Path Computation Element, abbreviated as PCE)) and the like, the specific construction process is not in the description range of the embodiment of the application, and the basic realization logic is shown in figure 6.
For example, the path constraint is that the source S1, destination Sn, require that all paths of the path do not intersect. By invoking the A2 interface NMPS can discover all available paths:
Path-1, { [ Leaf-1, out interface 1], [ Spine-1, out interface x ] };
........
Path-N { [ Leaf-1, out interface 1], [ Spine-N, out interface y ] }.
The network multipath service obtains the accurate path forwarded in the current network under the constraint condition of the designated message from the southbound interface adapter through the service interface A3, wherein the southbound interface adapter is responsible for the encapsulation and adaptation of the interfaces of all network devices. The basic implementation logic is shown in fig. 7.
For example:
path control information { source S1, destination Sn, TCP,3000,5000};
the output of the A3 interface is { [ Leaf1, outlet interface GE0/1], [ Spine1, outlet interface GE 0/5. };
The interface identifier can be a port name or a port number, and the interface identifier is required to be stable under special conditions such as restarting of equipment.
The southbound interface A4 is a forwarding plane interface exposed by the device, when the user designates the message path control information, the designated device in the current network can return the actual forwarding path for routing the characteristic message. The basic implementation logic is shown in fig. 8.
For example:
path control information input by the interface is { source S1, destination Sn, TCP,3000,5000};
the calling result of the A4 interface of the Leaf1 node is GE0/1;
the invoking result of the A4 interface of the LeafN node is GE0/5;
The interface identifier can be a port name or a port number, and the interface identifier is required to be stable under special conditions such as restarting of equipment.
Further, the following procedure for establishing NMPS path control information is given by two specific embodiments:
Specific example 1 control with five tuples;
Assume that the client controls the path through a five tuple, where source ip=s1, destination ip=sn, the protocol uses UDP, the destination port is fixed to 8000, and the variable range of the source port number is [3000,10000]. The device in the network realizes the network multipath load balance through the five-tuple hash algorithm, and does not carry out any constraint on the specific algorithm adopted by the device.
The process of establishing NMPS information is shown in fig. 9, and is briefly described as follows:
Step S901, a client (typically, a traffic sender) invokes path constraint information given by an A1 interface, and takes < source S1, destination Sn, UDP, destination port 8000, source port number variable range [3000,10000] > as input, hopes to obtain all available paths from S1 to Sn and which determined quintuples can walk a specified path;
Step S902, NMPS calls an A2 interface to query a topology database, and obtains all available paths between a designated source S1 and a destination Sn in a physical network, wherein the available paths can add constraint conditions, such as disjoint paths and the like, according to actual requirements.
Step S903, the network topology database returns all available paths, wherein 10 paths meeting the conditions are assumed to be respectively:
path-1 { [ Leaf-1, out interface 1], [ Spine-1, out interface 1. ], etc.,
........
Path-10 { [ Leaf-1, out interface 1], [ Spine-10, out interface 1. },
Step S904: NMPS traverses in the designated range [3000,10000] of the source port number by calling the southbound adapter A3 interface until the path control information combination corresponding to all 10 available paths is found, and the following path control information needs to be found by traversing:
< source S1, destination Sn, UDP,3005,8000> walks Path-1 according to the current network forwarding policy;
< source S1, destination Sn, UDP,4000,8000> walks Path-2 according to the current network forwarding policy;
.......
< source S1, destination Sn, UDP,4800,8000> walks Path-10 according to the current network forwarding policy;
in step S905, in the traversal process of step S904, the port adapter may call the A4 interfaces of all devices along the path to obtain the output interface information of the specific five-tuple, and form an actual forwarding path after splicing, and then compare with the path in step S903:
Step S906, NMPS stores the found path control information into a database;
specific embodiment 2, adopting extension field to make path control;
Assume that the client controls the path through four tuples { source IP, destination IP, protocol number, extension field }, where source ip=s1, destination ip=sn, the protocol uses UDP, the extension field uses a2 byte reserved field of VXLAN header, and the variable space is [0,65535]. After the equipment in the network is customized through the algorithm, the hash algorithm of the four-tuple is adopted to realize the multipath load balancing of the network.
The process of establishing NMPS information is shown in fig. 10, and is briefly described as follows:
Step S1001, a client (usually a traffic sender) calls given path constraint information of an A1 interface, takes < source S1, destination Sn, UDP, VXLAN extension field [0,65535] > as input, hopes to obtain all disjoint paths of S1 to Sn and can walk a specified path through which determined quadruples;
Step S1002: NMPS calls A2 interface to query topology database, and obtains all disjoint paths between appointed source S1 and destination Sn in physical network. Where disjoint is a constraint outside of the source, destination.
Step S1003, the network topology database returns all available paths, wherein 20 paths meeting the conditions are assumed to be respectively:
path-1 { [ Leaf-1, out interface 1], [ Spine-1, out interface 1. ], etc.,
........
Path-20 { [ Leaf-20, outlet interface 1], [ Spine-20, outlet interface 1. },
Step S1004: NMPS traverses in the appointed range [0,65535] of the extension field by calling the southbound adapter A3 interface until the path control information combination corresponding to all 20 available paths is found, and the following path control information needs to be found by traversing:
< source S1, destination Sn, UDP, extension field value 100> walk Path-1 according to the current network forwarding strategy;
< source S1, destination Sn, UDP, extension field value 400> walk Path-2 according to the current network forwarding strategy;
.......
< source S1, destination Sn, UDP, extension field value 12000> walk Path-20 according to the current network forwarding strategy;
In the traversal of step S1004, the port adapter calls the A4 interfaces of all devices along the path to obtain the output interface information of the specific quadruple, and after splicing, an actual forwarding path is formed, and then the path is compared with the path in step S1003:
step S1006, NMPS, storing the found path control information into a database;
further, a path control information update procedure of NMPS is given below by specific embodiment 3.
Specific example 3:
The update process of the network multipath information is shown in fig. 11, and is briefly described as follows:
Step 1101, equipment or link level faults occur in the network, or the forwarding behavior changes due to equipment restarting;
step 1102, reporting state update information by the network device, possibly adopting SNMP TRAP or other active reporting mechanisms, wherein the specific method is not restricted, and the capability of the existing management and control platform is reused;
and step S1103, the topology database receives the notification of the change of the network topology and updates the whole network topology database.
Step S1104, the topology data informs NMPS the related increment change information through an A2 interface according to the topology change condition;
step S1105: NMPS calls A3 interface to traverse variable parameter again until finding out path control information combination corresponding to all updated available paths;
step 1106, NMPS calls an A4 interface to write the updated path control information into a database;
step S1107, after the path control information is updated, the path control information is pushed to the client by NMPS according to the previous subscription information, or the client actively polls to obtain updated information.
By the method, the real-time performance and the accuracy of the path control information can be ensured.
Specifically, the embodiment of the application avoids the additional expenditure caused by complex path discovery and path control information establishment at the end side and does not need to pay the cost of introducing new network equipment form by introducing NMPS network multipath service in the management and control platform, solves the defects caused by incomplete path discovery coverage rate and untimely updating when the network topology changes in the traditional method through reasonable interaction of NMPS with the existing topology database, and has important significance for the actual production system;
The embodiment also provides a device for determining path control information, which is used for implementing the foregoing embodiments and preferred embodiments, and is not described in detail. As used below, the term "module" may be a combination of software and/or hardware that implements a predetermined function. While the devices described in the following embodiments are preferably implemented in software, implementations in hardware, or a combination of software and hardware, are also possible and contemplated.
Fig. 12 is a block diagram of a path control information determination system according to an embodiment of the present application. As shown in fig. 12, the path control information determination system includes:
The network multipath service NMPS is used for inquiring available paths corresponding to the path constraint information in the topology database according to the path constraint information input by the client;
A southbound interface adapter 1204 for determining path control information of the available paths in the physical network indicated by the path constraint information;
the southbound interface adapter is also used to encapsulate interfaces from the device forwarding plane.
A target database 1206 for storing the path control information and providing the path control information to the client.
The embodiment of the application queries an available path corresponding to the path constraint information in a topology database according to the path constraint information input by a client, determines path control information of the available path in a physical network indicated by the path constraint information through a southbound interface adapter, and stores the path control information in a target database, wherein the target database is used for providing the path control information for the client. That is, the network multipath service NMPS queries the available paths in the topology database according to the path constraint information input by the client, and queries the path control information corresponding to the available paths through the southbound interface adapter, so as to provide the path control information for the client, thereby solving the technical problem that the existing data center network cannot realize the discovery of the full amount of path control information and further perform absolute path control. The method and the device realize timely and full-quantity discovery of available paths of the network and realize the technical effects of determining the path forwarding of network traffic on the premise of not changing the form of the network equipment of the existing data center and the networking technology.
In one exemplary embodiment, the system further includes a topology database for providing available path information to the network multipath service NMPS and, in the event of a network topology change, sending incremental change information to the network multipath service NMPS.
In an exemplary embodiment, the network multipath service NMPS is further configured to determine a path constraint condition in the path constraint information, where the path constraint condition includes a source IP address, a destination IP address, and a constraint condition, and query an available path corresponding to the path constraint information in a topology database according to the path constraint condition.
In an exemplary embodiment, the southbound interface adapter 1204 is further configured to determine a traversal range corresponding to a variable in the path constraint information, determine, by using the southbound interface adapter, an actual forwarding path corresponding to target path control information in the traversal range, and determine, when the actual forwarding path corresponding to the target path control information is consistent with the available path, the target path control information as path control information of the available path.
In an exemplary embodiment, the southbound interface adapter 1204 is further configured to, in a case where the actual forwarding path corresponding to the target path control information is inconsistent with the available path, continue the traversing process until the control information that the actual forwarding path is consistent with the available path is found.
In an exemplary embodiment, the southbound interface adapter 1204 is further configured to determine target path control information within the traversal range, determine, by using the southbound interface adapter, outbound interface information of the target path control information, and splice the target path control information and the outbound interface information to obtain the actual forwarding path.
In an exemplary embodiment, the network multipath service NMPS is further configured to determine a path control information type corresponding to the path constraint information, where the path control information type includes, but is not limited to, one of a five-tuple and a four-tuple, and the path constraint information further includes path quality description information, where the path quality description information includes at least one of a delay, a jitter, and a packet loss, and the target path control information is the same as the path control information type of the path constraint information.
In an exemplary embodiment, the network multipath service NMPS is further configured to, when incremental change information of the topology database is received, determine an update available path according to the incremental change information and the available path, determine update path control information corresponding to the update available path in a traversal range of the physical network indicated by the path constraint information through a southbound interface adapter, and update path control information in the target database through the update path control information.
In one exemplary embodiment, the network multipath service NMPS is further configured to push update path control information to the client by determining path subscription information for the client, pushing the update path control information to the client according to the subscription information, and returning the update path control information to the client if a poll message for the client is received.
From the description of the above embodiments, it will be clear to a person skilled in the art that the method according to the above embodiments may be implemented by means of software plus the necessary general hardware platform, but of course also by means of hardware, but in many cases the former is a preferred embodiment. Based on such understanding, the technical solution of the present application may be embodied essentially or in a part contributing to the prior art in the form of a software product stored in a readable storage medium (e.g. ROM/RAM, magnetic disk, optical disk) comprising instructions for causing a terminal device (which may be a mobile phone, a computer, a server, or a network device, etc.) to perform the method of the embodiments of the present application.
In an exemplary embodiment, the computer readable storage medium may include, but is not limited to, a U disk, a Read-Only Memory (ROM), a random access Memory (Random Access Memory, RAM), a removable hard disk, a magnetic disk, or an optical disk, etc. various media in which a computer program may be stored.
Specific examples in this embodiment may refer to the examples described in the foregoing embodiments and the exemplary implementation, and this embodiment is not described herein.
An embodiment of the application also provides an electronic device comprising a memory having stored therein a computer program and a processor arranged to run the computer program to perform the steps of any of the method embodiments described above.
Alternatively, in the present embodiment, the above-described processor may be configured to execute the following steps by a computer program:
s1, inquiring available paths corresponding to path constraint information in a topology database according to the path constraint information input by a client;
s2, determining path control information of the available paths in the physical network indicated by the path constraint information through a southbound interface adapter;
and S3, storing the path control information in a target database, wherein the target database is used for providing the path control information for the client.
In an exemplary embodiment, the electronic apparatus may further include a transmission device connected to the processor, and an input/output device connected to the processor.
Alternatively, in the present embodiment, the above-described electronic apparatus may be further configured to execute the above-described steps by a computer program:
s1, inquiring available paths corresponding to path constraint information in a topology database according to the path constraint information input by a client;
s2, determining path control information of the available paths in the physical network indicated by the path constraint information through a southbound interface adapter;
and S3, storing the path control information in a target database, wherein the target database is used for providing the path control information for the client.
Specific examples in this embodiment may refer to the examples described in the foregoing embodiments and the exemplary implementation, and this embodiment is not described herein.
It will be appreciated by those skilled in the art that the modules or steps of the application described above may be implemented in a general purpose computing device, they may be concentrated on a single computing device, or distributed across a network of computing devices, they may be implemented in program code executable by computing devices, so that they may be stored in a storage device for execution by computing devices, and in some cases, the steps shown or described may be performed in a different order than that shown or described herein, or they may be separately fabricated into individual integrated circuit modules, or multiple modules or steps of them may be fabricated into a single integrated circuit module. Thus, the present application is not limited to any specific combination of hardware and software.
The above description is only of the preferred embodiments of the present application and is not intended to limit the present application, but various modifications and variations can be made to the present application by those skilled in the art. Any modification, equivalent replacement, improvement, etc. made within the principle of the present application should be included in the protection scope of the present application.

Claims (12)

1. A method for determining path control information, which is applied to a network multipath service NMPS, includes:
inquiring available paths corresponding to path constraint information in a topology database according to the path constraint information input by a client;
Determining path control information of the available paths in the physical network indicated by the path constraint information through a southbound interface adapter;
And storing the path control information in a target database, wherein the target database is used for providing the path control information for the client.
2. The method for determining path control information according to claim 1, wherein querying an available path corresponding to path constraint information in a topology database according to the path constraint information input by a client comprises:
determining path constraint conditions in the path constraint information, wherein the path constraint conditions comprise a source IP address, a destination IP address and constraint conditions;
and inquiring available paths corresponding to the path constraint information in a topology database according to the path constraint conditions.
3. The method for determining path control information according to claim 1, wherein determining, by the southbound interface adapter, path control information for the available path in the physical network indicated by the path constraint information, comprises:
determining a traversing range corresponding to a variable in the path constraint information;
determining an actual forwarding path corresponding to the target path control information in the traversal range through a southbound interface adapter;
And determining the target path control information as the path control information of the available path under the condition that the actual forwarding path corresponding to the target path control information is consistent with the available path.
4. The method for determining path control information according to claim 3, wherein after determining path control information corresponding to each path in the traversal range by the southbound interface adapter, the method further comprises:
And under the condition that the actual forwarding path corresponding to the target path control information is inconsistent with the available path, continuing the traversing process until the control information of the actual forwarding path consistent with the available path is found.
5. The method for determining path control information according to claim 3, wherein determining, by the southbound interface adapter, an actual forwarding path corresponding to the target path control information within the traversal range includes:
determining target path control information in the traversing range;
determining the output interface information of the target path control information through a southbound interface adapter;
And splicing the target path control information and the output interface information to obtain the actual forwarding path.
6. The method for determining path control information according to claim 3, wherein before determining, by the southbound interface adapter, an actual forwarding path corresponding to the target path control information within the traversal range, the method further comprises:
And determining a path control information type corresponding to the path constraint information, wherein the path control information type comprises one of five-tuple and four-tuple, the path constraint information also comprises path quality description information, the path quality description information comprises at least one of time delay, jitter and packet loss, and the target path control information is the same as the path control information type of the path constraint information.
7. The method for determining path control information according to claim 1, comprising:
Determining an updated available path according to the incremental change information and the available path under the condition that the incremental change information of the topology database is received;
Determining update path control information corresponding to the update available path in the traversal range of the physical network indicated by the path constraint information through a southbound interface adapter;
And updating the path control information in the target database through the updated path control information.
8. The method for determining path control information according to claim 7, wherein after updating path control information in the target database by the updated path control information, the method further comprises:
pushing update path control information to the client by one of:
determining path subscription information of the client, and pushing the updated path control information to the client according to the path subscription information;
And returning the updated path control information to the client under the condition that the polling message of the client is received.
9. A system for determining path control information, comprising:
The network multipath service NMPS is used for inquiring available paths corresponding to the path constraint information in the topology database according to the path constraint information input by the client;
The southbound interface adapter is used for determining path control information of the available paths in the physical network indicated by the path constraint information;
And the target database is used for storing the path control information and providing the path control information for the client.
10. The system for determining path control information according to claim 9, further comprising a topology database for providing available path information to the network multipath service NMPS, and
In the event of a network topology change, incremental change information is sent to the network multipath service NMPS.
11. A computer-readable storage medium, characterized in that the storage medium has stored therein a computer program, wherein the computer program is arranged to execute the method of any of the claims 1 to 8 when run.
12. An electronic device comprising a memory and a processor, the memory having stored therein a computer program, the processor being arranged to perform the method of any of claims 1 to 8 by means of the computer program.
CN202310794561.2A 2023-06-29 2023-06-29 Method and device for determining path control information, storage medium and electronic device Pending CN119232639A (en)

Priority Applications (2)

Application Number Priority Date Filing Date Title
CN202310794561.2A CN119232639A (en) 2023-06-29 2023-06-29 Method and device for determining path control information, storage medium and electronic device
PCT/CN2024/079407 WO2025001256A1 (en) 2023-06-29 2024-02-29 Path control information determination method and apparatus, and storage medium and electronic apparatus

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202310794561.2A CN119232639A (en) 2023-06-29 2023-06-29 Method and device for determining path control information, storage medium and electronic device

Publications (1)

Publication Number Publication Date
CN119232639A true CN119232639A (en) 2024-12-31

Family

ID=93937062

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202310794561.2A Pending CN119232639A (en) 2023-06-29 2023-06-29 Method and device for determining path control information, storage medium and electronic device

Country Status (2)

Country Link
CN (1) CN119232639A (en)
WO (1) WO2025001256A1 (en)

Family Cites Families (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN103873364B (en) * 2012-12-11 2017-04-19 清华大学 Inter-domain multi-path rooting implementation method
US9742626B2 (en) * 2014-09-16 2017-08-22 CloudGenix, Inc. Methods and systems for multi-tenant controller based mapping of device identity to network level identity
CN107959577A (en) * 2016-10-15 2018-04-24 广西民族大学 Credible route computing method between domain
WO2021232190A1 (en) * 2020-05-18 2021-11-25 Alibaba Group Holding Limited Forward path planning method in massive data center networks
CN112261690B (en) * 2020-10-10 2022-04-01 北京航空航天大学 Satellite network constrained multi-path routing setting method, electronic device and storage medium
CN116155795B (en) * 2023-04-07 2025-05-06 北京达佳互联信息技术有限公司 Routing update method and device

Also Published As

Publication number Publication date
WO2025001256A1 (en) 2025-01-02

Similar Documents

Publication Publication Date Title
Cascone et al. Fast failure detection and recovery in SDN with stateful data plane
EP1811724B1 (en) Determining data link (L2) network paths
US8953432B2 (en) Softrouter dynamic binding protocol
US9769054B2 (en) Network topology discovery method and system
US6757258B1 (en) Method and apparatus for reducing OSPF flooding
Cascone et al. SPIDER: Fault resilient SDN pipeline with recovery delay guarantees
JP2022533238A (en) Methods, devices, and systems for communication between controllers in the TSN
CN110838935B (en) High-availability SDN controller clustering method, system, storage medium and equipment
EP3461080B1 (en) Topology determination method, message response method, controller and switch
CN106797319A (en) Network Service Aware Router and Its Application
WO2020244550A1 (en) Path computing method, storage medium, and electronic device
CN107645394B (en) Switch configuration method in SDN network
US7633874B1 (en) Soft notification messaging for a routing protocol
CN101466126B (en) Route control method, host equipment, router and route control system
CN116112423B (en) A method, device and equipment for determining a path
US20200403908A1 (en) Fault diagnosis method and apparatus thereof
CN114422415A (en) The egress node in segment routing processes the flow
CN100461764C (en) Method and system for realizing consistent message forwarding path
CN114531392B (en) Multicast service design method, server and storage medium
US20060176819A1 (en) Method for operating a network node of a communication network and network node of a communication network
CN119232639A (en) Method and device for determining path control information, storage medium and electronic device
CN101094168A (en) Method for discovering unit of route calculation
Cisco Network Protocols Command Reference, Part 3 Cisco IOS Release 12.0 Apollo Domain, Banyan VINES, DECnet, ISO CLNS, XNS
Cisco Configuring Novell IPX
Cisco Configuring Novell IPX

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication