CN115941423A

CN115941423A - Method for acquiring, judging and correcting data of router NetFlow

Info

Publication number: CN115941423A
Application number: CN202211463275.XA
Authority: CN
Inventors: 武迎春; 孙萍; 刘艳朋
Original assignee: Beijing Runstone Technology Inc
Current assignee: Beijing Runstone Technology Inc
Priority date: 2022-11-22
Filing date: 2022-11-22
Publication date: 2023-04-07
Anticipated expiration: 2042-11-22
Also published as: CN115941423B

Abstract

The invention discloses a method for acquiring, judging and correcting data of a router NetFlow, which comprises the following steps: step 1) data acquisition of a NetFlow and SNMP interface: acquiring a flow statistic log of the router by using a NetFlow protocol, and acquiring interface description information and flow information of the router by using an SNMP protocol to serve as reference data of a subsequent step; step 2), judging the NetFlow collection interface and direction of the router; step 3), judging the accuracy of the data statistical result and correcting errors: and judging whether the data statistical analysis result is correct or not based on the NetFlow data acquisition configuration of the router equipment, judging which error results can be corrected, and carrying out error correction processing on the error-correctable results.

Description

Method for acquiring, judging and correcting data of router NetFlow

Technical Field

The invention belongs to a method for judging and correcting the NetFlow data acquisition of a router, in particular to a method for automatically finding the NetFlow data acquisition configuration information of the router and judging and correcting the accuracy of flow summarizing statistical results.

Background

The NetFlow is used as a flow statistic technology and widely applied to flow analysis scenes such as flow direction analysis, abnormal flow detection, flow metering and charging and the like of an IP network. The basic structure of a system for performing traffic analysis based on the NetFlow technology is shown in fig. 1.

The general structure of the NetFlow flow acquisition and analysis system comprises an IP network consisting of routers and a NetFlow data acquisition and analysis system. The overall work flow of the system is as follows:

1. according to the configuration of a system administrator, a network interface of a router in an IP network can sample the flow in the inflow direction or the outflow direction;

2. the router counts the sampling flow according to the fields required in the NetFlow protocol to form a flow log, and sends the statistical result to a NetFlow data analysis and acquisition system through the network according to the NetFlow protocol;

3. and the NetFlow analysis and acquisition system analyzes the received data and completes the subsequent data analysis work.

In the above process, a network administrator needs to manually configure the network interfaces of the router, determine which network interfaces of the router have their traffic collected and counted, and determine the direction of collecting the traffic of the corresponding network interfaces, where the collecting direction includes three conditions:

sampling statistics only for network interface inflow direction flow

Sampling statistics only for network interface outflow direction flow

Sampling statistics for bidirectional flow of network interface inflow and outflow

In order to ensure the accuracy of flow analysis, the NetFlow data acquisition and analysis system must accurately know the configuration information and perform flow deduplication operation, so as to avoid errors in analysis results caused by repeated statistics of network flows.

Taking the scenario shown in fig. 2 as an example, the router in the figure has three interfaces, i.e., interface 1, interface 2, and interface 3, and forwards IP packets between the three interfaces according to the routing table query result, and each interface has incoming and outgoing IP packet traffic.

The system administrator configures the NetFlow data acquisition of each interface of the router as follows:

interface 1: sampling and counting the flow in the inflow direction;

and (3) interface 2: sampling and counting the flow in the outflow direction and the outflow direction;

and interface 3: flow sampling statistics is not carried out;

under the above NetFlow data acquisition configuration, the case that NetFlow logs are generated in traffic forwarded by the router between interfaces is as follows:

flow in from interface 1 and flow out of interface 1 for each stream of traffic: only generating a NetFlow flow statistic log at the interface 1;

flow into interface 1 and flow out of interface 2 for each of the flows: generating a NetFlow flow statistic log at the interface 1, simultaneously generating the same NetFlow flow statistic log at the interface 2, and generating two NetFlow flow statistic logs in total

Flow into interface 1 and flow out of interface 3 for each of the flows: only generating a NetFlow flow statistic log at the interface 1;

flow in from interface 2 and flow out from interface 1 for each of the flows: only generating a NetFlow flow statistic log at the interface 2;

flow into interface 2 and flow out of interface 2 for each of the flows: generating a NetFlow flow statistic log in the inflow direction of the interface 2, simultaneously generating the same NetFlow flow statistic log in the outflow direction of the interface 2, and generating two NetFlow flow statistic logs in total;

flow into interface 2 and flow out of interface 3 for each of the flows: only generating a NetFlow flow statistic log at the interface 2;

flow into interface 3 and flow out of interface 1 for each of the flows: a NetFlow flow statistic log cannot be generated;

flow into interface 3 and flow out of interface 2 for each of the flows: only generating a NetFlow flow statistic log at the interface 2;

flow into interface 3 and flow out of interface 3 for each of the flows: a NetFlow flow statistic log cannot be generated;

the NetFlow data acquisition and analysis system usually performs summary statistics on the following two basic types of flows based on the interface information carried in the NetFlow log:

statistical type 1: summarizing forwarding traffic between any two interfaces, for example summarizing and counting traffic flowing in from interface 1 and flowing out from interface 2 in fig. 2; "any two interfaces" may also try the same interface, e.g. making summary statistics of the traffic flowing in from interface i and out from interface 1 in fig. 2;

statistical type 2: summarizing and counting the flow in any interface inflow direction or outflow direction, for example summarizing and counting the flow in the interface 1 inflow direction in fig. 2, or summarizing and counting the flow in the interface 1 outflow direction;

if the NetFlow data acquisition and analysis system directly performs summary statistics on the flows of the two types based on the interface information carried in the NetFlow log, the following two types of statistical errors cannot be avoided:

type 1.1: the aggregate flow coming in from interface 1 and coming out from interface 2 is doubled repeatedly;

type 1.2: the aggregate flow into interface 2 and out of interface 2 is double calculated repeatedly;

meanwhile, the NetFlow data acquisition and analysis system cannot judge whether the following types of flow summarizing statistical results are correct or whether summarizing statistical results can be obtained;

type 2.1: the aggregate traffic flowing in from the interface 3 and flowing out from the interface 1, and the aggregate traffic flowing in from the interface 3 and flowing out from the interface 3 cannot be counted;

type 2.2: the statistical results of the flow rate summarized in the inflow direction of the interface 1, the flow rate summarized in the inflow direction of the interface 2 and the flow rate summarized in the outflow direction of the interface 2 are larger than the actual flow rate;

type 2.3: the statistical results of the flow rate summarized in the outflow direction of the interface 1 and the flow rate summarized in the inflow direction of the interface 3 are smaller than the actual flow rate;

type 2.4: the flow rate in the outlet direction of the interface 3 is consistent with the actual flow rate;

in order to judge whether the statistical analysis results of the data of the 'statistical type 1' and the 'statistical type 2' in the NetFlow flow acquisition and analysis system are correct, whether the error results can be corrected and correcting the error results, it is necessary to ensure that the information about the NetFlow data acquisition configuration is synchronized between the NetFlow flow acquisition and analysis system and router equipment. However, in an actual application environment, the number of routers in the IP network is from several to several thousand, and the number of interfaces for configuring NetFlow data acquisition is from several tens to several tens of thousands, so that the workload for manually maintaining NetFlow data acquisition configuration information of the router device is extremely large, and particularly under the condition that the network device is frequently changed, the information about NetFlow data acquisition configuration between the NetFlow data acquisition and analysis system and the router device is easily inconsistent and difficult to find, so that the reliability of the NetFlow data acquisition and analysis system on the statistical analysis results of the data of "statistical type 1" and "statistical type 2" cannot be determined, and error correction cannot be performed.

Disclosure of Invention

The invention aims to solve the technical problem of providing a method for acquiring, judging and correcting data of a router NetFlow.

The technical scheme adopted by the invention for solving the technical problems is as follows:

a method for judging and correcting data acquisition and error of a router NetFlow comprises the following steps:

step 1) acquiring NetFlow and SNMP interface data:

collecting a flow statistic log of the router by using a NetFlow protocol, and collecting interface description information and flow information of the router by using an SNMP protocol to serve as reference data of subsequent steps;

step 2), judging the interface and direction of the NetFlow acquisition of the router:

judging the NetFlow data acquisition configuration of all router devices in the IP network in real time based on the NetFlow log and the SNMP log;

step 3), judging the accuracy of the data statistical result and correcting errors:

and judging whether the data statistical analysis result is correct or not based on the NetFlow data acquisition configuration of the router equipment, judging which error results can be corrected, and carrying out error correction processing on the error-correctable results.

By using the method, the NetFlow data acquisition and analysis system directly and automatically acquires NetFlow data acquisition and configuration information of the router equipment based on the received NetFlow log, automatically keeps the consistency of the NetFlow data acquisition and configuration information between the NetFlow flow acquisition and analysis system and the router equipment under the condition of router configuration change, automatically judges the accuracy of NetFlow data analysis based on the information and automatically corrects a correctable error statistical result. The maintenance workload and the system complexity of similar systems are greatly reduced without manual maintenance of an administrator or maintenance through a system of a third party.

Additional features and advantages of the invention will be set forth in the description which follows, and in part will be obvious from the description, or may be learned by the practice of the invention. The objectives and other advantages of the invention will be realized and attained by the structure particularly pointed out in the written description and claims hereof as well as the appended drawings.

Drawings

The present invention will be described in detail below with reference to the accompanying drawings so that the above advantages of the present invention will be more apparent. Wherein,

FIG. 1 is a diagram of a general system for NetFlow flow collection and analysis;

FIG. 2 is a schematic diagram of a scene of the overall system for NetFlow traffic collection and analysis;

fig. 3 is a schematic flow chart of a method for judging and correcting data acquisition and error of a router NetFlow according to the present invention.

Detailed Description

The following detailed description of the embodiments of the present invention will be provided with reference to the drawings and examples, so that how to apply the technical means to solve the technical problems and achieve the technical effects can be fully understood and implemented. It should be noted that, as long as there is no conflict, the embodiments and the features in the embodiments of the present invention may be combined with each other, and the technical solutions formed are within the scope of the present invention.

Specifically, in order to solve the above problems, a method is proposed herein for automatically discovering the NetFlow data acquisition initial configuration of each interface of a router and automatically tracking configuration change in a NetFlow data acquisition analysis system based on a log of NetFlow version 9 or more (note that all "NetFlow" in the following description is specifically referred to as "NetFlow version 9 or more"), and using the automatically discovered NetFlow data acquisition configuration information of each interface of the router, the following automatic error correction of statistical results and the judgment of correctness of statistical results are realized. As shown in fig. 3, the method specifically includes:

step 1) acquiring NetFlow and SNMP interface data:

acquiring a flow statistic log of the router by using a NetFlow protocol, and acquiring interface description information and flow information of the router by using an SNMP protocol to serve as reference data of a subsequent step;

In one embodiment, it consists essentially of:

1. acquiring data of a NetFlow and SNMP interface: acquiring a flow statistic log of the router by using a NetFlow protocol, and acquiring interface description information and flow information of the router by using an SNMP protocol to serve as reference data of a subsequent step;

2. the router NetFlow gathers interface and direction and judges: judging the NetFlow data acquisition configuration of all router devices in the IP network in real time based on the NetFlow log and the SNMP log;

3. data statistics accuracy judgment and error correction (statistics type 1): automatically judging whether the statistical analysis result of the data of the statistical type 1 is correct or not based on the NetFlow data acquisition configuration of the router equipment, judging which error results can be corrected, and carrying out error correction processing on the error-correctable results;

4. data statistics accuracy judgment and error correction (statistics type 2): and automatically judging whether the statistical analysis result of the data of the statistical type 2 is correct or not based on the NetFlow data acquisition configuration of the router equipment, judging which error results can be corrected, and carrying out error correction processing on the error-correctable results.

The embodiment of each step is specifically realized as follows:

2.1 NetFlow and SNMP interface data acquisition

Collecting the forwarding flow of the router among the interfaces by using a NetFlow protocol to generate a NetFlow log, wherein the log content comprises but is not limited to the following fields:

IPV4_ SRC _ ADDR: this field stores the source IPv4 address of the flow;

INPUT _ SNMP: this field stores the ingress interface snmp _ ifindex of the flow;

IPV4_ DST _ ADDR: this field stores the destination IPv4 address of the flow;

OUTPUT _ SNMP: this field stores the outgoing interface snmp _ ifindex of the flow;

DIRECTION: the field stores the interface direction of the generated log, and the value is ingress or egress;

IN _ BYTES: this field stores the total number of bytes of the incoming direction flow of the interface;

IN _ PKTS: this field stores the total number of packets of the incoming direction flow of the interface;

OUT _ BYTES: this field stores the total number of bytes of the interface outgoing direction flow;

OUT _ PKTS: this field stores the total number of packets of the interface outgoing direction flow;

still taking the scenario shown in fig. 2 as an example, assume that the information of each interface of the router is as follows:

snmp _ ifindex =1 for interface 1, and the interface is described as GEl;

snmp _ ifindex =2 of the interface 2, and the interface is described as GE2;

snmp _ ifindex =3 for interface 3, interface description GE3;

assuming that forwarding traffic exists between any two interfaces of the router, the log condition of the forwarding traffic between any interfaces of the router is as follows:

1. each flow which flows in from the interface 1 and flows out from the interface 1 generates a NetFlow flow statistic log only in the inflow direction of the interface 1, and the following field assignments of the logs generated by each flow are the same:

DIRECTION＝ingress；

INPUT_SNMP＝1；

OUTPUT_SNMP＝1；

the total number of BYTES of each stream is recorded IN the IN _ BYTES field;

the total number of packets of each stream is recorded IN the IN _ PKTS field;

2. each flow flows from the interface 1 and flows from the interface 2, one NetFlow flow statistic log is generated in the interface 1 inflow direction and the interface 2 outflow direction, and two NetFlow flow statistic logs are generated in total. Wherein:

the following field assignments are given for the log generated by each flow in the direction of the interface 1 inflow:

DIRECTION＝ingress；

INPUT_SNMP＝1；

OUTPUT_SNMP＝2；

the total byte number of each flow is recorded IN an IN _ BYTES field;

the total data packet number of each stream is recorded IN the IN _ PKTS field;

the log generated by each flow in the direction of the flow out of the interface 2 has the same field assignment as follows:

DIRECTION＝egress；

INPUT_SNMP＝1；

OUTPUT_SNMP＝2；

the total byte number of each stream is recorded in the OUT _ BYTES field;

the total data packet number of each stream is recorded in an OUT _ PKTS field;

3. each flow flowing in from the interface 1 and flowing out from the interface 3 generates a NetFlow flow statistic log only in the inflow direction of the interface 1, and the following field assignments of the logs generated by each flow are the same:

DIRECTION＝ingress；

INPUT_SNMP＝1；

OUTPUT_SNMP＝3；

the total number of BYTES of each stream is recorded IN the IN _ BYTES field;

the total data packet number of each stream is recorded IN the IN _ PKTS field;

4. each flow flowing in from the interface 2 and flowing out from the interface 1 generates a NetFlow flow statistic log only in the inflow direction of the interface 2, and the following field assignments of the logs generated by each flow are the same:

DIRECTION＝ingress；

INPUT_SNMP＝2；

OUTPUT_SNMP＝1；

the total number of BYTES of each stream is recorded IN the IN _ BYTES field;

the total data packet number of each stream is recorded IN the IN _ PKTS field;

5. each flow which flows in from the interface 2 and flows out from the interface 2 generates one NetFlow flow statistic log in the inflow direction and the outflow direction of the interface 2, and two NetFlow flow statistic logs are generated in total. Wherein:

the following field assignments are given for the log generated by each flow in the direction of the interface 2 inflow:

DIRECTION＝ingress；

INPUT_SNMP＝2；

OUTPUT_SNMP＝2；

the total number of BYTES of each stream is recorded IN the IN _ BYTES field;

the total data packet number of each stream is recorded IN the IN _ PKTS field;

the log generated by each flow at the output of the interface 2 has the same field assignment as follows:

DIRECTION＝egress；

INPUT_SNMP＝2；

OUTPUT_SNMP＝2；

the total byte number of each stream is recorded in the OUT _ BYTES field;

the total data packet number of each stream is recorded in the field of OUT _ PKTS;

6. each flow flowing in from the interface 2 and flowing out from the interface 3 generates a NetFlow flow statistic log only in the inflow direction of the interface 2, and the following field assignments of the log generated by each flow are the same:

DIRECTION＝ingress；

INPUT_SNMP＝2；

OUTPUT_SNMP＝3；

the total number of BYTES of each stream is recorded IN the IN _ BYTES field;

the total data packet number of each stream is recorded IN the IN _ PKTS field;

7. flow into interface 3 and flow out of interface 1 for each of the flows: a NetFlow flow statistic log cannot be generated;

8. each flow flows in from the interface 3 and flows out from the interface 2, and only one NetFlow flow statistic log is generated in the outflow direction of the interface 2, and the following field assignments of the logs generated by each flow are the same:

DIRECTION＝egress；

INPUT_SNMP＝3；

OUTPUT_SNMP＝2；

the total byte number of each stream is recorded in the OUT _ BYTES field;

the total data packet number of each stream is recorded in an OUT _ PKTS field;

9. flow into interface 3 and flow out of interface 3 for each of the flows: a NetFlow flow statistic log cannot be generated;

acquiring interface description, interface SNMP _ ifindex, interface inflow direction traffic count and interface outflow direction traffic count of a router by using an SNMP protocol in a time period T;

taking each interface of the router shown in fig. 1 as an example, at time T0, SNMP collects data as follows:

snmp_ifindex＝1，ifDescr＝GE1，IfInOctet＝in11，IfOutOctet＝out11；

snmp_ifindex＝2，ifDescr＝GE2，IfInOctet＝in21，IfOutOctet＝out21；

snmp_ifindex＝3，ifDescr＝GE3，IfInOctet＝in31，IfOutOctet＝out31；

at time T0+ T, SNMP collected data as follows:

snmp_ifindex＝1，ifDescr＝GE1，IfInOctet＝in12，IfOutOctet＝out12；

snmp_ifindex＝2，ifDescr＝GE2，IfInOctet＝in22，IfOutOctet＝out22；

snmp_ifindex＝3，ifDescr＝GE3，IfInOctet＝in32，IfOutOctet＝out32；

assuming that the inflow direction flow and the outflow direction flow of each interface in the period T are not 0, then:

in12＞in11、in22＞in21、in32＞in31、out12＞out11、out22＞out21、out32＞out31

2.2 interface and direction are gathered to router NetFlow and are judged:

based on the data acquired by 2.1, the NetFlow data acquisition configuration information is automatically judged, that is, which interfaces and flow acquisition directions are acquired by each router are judged, and the specific judgment method comprises the following steps:

1. summarizing all NetFlow logs generated by the same router by taking the SNMP acquisition time period T in the step 2.1 as a unit, and summarizing the NetFlow logs of all flows acquired in the step 2.1 in the time period T by taking the router shown in the figure 2 as an example;

2. in all NetFlow logs collected In step 1, all logs with DIRECTION = ingress are searched, and values In INPUT _ SNMP fields In the logs are extracted and subjected to deduplication processing, so that a router interface List If _ In _ List1 with SNMP _ ifindex as an index is obtained. Taking the router shown In fig. 2 as an example, if _ In _ List1= {1,2};

3. in all NetFlow logs collected in step 1, all logs with DIRECTION = egr are searched, and values in OUTPUT _ SNMP fields in the logs are extracted and subjected to deduplication processing, so that a router interface List If _ Out _ List1 with SNMP _ ifindex as an index is obtained. Taking the router shown in fig. 2 as an example, if _ Out _ List1= {2};

4. using the SNMP data in step 1, calculating whether the traffic of each interface in and out direction is 0 in the time period T, taking the router shown in fig. 2 as an example, according to the SNMP data collected by 2.1, since in12 > in11, in22 > in21, in32 > in31, out12 > out11, out22 > out21, out32 > out31, the calculation result is as follows:

the inflow direction flow = in21-in11 > 0 of the interface 1;

the direction flow = out21-out11 > 0 when flowing out from the interface 1;

the inflow direction flow = in22-in12 > 0 of the interface 2;

the outflow direction flow = out22-out12 > 0 at the interface 2;

the inflow direction flow = in32-in31 > 0 of the interface 3;

the outgoing directional flow = out32-out31 > 0 at the interface 3;

according to the calculation result, an interface List If _ In _ List2= {1,2,3}, wherein the inflow flow rate with the snmp _ ifindex as an index is not 0, is obtained; an interface List If _ Out _ List2= {1,2,3} with an outgoing flow not 0 indexed by snmp _ ifindex;

5. and calculating the intersection of the If _ In _ List1 and the If _ In _ List2 to obtain an interface List If _ In _ List3 which takes all snmp _ ifindexes as indexes and is configured with inflow direction NetFlow data acquisition. Taking the router shown In fig. 2 as an example, if _ In _ List3= {1,2};

6. and calculating the intersection of the If _ Out _ List1 and the If _ Out _ List2 to obtain an interface List If _ Out _ List3 which takes all snmp _ ifindexes as indexes and is configured with outflow direction NetFlow data acquisition. Taking the router shown in fig. 2 as an example, if _ Out _ List3= {2};

7. and calculating the intersection of the If _ In _ List1 and the If _ In _ List2 complement to obtain all interface lists If _ Non _ In _ List which take the snmp _ ifindex as an index and are not configured with inflow direction NetFlow data acquisition. Taking the router shown In fig. 2 as an example, if _ Non _ In _ List = empty set;

8. and calculating the intersection of the If _ Out _ List1 and the If _ Out _ List2 complement to obtain all interface lists Ifnon _ Out _ List which take snmp _ ifindex as an index and are not configured with inflow direction NetFlow data acquisition. Taking the router shown in fig. 2 as an example, if _ Non _ Out _ List = empty set;

9. calculating a complementary set of a union set of If _ In _ List3 and If _ Non _ In _ List to obtain all interface lists If _ Un _ In _ List with snmp _ ifindex as an index and unknown acquisition configuration states, wherein If _ Un _ In _ List = an empty set, taking the router shown In fig. 2 as an example;

10. calculating a complement of a union set of If _ Out _ List3 and If _ Non _ Out _ List to obtain all interface lists If _ Un _ Out _ List with snmp _ ifindex as an index and unknown acquisition configuration state, where If _ Un _ Out _ List = = empty set, taking the router shown in fig. 2 as an example;

in summary, through the above calculation steps, the information of the NetFlow data acquisition configuration of the router is obtained as follows:

1. a set If _ In _ List3 of interfaces for data acquisition In the ingress direction NetFlow is configured, and taking the router shown In fig. 2 as an example, the set is {1,2};

2. a set If _ Out _ List3 of interfaces for acquiring the outgoing direction NetFlow data is configured, and taking the router shown in fig. 2 as an example, the set is {2};

3. a set If _ Non _ In _ List of interfaces not configured for data acquisition In the direction NetFlow, which is {3} taking the router shown In fig. 2 as an example;

4. a set If _ Non _ Out _ List of interfaces for which data acquisition in the direction NetFlow is not configured is {1,3} taking the router shown in fig. 2 as an example;

5. taking the router shown In fig. 2 as an example, the set of interfaces If _ Un _ In _ List whose ingress direction NetFlow data acquisition configuration state is unknown is an empty set;

6. a set If _ Un _ Out _ List of an interface with unknown data acquisition configuration state in the outgoing direction NetFlow takes the router shown in fig. 2 as an example, and the set is an empty set;

2.3 data statistics accuracy judgment and error correction (statistics type 1):

and (3) according to the NetFlow data acquisition configuration information of the router obtained in the step (2.2), carrying out accuracy judgment on the data statistical result of the statistical type 1, and judging which wrong statistical results can be subjected to error correction and error correction processing.

"type 1" of statistics refers to summarizing statistics of forwarding traffic between any two interfaces of a router, for example summarizing statistics of traffic flowing from interface 1 and flowing from interface 2 in fig. 2, if such statistics is directly performed based on values of two fields of INPUT _ SNMP and OUTPUT _ SNMP of a NetFlow log collected in step 2.1, there will be three cases of correct traffic statistics, repeated traffic statistics, and missed traffic statistics, where the repeated traffic statistics can be corrected according to NetFlow data collection configuration information of the router calculated in step 2.2.

Taking the forwarding traffic from the interface IF1 to the interface IF2 as an example, a specific method for judging the accuracy of the traffic is as follows:

determining which interface set in step 2.2 the interface IF1 and the interface IF2 belong to, and according to various combinations of different sets in step 2.2 the interface IF1 and the interface IF2 belong to, determining the method as follows:

1. IF1 belongs to IF _ In _ List3 and IF2 belongs to IF _ Out _ List3, the traffic is doubled repeatedly and error correction is possible. The error correction method is that the statistical result is directly divided by 2;

2. IF the interface IF1 belongs to the set IF _ In _ List3 and the interface IF2 belongs to the IF _ Non _ Out _ List, the statistical result is a correct statistical result;

3. IF the interface IF1 belongs to the set IF _ Non _ In _ List and the interface IF2 belongs to the set IF _ Out _ List3, the statistical result is a correct statistical result;

4. IF interface IF1 belongs to set IF _ Non _ In _ List and IF2 belongs to IF _ Non _ Out _ List, the statistical result is an error result, and since no forwarding traffic from interface IF1 to interface IF2 is collected In this case, no error correction is possible In this case.

In addition to the four cases described above, other combinations of interface IF1 and interface IF2 home sets do not normally occur.

2.4 data statistics accuracy judgment and error correction (statistics type 2):

and (3) according to the NetFlow data acquisition configuration information of the router obtained in the step (2.2), carrying out accuracy judgment on the data statistical result of the statistical type 2, and judging which wrong statistical results can be subjected to error correction and error correction.

"type 2" means to collect statistics on the flows in or out of any interface of the router, if the statistics is directly performed based on the values of the INPUT _ SNMP and OUTPUT _ SNMP fields in the NetFlow log collected in step 2.1, there will be three cases of correct flow statistics, repeated flow statistics, and missed flow statistics, where the repeated flow statistics can be performed according to the NetFlow data collection configuration information of the router calculated in step 2.2.

Taking interface IF1 as an example, the specific method for summarizing and counting the accuracy of the inflow direction flow is as follows:

1. IF the interface IF1 belongs to the set IF _ In _ List3 and all other interfaces of the same router and the interface IF1 do not belong to the set IF _ Out _ List3, the flow statistical result of the inflow direction of the interface IF1 is a correct result;

2. IF interface IF1 belongs to set IF _ In _ List3 and IF there are some interfaces belonging to set IF _ Out _ List3 for all other interfaces of the same router or interface IF1 itself, the ingress direction traffic statistics for interface IF1 may be larger than the normal results and the error may be corrected. The concrete correction steps are as follows: only using the log with the field direct = ingress and the field INPUT _ SNMP = interface IF1 in the NetFlow log to perform flow summary statistics, so as to obtain a correct summary statistical result;

3. IF the interface IF1 belongs to the set IF _ Non _ In _ List and all other interfaces of the same router and the interface IF1 belong to the set IF _ Out _ List3, the flow direction statistical result of the interface IF1 flow is a correct result;

4. IF the interface IF1 belongs to the set IF _ Non _ In _ List and some interfaces do not belong to the set IF _ Out _ List3 exist In all other interfaces of the same router and the interface IF1 itself, the inflow direction traffic statistical result of the interface IF1 may be smaller than the normal result and the error cannot be corrected;

5. IF the interface IF1 belongs to the set IF _ Un _ In _ List, the flow direction statistics result of the interface IF1 is a correct result (result is 0).

Taking interface IF1 as an example, the specific method for summarizing and counting the accuracy of the flow in the outflow direction is as follows:

1. IF the interface IF1 belongs to the set IF _ Out _ List3 and all other interfaces of the same router and the interface IF1 do not belong to the set IF _ In _ List3, the flow statistical result of the outflow direction of the interface IF1 is a correct result;

2. IF interface IF1 belongs to set IF _ Out _ List3 and IF there are some interfaces belonging to set IF _ In _ List3 for all other interfaces of the same router or interface IF1 itself, the outgoing direction traffic statistics for interface IF1 may be larger than the normal results and the error can be corrected. The concrete correction steps are as follows: only using the logs with the field DIRECTION = egr and the field OUTPUT _ SNMP = interface IF1 in the NetFlow log to carry out flow summary statistics, so as to obtain a correct summary statistical result;

3. IF the interface IF1 belongs to the set IF _ Non _ Out _ List and all other interfaces of the same router and the interface IF1 belong to the set IF _ In _ List3, the flow statistical result of the outflow direction of the interface IF1 is a correct result;

4. IF the interface IF1 belongs to the set IF _ Non _ Out _ List and some interfaces do not belong to the set IF _ In _ List3 exist In all other interfaces and the interface IF1 of the same router, the flow statistical result In the outflow direction of the interface IF1 may be smaller than the normal result and the error cannot be corrected;

5. IF the interface IF1 belongs to the set IF _ Un _ Out _ List, the flow statistics of the outgoing direction of the interface IF1 is correct (result is 0).

Finally, it should be noted that: although the present invention has been described in detail with reference to the foregoing embodiments, it will be apparent to those skilled in the art that changes may be made in the embodiments and/or equivalents thereof without departing from the spirit and scope of the invention. Any modification, equivalent replacement, or improvement made within the spirit and principle of the present invention should be included in the protection scope of the present invention.

Claims

1. A method for judging and correcting data acquisition and error of a router NetFlow is characterized by comprising the following steps:

step 1) acquiring NetFlow and SNMP interface data:

based on the NetFlow flow statistic log, judging which interfaces and flow collection directions are collected by each router according to NetFlow data collection configuration information;

judging the NetFlow data acquisition configuration of all router equipment in the IP network in real time based on the NetFlow logs and the SNMP logs;

2. The method for acquiring, judging and correcting data of the NetFlow of the router according to claim 1, wherein the step 1) specifically includes:

acquiring the forwarding flow of the router among the interfaces by using a NetFlow protocol to generate a NetFlow log, wherein the log content comprises the following fields:

IPV4_ SRC _ ADDR: this field stores the source IPv4 address of the flow;

IPV4_ DST _ ADDR: this field stores the destination IPv4 address of the flow;

OUTPUT _ SNMP: the field stores an egress interface snmp _ ifindex of the stream;

IN _ BYTES: this field stores the total number of bytes of the incoming directional flow of the interface;

OUT _ PKTS: this field stores the total number of packets of the interface outgoing direction flow.

3. The method for collecting, judging and correcting data of the NetFlow of the router according to claim 2, wherein the step 1) specifically includes:

setting the information of each interface of the router as follows:

snmp _ ifindex =1 for interface 1, interface description GE1;

snmp _ ifindex =2 for interface 2, interface description is GE2;

snmp _ ifindex =3 for interface 3, interface description GE3;

and acquiring a NetFlow flow statistic log comprising the interface description of the router, the interface SNMP _ ifindex, the interface inflow direction flow count and the interface outflow direction flow count at a time period T by using the SNMP protocol.

4. The method for acquiring, judging and correcting data of the NetFlow of the router according to claim 3, wherein in the step 1), the flow of each interface follows the following rule;

11 Each flow that flows in from interface 1 and flows out from interface 1, a NetFlow flow statistics log is generated only in the direction of interface 1 inflow, and the following field assignments are assigned to the generated log of each flow:

DIRECTION＝ingress

INPUT_SNMP＝1

OUTPUT_SNMP＝1

the total byte number of each flow is recorded IN an IN _ BYTES field;

the total data packet number of each stream is recorded IN the IN _ PKTS field;

12 Flows in from interface 1 and flows out from interface 2, one NetFlow flow statistics log is generated in the interface 1 inflow direction and the interface 2 outflow direction, respectively, and two NetFlow statistics logs are generated in total, wherein:

DIRECTION＝ingress

INPUT_SNMP＝1

OUTPUT_SNMP＝2

the total number of BYTES per stream is recorded IN the IN _ BYTES field

The total number of packets per stream is recorded IN the IN _ PKTS field

DIRECTION＝egress

INPUT_SNMP＝1

OUTPUT_SNMP＝2

the total number of BYTES per stream is recorded in the OUT _ BYTES field

The total number of packets per stream is recorded in the OUT _ PKTS field

13 Each flow flowing in from interface 1 and flowing out from interface 3, generating a NetFlow flow statistic log only in the flow-in direction of interface 1, and generating the log of each flow with the same following field assignments:

DIRECTION＝ingress

INPUT_SNMP＝1

OUTPUT_SNMP＝3

the total number of BYTES per stream is recorded IN the IN _ BYTES field

The total number of packets per stream is recorded IN the IN _ PKTS field

14 Each flow that flows in from interface 2 and flows out from interface 1, a NetFlow flow statistics log is generated only in the direction of interface 2 inflow, and the following field assignments are assigned to the generated log of each flow:

DIRECTION＝ingress

INPUT_SNMP＝2

OUTPUT_SNMP＝1

the total number of BYTES per stream is recorded IN the IN _ BYTES field

The total number of packets per stream is recorded IN the IN _ PKTS field

15 Flows in from the interface 2, and flows out from the interface 2 generate one NetFlow flow statistic log in the inflow and outflow directions of the interface 2, respectively, and two NetFlow flow statistic logs are generated in total. Wherein:

DIRECTION＝ingress

INPUT_SNMP＝2

OUTPUT_SNMP＝2

the total number of BYTES per stream is recorded IN the IN _ BYTES field

The total number of packets per stream is recorded IN the IN _ PKTS field

The log generated by each flow at the egress of interface 2 has the same value in the following fields:

DIRECTION＝egress

INPUT_SNMP＝2

OUTPUT_SNMP＝2

the total number of BYTES per stream is recorded in the OUT _ BYTES field

The total number of packets per stream is recorded in the OUT _ PKTS field

16 Each flow that flows in from interface 2 and flows out from interface 3, generates a NetFlow flow statistics log only in the direction of interface 2 inflow, and the log generated by each flow has the same following field assignments:

DIRECTION＝ingress

INPUT_SNMP＝2

OUTPUT_SNMP＝3

the total number of BYTES per stream is recorded IN the IN _ BYTES field

The total number of packets per stream is recorded IN the IN _ PKTS field

17 Flows in from interface 3 and flows out of interface 1 for each of the flows: a NetFlow flow statistic log cannot be generated;

18 Each flow that flows in from the interface 3 and flows out from the interface 2, and only one NetFlow flow statistics log is generated in the outflow direction of the interface 2, and the following field assignments of the logs generated by each flow are all the same:

DIRECTION＝egress

INPUT_SNMP＝3

OUTPUT_SNMP＝2

the total number of BYTES per stream is recorded in the OUT _ BYTES field

The total number of packets per stream is recorded in the OUT _ PKTS field

19 Flows in from the interface 3 and flows out from the interface 3 for each of the flows: netFlow traffic statistics logs are not generated.

5. The method for acquiring, judging and correcting the NetFlow data of the router according to claim 1, wherein the step 2) specifically comprises:

substep 21) summarizing all NetFlow logs generated by the same router by taking the SNMP acquisition time period T as a unit;

substep 22) searching all direct = ingress logs In all NetFlow logs collected In substep 22, extracting values In INPUT _ SNMP fields In the logs, and performing deduplication processing to obtain a router interface List If _ In _ List1 using SNMP _ ifindex as an index;

substep 23) searching all direct = egr logs in all NetFlow logs collected in substep 21, extracting values in OUTPUT _ SNMP fields in the logs, and performing deduplication processing to obtain a router interface List If _ Out _ List1 using SNMP _ ifindex as an index;

substep 24) calculating whether the flow rate of each interface in and out direction is 0 in the time period T by using the snmp data in substep 21;

according to the calculation result, an interface List If _ In _ List2= {1,2,3} with the snmp _ ifindex as an index and the inflow traffic not being 0 is obtained; an interface List If _ Out _ List2= {1,2,3} with an outgoing flow not 0 indexed by snmp _ ifindex;

substep 25) calculating the intersection of If _ In _ List1 and If _ In _ List2 to obtain an interface List If _ In _ List3 with inflow direction NetFlow data acquisition configured by all snmp _ ifindexes as indexes;

substep 26) calculating the intersection of If _ Out _ List1 and If _ Out _ List2 to obtain an interface List If _ Out _ List3 which takes all snmp _ ifindexes as indexes and is configured with outflow direction NetFlow data acquisition;

substep 27) calculating the intersection of If _ In _ List1 and If _ In _ List2 complement to obtain all interface lists If _ Non _ In _ List not configured with inflow direction NetFlow data acquisition by using snmp _ ifindex as an index;

substep 28) calculating the intersection of If _ Out _ List1 and If _ Out _ List2 complement to obtain all interface lists If _ Non _ Out _ List which take snmp _ ifindex as index and are not configured with inflow direction NetFlow data acquisition;

substep 29) calculating a complement of a union of If _ In _ List3 and If _ Non _ In _ List to obtain all interface lists If _ Un _ In _ List with snmp _ ifindex as an index and unknown acquisition configuration state;

substep 210) calculating a complement of a union of If _ Out _ List3 and If _ Non _ Out _ List to obtain all interface lists If _ Un _ Out _ List with snmp _ ifindex as an index and unknown acquisition configuration state;

substep 211) integrating the information for obtaining the NetFlow data acquisition configuration of the router as follows:

a set If _ In _ List3 of interfaces for acquiring data In the NetFlow direction is configured;

a set If _ Out _ List3 of interfaces for acquiring the outgoing direction NetFlow data is configured;

a set If _ Non _ In _ List of interfaces which are not configured into the NetFlow data acquisition direction;

a set If _ Non _ Out _ List of interfaces for acquiring data of the direction NetFlow is not configured;

a set If _ Un _ In _ List of interfaces with unknown incoming direction NetFlow data acquisition configuration states;

and acquiring a set If _ Un _ Out _ List of interfaces with unknown configuration states in the outgoing direction NetFlow data.

6. The method for acquiring, judging and correcting data of a NetFlow router according to claim 5, wherein in step 3), the accuracy of the statistical results of the data is judged, and it is judged which erroneous statistical results can be corrected and subjected to error correction, and the method specifically comprises:

summarizing and counting the forwarding flow between any two interfaces of the router;

and correcting the data acquisition configuration information of the NetFlow of the router with repeated flow statistics.

7. The method for acquiring, judging and correcting data of the NetFlow of the router according to claim 6, wherein the step 3) specifically includes:

judging which interface set in the step of attributing the interface IF1 and the interface IF2, and according to various combinations of different sets in attribution of the interface IF1 and the interface IF2, judging as follows:

IF1 belongs to set IF _ In _ List3 and IF2 belongs to IF _ Out _ List3, the flow is repeatedly counted twice and error correction is possible, the error correction method is that the statistical result is directly divided by 2;

IF the interface IF1 belongs to the set IF _ In _ List3 and the interface IF2 belongs to the IF _ Non _ Out _ List, the statistical result is a correct statistical result;

IF the interface IF1 belongs to the set IF _ Non _ In _ List and the interface IF2 belongs to IF _ Out _ List3, the statistical result is the correct statistical result;

IF interface IF1 belongs to set IF _ Non _ In _ List and IF2 belongs to IF _ Non _ Out _ List, the statistical result is an error result, and since no forwarding traffic from interface IF1 to interface IF2 is collected In this case, no error correction is possible In this case.

8. The method for acquiring, judging and correcting data of the NetFlow of the router according to claim 6, wherein the step 3) further comprises:

IF the interface IF1 belongs to the set IF _ In _ List3 and all other interfaces of the same router and the interface IF1 do not belong to the set IF _ Out _ List3, the flow statistical result of the inflow direction of the interface IF1 is a correct result;

IF the interface IF1 belongs to the set IF _ In _ List3 and some interfaces belong to the set IF _ Out _ List3 In all other interfaces of the same router or the interface IF1 itself, the inflow direction traffic statistic result of the interface IF1 may be larger than the normal result, and the error may be corrected;

the concrete correction steps are as follows: only using the log with the field direct = ingress and the field INPUT _ SNMP = interface IF1 in the NetFlow log to perform flow summary statistics, so as to obtain a correct summary statistical result;

IF the interface IF1 belongs to the set IF _ Non _ In _ List and all other interfaces of the same router and the interface IF1 belong to the set IF _ Out _ List3, the flow statistical result of the inflow direction of the interface IF1 is a correct result;

IF the interface IF1 belongs to the set IF _ Non _ In _ List and some interfaces do not belong to the set IF _ Out _ List3 exist In all other interfaces of the same router and the interface IF1 itself, the inflow direction traffic statistical result of the interface IF1 may be smaller than the normal result and the error cannot be corrected;

IF the interface IF1 belongs to the set IF _ Un _ In _ List, the flow direction statistic result of the interface IF1 is a correct result and the result is 0.

9. The method for acquiring, judging and correcting data of the NetFlow of the router according to claim 6, wherein the step 3) further comprises:

IF the interface IF1 belongs to the set IF _ Out _ List3 and all other interfaces of the same router and the interface IF1 do not belong to the set IF _ In _ List3, the flow statistical result of the outflow direction of the interface IF1 is a correct result;

IF the interface IF1 belongs to the set IF _ Out _ List3 and some interfaces belong to the set IF _ In _ List3 In all other interfaces of the same router or the interface IF1 itself, the flow statistical result of the outflow direction of the interface IF1 may be larger than the normal result, and the error may be corrected;

the concrete correction steps are as follows: only using the logs with the field direct = egr and the field OUTPUT _ SNMP = interface IF1 in the NetFlow log to perform flow summary statistics, so as to obtain a correct summary statistical result;

IF the interface IF1 belongs to the set IF _ Non _ Out _ List and all other interfaces of the same router and the interface IF1 belong to the set IF _ In _ List3, the flow statistical result In the outflow direction of the interface IF1 is a correct result;

IF the interface IF1 belongs to the set IF _ Non _ Out _ List and some interfaces do not belong to the set IF _ In _ List3 exist In all other interfaces of the same router and the interface IF1, the flow statistical result In the outflow direction of the interface IF1 may be smaller than the normal result and the error cannot be corrected;

IF the interface IF1 belongs to the set IF _ Un _ Out _ List, the flow statistics of the outgoing direction of the interface IF1 is correct and the result is 0.