TWI548235B - Network anomaly traffic monitoring system with normal distribution mode - Google Patents

Description

Network abnormal traffic monitoring system in normal distribution mode

The present invention relates to a network abnormal traffic monitoring system; in particular, a network abnormal traffic monitoring system for a normal distribution mode, and refers to a technically mature statistical process in a general manufacturing industry according to network traffic characteristics. The Statistical Process Control (SPC) method, combined with the general network traffic anomaly monitoring technology, is a new monitoring system developed in the field of network traffic management.

In the pre-patent integration network communication management system (public date: 2003/11/01, patent application number: 560149), the traffic information of the traditional telephone is used as the monitoring target, and the patent is transmitted by the data network. Traffic is the monitoring target. The behavioral patterns of the two messengers are essentially different in statistics, so the monitoring methods used are also significantly different.

System, device and method for network data analysis in the pre-patent case/SYSTEMS, APPARATUS, AND METHODS FOR NETWORK DATA ANALYSIS (publication date: 2012/10/01, patent publication number: 201239665), between before and after data Difference comparison method (offset score), A pure numerical comparison method without a parent number, using time-series values for comparison before and after. The present invention uses historical data in conjunction with a mobile modeling approach to establish a baseline.

It can be seen that there are still many shortcomings in the above-mentioned methods of use, which is not a good design, but needs to be improved.

In view of the shortcomings derived from the above-mentioned conventional methods, the inventor of the present invention has improved and innovated, and after years of painstaking research, he finally succeeded in researching and developing this invention.

In view of the foregoing, the present invention is different from the advantages of the prior patents in that the present invention proposes a more innovative technology, and the present invention increases the influence of changes in the environment on the traffic, and can reduce the situation of misjudgment. Non-authentic anomalies, such as the increase in traffic caused by a large number of users, also provide a manual benchmarking mechanism to eliminate, and can be used on circuits with a certain amount of traffic, due to statistical The probability distribution model is used as the background, and the threshold of judgment obtained is similar to the patent. It has an academic theoretical basis and considers the impact of environmental changes on the traffic, which can reduce the misjudgment situation, and also provides abnormal events. Manual manual adjustment mechanism.

On all networks that can provide traffic transmission, whether it is an access circuit or a collection circuit of the backbone, when the number of users using a circuit reaches a certain number, the communication behavior of the circuit will be more stable, especially The closer to the collection circuit of the backbone end. And the traffic distribution in a fixed period of time can usually meet the statistical normality or symmetry. This feature can be used to detect whether there is abnormal monitoring function of the traffic, and discover the equipment network management system. Unusual conditions that cannot be monitored.

The invention provides a network abnormal traffic monitoring system in a normal distribution mode. The invention is based on network traffic, including traffic, number of users, IP pool number and other continuity in the circuit. The data has observation and analysis of its stability and normal distribution patterns in a certain period of time. On the other hand, it is derived from the statistical process control method of the manufacturing industry, and its principles and practices are corrected and applied to the network communication of the telecommunications industry. Monitoring within management.

The technical features provided by the present invention have the following advantages when compared with other conventional technologies:

1. The invention can be used for detecting network abnormal traffic, instantly generating an alarm message, and informing the network administrator to enable immediate response measures to quickly restore network service quality and maintain network user rights.

2. The invention can make up for the deficiencies of the equipment network management, and uncover abnormal traffic that cannot be monitored by the equipment network management system, or those abnormalities hidden behind the scenes.

3. The monitoring mechanism proposed by the invention has self-correcting capability, so that the system can adjust its monitoring mode automatically or manually according to the change of the network environment, so as to reduce the problem of false alarm events.

4. In addition to the statistical theoretical basis, the present invention takes into account the practical experience of broadband network traffic management, so that it can achieve practical purposes in the traffic monitoring of broadband networks.

5. The past methods mostly use the abnormal conditions of the large-scale change (sudden or steep) of the traffic to be monitored. The threshold value is mostly determined by the experts based on experience. The invention breaks the uncertainty of the method and changes to the statistical theory. Based on this, the threshold of monitoring can be followed and the future work can be improved and improved.

100‧‧‧Information Data Module

200‧‧‧Monitoring threshold establishment and management module

300‧‧‧Alarm Event Generator

400‧‧‧ Instant Traffic Monitoring Module

500‧‧‧Baseline Change Monitoring and Management Module

600‧‧‧Database

S710, S720~S724, S730, S740, S750, S760, S770~S772, S810, S820, S830‧‧‧ process steps

S100, S101, S200, S201, S300, S301, S400, S500, S501, S503‧‧‧ normal distribution mode network abnormal traffic monitoring method flow steps

S210~S270‧‧‧ Process steps for establishing and managing thresholds

S510~S530‧‧‧Procedure change monitoring and management process steps

Please refer to the detailed description of the present invention and the accompanying drawings, and the technical contents of the present invention and its effects can be further understood; the related drawings are:

FIG. 1 is a schematic diagram of a network abnormality traffic monitoring system in a normal distribution mode according to the present invention.

FIG. 2 is a flow chart of a network abnormal traffic monitoring method according to the normal distribution mode of the present invention.

FIG. 3 is another flow chart of the network abnormal traffic monitoring method in the normal distribution mode of the present invention.

Figure 4 is a flow chart showing the establishment and management threshold of the network abnormal traffic monitoring method in the normal distribution mode of the present invention.

Figure 5 is a flow chart showing the monitoring and management of the baseline change of the network abnormal traffic monitoring method in the normal distribution mode of the present invention.

Figure 6 is an interface diagram of the network abnormality traffic monitoring system of the normal distribution mode of the present invention.

FIG. 7 is a diagram showing an embodiment of a network abnormal traffic monitoring method in a normal distribution mode according to the present invention.

The present invention will be further described in detail below with reference to the accompanying drawings and embodiments. It is understood that the specific embodiments described herein are merely illustrative of the invention and are not intended to limit the invention.

Hereinafter, the present invention will be further described with reference to the accompanying drawings:

The software authorization method proposed by the present invention, in addition to the above-mentioned serial number generation and verification method, includes a software authorization method combining the above-mentioned serial number generation and verification method, to solve the problem that the software vulnerable to being cracked is decompiled into the original code, and the verification authorization program fragment is falsified. Recompile, or directly imitate the issue of the sequencer generator.

Please refer to FIG. 1. FIG. 1 is a schematic diagram of a network abnormality traffic monitoring system in a normal distribution mode according to the present invention. As shown in FIG. 1 , it includes at least a traffic data module 100, a monitoring threshold establishment and management module 200, an alarm event generator 300, an instant traffic monitoring module 400, a baseline change monitoring and management module 500, and data. Library 600. The traffic data module 100 receives and transmits the network traffic data, and transmits the traffic data to each module and device. The monitoring threshold establishing and management module 200 receives the traffic data and collects the data. The data of the service data is set and set to a threshold value, and then the information and the threshold values are received by the instant messaging monitoring module 400, and the traffic data and the threshold values are outputted to the display device. The alarm event generator 300 receives the traffic data and compares the data with the threshold according to the threshold value. If the threshold value is exceeded, an abnormal event alarm is generated, indicating that the traffic data is abnormal, and the The alarm events are stored in the abnormal event alarm record table of the database, and the baseline change monitoring and management module 500 receives the alarm events, analyzes and determines the reference value.

Please refer to FIG. 2, which is a flow chart of the network abnormality monitoring method in the normal distribution mode of the present invention. As shown in Figure 2, a network traffic control mode is established for each type of aggregation circuit to instantly react to abnormal traffic circuits occurring on the network. Then, a criterion for establishing a criterion for the abnormality of the traffic is provided. For example, the threshold value is used for the analysis of the traffic, and the abnormal traffic is mainly excluded, and the reliability of the traffic data is improved. Next, provide an exception threshold to establish a standard procedure.

Further explaining that the steps may include:

The S710 collects traffic data through the network.

The S720 performs stability analysis on the traffic data, and the excessive outliers can determine that the circuit is in an unstable state.

An outlier analysis is performed after S730 and a control mode is established in S740.

The S750 performs a policing mode to monitor network anomaly traffic monitoring.

Further, to determine the stability analysis of S720, if the network traffic is unstable, S721 performs instability analysis and determines whether S722 can perform correction to restore network traffic operation. If so, S724 improves stability measures and returns the traffic information. Pass to S710, if not, perform S723 periodic basic control. The periodic basic control can be controlled by one week, two weeks or one month depending on the number of data generation.

In addition, the network abnormal traffic monitoring is abnormal due to external causes or special reasons. For example, the S810 construction or activity information causes the network traffic to be unstable. Then, the S820 reference status changes, and the reasons for recalculating the reference value are included. S710, or not listed as a normal operating reference value back to S750 implementation control.

When the S750 performs control to generate network abnormal traffic, it is determined whether the S760 has an abnormal value. If it is not returned to the S750 for execution control, if the S770 abnormality exclusion management operation is performed, the S771 abnormal cause analysis and the S772 improvement measure are included, and the abnormality is excluded.

Please refer to FIG. 4, which is a flow chart of the establishment and management threshold of the network abnormality monitoring method in the normal distribution mode of the present invention. As shown in FIG. 4, the main technical feature of the present invention is to establish and manage the threshold value flow. The steps may include at least the following

The S210 uses a box-and-whisker plot (Box Plot) method and arranges the traffic data. In actual operation, the clusters can be sorted according to time and then sorted within each group.

S220 performs quartile calculations to calculate Q1 (1/4 quantile), median (Median, Q2, 1⁄2 quantile), and Q3 (3/4 quantile).

S230 calculates the outlier boundary.

IQR (interquartile range) = Q3-Q2

Outlier limit UOL=Q3+IQR*2

Outlier lower limit LOL=Q1-IQR*2

S240 determines the outlier value, which is greater than the outlier upper limit UOL, or less than the outlier lower limit LOL.

S260 calculates the average and standard deviation of the outliers, for example: (2*IQR).

S270 calculates the control boundary and can perform hierarchical alarms. It can also divide the upper and lower control boundaries and control according to the circuit characteristics, which are divided into working days and holidays.

UCL = average + k * standard deviation

LCL = average - k * standard deviation

The size of K is adjusted according to the circuit characteristics. The larger the value, the smaller the probability of occurrence, and the more likely it is to be an abnormal value.

The alarm is divided into three levels, for example, k=3 is the first level, k=4 is the second level, and k=6 is the third level.

In the actual operation, please refer to FIG. 3 and FIG. 6 , and FIG. 3 is another flow chart of the network abnormal traffic monitoring method in the normal distribution mode of the present invention. Figure 6 is an interface diagram of the network abnormality traffic monitoring system of the normal distribution mode of the present invention. As shown in FIG. 3 and FIG. 6, the traffic information of the present invention mainly comes from a communication network for providing a service service, and the device and interface information in these networks are managed by the network device resource management system. The traffic data is provided by the device network management system. In the management of the traffic data, it is usually necessary to analyze the acquired traffic data through the "Service Data Analysis" function of the interface and deposit it into the information database for use by other modules. The modules that use the traffic information in this example can be the main modules such as "Gate Establishment and Management", "Alarm Event Generator", "Base Change Monitoring and Management" and "Instant Traffic Monitoring". The implementation mechanism within.

Referring to FIG. 1 and FIG. 6 , the display threshold of the monitoring threshold establishing and management module 200 can be established and managed as a threshold. The present invention mainly discloses establishing a threshold value. According to the characteristics of the long-term and short-sighted network and service services, in the present embodiment, the historical data of one month is used for description, and the collected traffic data is verified by statistical reasoning and statistical rules. Whether the data of the data in different date types and time points conforms to the normal distribution or symmetry. If the above two characteristics are available, the thresholds required for abnormal traffic judgment can be performed according to the following steps.

A1. Verify the same topological position according to different network pipeline topological positions and different date types, for example, whether it is a working day or a holiday, and different time periods, for example, data of the same hour or the same five minutes within 24 hours. Whether the data in each period of the circuit within the circuit conforms to the normal distribution or has symmetry.

A2. If you have the above two characteristics, use a box-and-whisker plot. The Box Plot method performs outlier analysis of the data within each group and removes outliers, leaving normal data. The method of box and whisker is to calculate the quartile value of a data, that is, the data value (Q1) at the 1/4th of the total number of items after sorting, the value (Q2) at 2/4, and the 3/4 Data value (Q3), where Q2 is the median (Median), and Q3-Q1 is the range of 50% of the data in the middle, called the interquartile range (IQR, Interquartile Range), within the sample data. The data is less than Q1-1.5*IQR (also available Q1-2*IQR), or greater than Q3+1.5*IQR (or Q3+1.5*IQR), the data is an outlier.

A3. Determine the conditions according to the data after the outliers, according to whether the data of different classes of circuits, date type or time period have the same distribution mode. Treat the conditions of the same distribution pattern as the same group, that is, the same group can calculate its threshold using the same distribution pattern.

A4. Calculate the data in the same group, and consider the sample average and standard deviation of the samples taken by the same parent. Take the average ±n* standard deviation as the judgment of whether it is abnormal traffic. Upper/lower boundary limit or upper/lower base line, where n needs to be at least 3. In order to show the severity of the abnormality, different n values can be selected according to the type of the circuit, and different threshold values are established. The larger the value of n, the more serious, and the lower the probability of occurrence.

As shown in FIGS. 1 and 6, the alarm event generator 300 is an implementation method for determining a mechanism. The received traffic data is compared to the established threshold. For example, if the data is less than the lower threshold or greater than the upper threshold, an alarm event record is generated, and the severity level is noted according to the difference of the threshold value, so the abnormality of the traffic data can be set for classification.

Please refer to FIG. 7 , which is a normal distribution mode of the present invention. A diagram of an embodiment of a network anomaly monitoring method. As shown in FIG. 1 , FIG. 6 and FIG. 7 , the instant messaging monitoring module 400 can be an instant messaging monitor or an instant messaging control map on the display interface. The received traffic data can be displayed in time series and displayed in the chart with the upper and lower thresholds. In this embodiment, it can be a line chart, in the line chart, because each fixed time period, this embodiment takes every hour. For example, the same parent, so each has its upper and lower thresholds, and the result is output to the display device, so that the user can immediately watch whether the signal of a circuit exceeds the threshold, and can also be used for observing the development or recovery state of the signal. So that the network administrator can grasp the timing of the strain at any time, as shown in Figure 7.

The reference change monitoring and management module 500 can be used as a reference change monitoring and management on the display interface, and the base-line can be an abnormal threshold. Since the circuit information changes with the environment of the network, the mode will change, and the threshold will change. Therefore, it is necessary to monitor whether the benchmark has changed. The system we invented uses both routine automatic changes and individual changes in specificity. Routine changes take multi-day moving averages, for example: every two weeks, the previous month's data is re-established with a threshold, so that the latest data can be used for monitoring.

The other method is the management of the reference value under special conditions, usually the reference change caused by individual circuit traffic growth or special events. Please refer to FIG. 5, which is a flow chart of the monitoring and management of the baseline change of the network abnormal traffic monitoring method in the normal distribution mode of the present invention. As shown in Figure 3, we manage in the following ways:

S510 watched the recent period of time, depending on the data collection status, such as the day, half day, or one hour of the collected traffic information, whether there is continuous or greater than the average value, if this phenomenon indicates that the data has not met The original benchmark needs to be recalculated.

S520 views the circuit information of the same group. If the number of abnormal events exceeding the threshold exceeds a certain number, if the number of consecutive occurrences exceeds 10 times, it means that the circuit communication is no longer in a stable state, and the cause needs to be ascertained and corrected. .

Has the internal structure of the S530 circuit been changed? If the number of people using the circuit increases/decreases, there are special holidays, company policy adjustments, or special seasons, such as: winter and summer vacations, if there is any change, the benchmark should be suspended, after the event or after a new stable state is formed, Re-collect the data and rebuild the monitoring threshold.

Please refer to Figures 1 and 3 for actual operation. Figure 1 is a combination of Figure 3 and the example. It is assumed that the circuit for collecting traffic in a five-minute cycle is monitored. From the data analysis of the past month (30 days), it is collected every day in the same hour. The traffic volume has a normal distribution mode, but after being verified in different time periods, the judgment has significant differences, so it can be grouped according to individual hours. After analysis, it is found that the traffic data of each group is stable, so as shown in Figure 3 The abnormality detection of the circuit is shown, and the implementation steps of each component are as follows:

1. The traffic data module 100S100 collects and analyzes the traffic data every 5 minutes, and the S101 traffic data is stored in the relevant table of the transaction database.

2. The monitoring threshold establishment and management module 200 performs the S200 establishment and management threshold, and the same hour data sample, the sample number is 12*30=360, and the three quartile values (Q1, Q2, Q3) are calculated and calculated. The outlier threshold is less than Q1-1.5*IQR or greater than Q3+1.5*IQR. After the outliers are removed, the average and standard deviation of the remaining data are calculated. According to the formula [mean ± 3 * standard deviation], [mean ± 4 * standard deviation] and [mean ± 6 * standard deviation], establish the upper and lower thresholds of the three-level monitoring, and store the threshold data table of the database. In the middle Line S201 monitors the threshold value.

3. The alarm event generator 300 performs S300 traffic flow monitoring, reads the traffic data every 5 minutes, and compares with the S201 monitoring threshold. If there is more than the threshold, the S301 generates an abnormal event record and stores it in the database. in.

4. Instant messaging monitoring module 400 paper type S400 real-time traffic monitoring to read whether the information is between the upper and lower limits. If there is a threshold, the development status can be observed as a response. reference.

5. The benchmark change monitoring and management module 500 continues to analyze the traffic data by the S500 to determine whether the S501 deviates from the current baseline situation. If there is a deviation, the current monitoring mechanism is suspended and the basic monitoring standard is changed, such as monitoring only. Whether the traffic has a steep drop/steep rise condition or below the minimum traffic or exceeds the interface bandwidth, etc., basic monitoring of S503 is performed to avoid a window period during the baseline change, causing serious anomalies that cannot be detected.

It is applied to the monitoring of the total communication of two or more similar circuits. For example, there are two circuits between the two devices that are mutually redundant or have a load balancing mechanism. The flow of each circuit at the same time point can be monitored for total. If the traffic is abnormal, it means that there must be a circuit abnormality in the circuit before and after it, which can be used by the network administrator for tracking analysis.

The present invention has been described in the telecommunications network applied to this embodiment, and can immediately detect abnormal conditions of various network services, including packet loss forward, device obstacles, and hacking attacks (DDoS). Wait, play the role of early detection and early repair, reduce losses, and maintain the quality of network services.

The detailed description of the present invention is intended to be illustrative of a preferred embodiment of the invention, and is not intended to limit the scope of the invention. The case In the scope of patents.

To sum up, this case is not only innovative in terms of technical thinking, but also has many of the above-mentioned functions that are not in the traditional methods of the past. It has fully complied with the statutory invention patent requirements of novelty and progressiveness, and applied for it according to law. Approved this invention patent application, in order to invent invention, to the sense of virtue.

S100, S101, S200, S201, S300, S301, S400, S500, S501, S503‧‧‧ normal distribution mode network abnormal traffic monitoring method flow steps

Claims (11)

A normal distribution mode network abnormality traffic monitoring system includes at least: a traffic data module, which receives and transmits network traffic information; a monitoring threshold establishment and management module, receives the traffic data and collects statistics The data of the traffic data establishes and sets a threshold value; an alarm event generator receives the traffic data and compares the threshold value according to the threshold value; if the threshold value is exceeded, an abnormal event alarm is generated; The monitoring module receives the traffic data and the threshold values, and outputs the traffic data and the threshold values to the display device; a reference change monitoring and management module, receives the alarm events, analyzes and Determining a reference value; and a database, generating an abnormal event alarm record table, and receiving the abnormal event alarms in the abnormal event alarm record table; wherein the traffic data module receives the service data for analysis and judgment and transmits to the Monitoring threshold establishment and management module, the alarm event generator, the instant traffic monitoring module and the baseline change monitoring and management module, and the monitoring The setting and management module transmits the data and threshold of the traffic data to the instant messaging monitoring module and to the alarm event generator, and the alarm event generator and the baseline change monitoring and management module and The database is connected. A network abnormal traffic monitoring method in a normal distribution mode, the steps including at least: A. collecting and transmitting network traffic data; B. receiving the traffic data and counting data of the traffic data, establishing and managing thresholds Value, monitoring threshold and real-time traffic monitoring; C. According to the threshold value for comparison, if the threshold value is exceeded, the traffic data is abnormal, an abnormality generates an alarm event, and the alarm event generator generates an alarm event and stores it in the abnormality. The event alarm record table; and D. the base change monitoring and management module receives the alarm events, analyzes and determines the reference value execution control. The step of establishing and managing the threshold value in step B further includes: B1. Using a box-and-whisker plot (Box Plot) method, and arranging the traffic data; B2. performing the quantile calculation and calculation B3. Calculate outlier boundaries; B4. Determine outliers; B5. Calculate outliers and boundaries; B6. Calculate the average and standard deviation of outliers; and B7 Calculate control boundaries and perform hierarchical alarms. For example, the network abnormality monitoring method of the normal distribution mode described in claim 2, wherein the arrangement of the traffic data in the step B1 can be arranged according to time, and the periodicity is one week and two weeks. Or one month. For example, the network abnormal traffic monitoring method of the normal distribution mode described in claim 3, wherein the threshold for establishing and managing the threshold is a statistical network traffic monitoring threshold, according to the bandwidth of each circuit. The usage amount provides the minimum threshold value, and the percentage of the instantaneous traffic volume difference between the two periods before and after is used to establish the threshold value. For example, the network abnormality monitoring method of the normal distribution mode described in claim 2, wherein the step B2 performs a periodic periodic data sample, and calculates three quartile values as Q1. Q2 and Q3, calculate the outlier threshold, which is less than Q1-1.5*IQR or greater than Q3+1.5*IQR. After removing the outliers, calculate the mean and standard deviation of the remaining data. For example, the network abnormal traffic monitoring method according to the normal distribution mode described in claim 5, wherein the calculated average value and the standard deviation are respectively ±3* standard deviation of the formula, and the mean number is ±4* standard deviation. And the average number of ± 6 * standard deviation, establish the upper and lower thresholds of the three-level monitoring, there is no threshold data table stored in the database. For example, the network abnormality monitoring method of the normal distribution mode described in claim 2, wherein the instant information monitoring is an instant traffic control map, and the acquired traffic data is in time sequence. It is displayed in the chart with the upper and lower thresholds. Since each fixed time period is the same parent, each has its upper and lower thresholds. The network abnormality monitoring method according to the normal distribution mode described in claim 2, wherein the alarm event generator is configured to receive the traffic data and compare the threshold value, if the threshold is less than the lower limit If the threshold value is greater than or equal to the upper threshold, an alarm event record is generated and its level is noted according to the size of the threshold. The network abnormality monitoring method according to the normal distribution mode described in claim 2, wherein the base-line of the baseline change monitoring and management in the step D is an abnormal threshold value, and This threshold varies with the environment in which the network is used. For example, the network abnormality monitoring method of the normal distribution mode described in claim 2, wherein the step D is the monitoring and management of the baseline change, wherein the benchmark value management step further comprises: D1. Collecting the traffic information, judging whether the analysis is continuous greater than or less than the average value, if it is responsible for recalculation; D2. According to the communication data of the same group circuit, whether the number of abnormal events exceeding the threshold exceeds a certain number, if any The signal of the circuit is no longer in a stable state, and the cause of instability is analyzed and corrected; and D3. Whether the internal use structure of the circuit is changed, if there is a special reason or an external reason, the reference value needs to be suspended. After these special reasons or external causes or new stable conditions are formed, collect data again and reconstruct the monitoring threshold. For example, the network abnormal traffic monitoring method of the normal distribution mode described in claim 2, wherein the special reason or external reason is the increase/decrease in the number of people using the circuit, special holidays, company policy adjustments or special The season is like a winter vacation.