CN102821002B - Network flow abnormal detecting method and system - Google Patents
Network flow abnormal detecting method and system Download PDFInfo
- Publication number
- CN102821002B CN102821002B CN201110154226.3A CN201110154226A CN102821002B CN 102821002 B CN102821002 B CN 102821002B CN 201110154226 A CN201110154226 A CN 201110154226A CN 102821002 B CN102821002 B CN 102821002B
- Authority
- CN
- China
- Prior art keywords
- flow
- output
- attack behavior
- network
- data
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000000034 method Methods 0.000 title claims abstract description 44
- 230000002159 abnormal effect Effects 0.000 title claims description 38
- 230000006399 behavior Effects 0.000 claims abstract description 180
- 238000001514 detection method Methods 0.000 claims abstract description 82
- 238000012544 monitoring process Methods 0.000 claims abstract description 6
- 238000013528 artificial neural network Methods 0.000 claims description 39
- 238000001914 filtration Methods 0.000 claims description 37
- 238000004458 analytical method Methods 0.000 claims description 31
- 238000007726 management method Methods 0.000 claims description 31
- 210000002569 neuron Anatomy 0.000 claims description 28
- 241000700605 Viruses Species 0.000 claims description 27
- 239000013598 vector Substances 0.000 claims description 21
- 238000007405 data analysis Methods 0.000 claims description 16
- ZXQYGBMAQZUVMI-GCMPRSNUSA-N gamma-cyhalothrin Chemical compound CC1(C)[C@@H](\C=C(/Cl)C(F)(F)F)[C@H]1C(=O)O[C@H](C#N)C1=CC=CC(OC=2C=CC=CC=2)=C1 ZXQYGBMAQZUVMI-GCMPRSNUSA-N 0.000 claims description 16
- 238000004364 calculation method Methods 0.000 claims description 11
- 230000008569 process Effects 0.000 claims description 11
- 238000012549 training Methods 0.000 claims description 11
- 238000003062 neural network model Methods 0.000 claims description 9
- 210000004205 output neuron Anatomy 0.000 claims description 9
- 238000013075 data extraction Methods 0.000 claims description 6
- 230000005284 excitation Effects 0.000 claims description 6
- 238000013519 translation Methods 0.000 claims description 5
- 230000010355 oscillation Effects 0.000 claims description 3
- 230000016571 aggressive behavior Effects 0.000 claims 3
- 239000002131 composite material Substances 0.000 claims 3
- 206010001488 Aggression Diseases 0.000 claims 1
- 239000010410 layer Substances 0.000 description 53
- 230000006870 function Effects 0.000 description 32
- 239000000284 extract Substances 0.000 description 9
- 206010000117 Abnormal behaviour Diseases 0.000 description 7
- 230000009467 reduction Effects 0.000 description 7
- 238000013500 data storage Methods 0.000 description 5
- 238000010586 diagram Methods 0.000 description 5
- 230000000694 effects Effects 0.000 description 5
- 238000012545 processing Methods 0.000 description 5
- 230000000903 blocking effect Effects 0.000 description 4
- 238000012423 maintenance Methods 0.000 description 4
- 238000012986 modification Methods 0.000 description 4
- 230000004048 modification Effects 0.000 description 4
- 230000005540 biological transmission Effects 0.000 description 3
- 230000007423 decrease Effects 0.000 description 3
- 238000005516 engineering process Methods 0.000 description 3
- 238000005259 measurement Methods 0.000 description 3
- 238000005065 mining Methods 0.000 description 3
- 230000004913 activation Effects 0.000 description 2
- 230000009286 beneficial effect Effects 0.000 description 2
- 238000011217 control strategy Methods 0.000 description 2
- 238000000605 extraction Methods 0.000 description 2
- 238000007689 inspection Methods 0.000 description 2
- 238000005070 sampling Methods 0.000 description 2
- 238000012546 transfer Methods 0.000 description 2
- 241000283086 Equidae Species 0.000 description 1
- 230000008859 change Effects 0.000 description 1
- 239000003086 colorant Substances 0.000 description 1
- 239000012792 core layer Substances 0.000 description 1
- 238000012937 correction Methods 0.000 description 1
- 238000010219 correlation analysis Methods 0.000 description 1
- 238000013523 data management Methods 0.000 description 1
- 238000007418 data mining Methods 0.000 description 1
- 230000001066 destructive effect Effects 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 230000014509 gene expression Effects 0.000 description 1
- 230000006872 improvement Effects 0.000 description 1
- 238000010801 machine learning Methods 0.000 description 1
- 230000001537 neural effect Effects 0.000 description 1
- 238000010606 normalization Methods 0.000 description 1
- 238000007781 pre-processing Methods 0.000 description 1
- 238000003672 processing method Methods 0.000 description 1
- 238000012216 screening Methods 0.000 description 1
- 230000001568 sexual effect Effects 0.000 description 1
- 238000012360 testing method Methods 0.000 description 1
Landscapes
- Data Exchanges In Wide-Area Networks (AREA)
Abstract
本发明公开了一种网络流量异常检测方法和系统,该方法包括:监控网络中的流量,提取网络流量的基本特征数据;根据提取的基本特征数据,确定选定的攻击行为的组合特征数据,其中,所述组合特征数据为基本特征数据的子集;将确定的组合特征数据输入对应的所述选定的攻击行为的流量模型,得到输出结果;所述流量模型为根据样本特征库中的选定的攻击行为的样本数据预先建立的;根据得到的输出结果,确定网络流量中是否存在选定的攻击行为。实现了流量检测多元化、识别更准确,可扩展性强。
The invention discloses a method and system for detecting network traffic anomalies. The method includes: monitoring the traffic in the network, extracting basic feature data of the network traffic; determining combined feature data of selected attack behaviors according to the extracted basic feature data, Wherein, the combined feature data is a subset of the basic feature data; input the determined combined feature data into the corresponding flow model of the selected attack behavior to obtain an output result; the flow model is based on the data in the sample feature library The sample data of the selected attack behavior is pre-established; according to the obtained output results, it is determined whether the selected attack behavior exists in the network traffic. It realizes the diversification of traffic detection, more accurate identification and strong scalability.
Description
技术领域technical field
本发明涉及网络信息安全技术领域,尤指一种适用于高速IP城域网的网络流量异常检测方法和系统。The invention relates to the technical field of network information security, in particular to a method and a system for detecting network traffic anomalies suitable for high-speed IP metropolitan area networks.
背景技术Background technique
随着Internet的发展,网络流量飞速增长,互联网已成为不可或缺的信息载体。与此同时,网络流量也经常会出现偏离正常范围的异常流量,主要是由蠕虫传播、DOS攻击、DDOS攻击、僵尸网络等恶意网络攻击行为以及网络配置失误、偶发性线路中断等引起。这些异常流量往往会导致整个网络服务质量急剧下降,使受害端主机、网络直接瘫痪。因此,如何在大规模网络环境下进行网络异常检测并及时提供预警信息,对保障网络正常运行具有重要意义。With the development of the Internet and the rapid growth of network traffic, the Internet has become an indispensable information carrier. At the same time, abnormal traffic that deviates from the normal range often occurs in network traffic, which is mainly caused by malicious network attacks such as worm propagation, DOS attacks, DDOS attacks, botnets, network configuration errors, and occasional line interruptions. These abnormal traffic often lead to a sharp decline in the service quality of the entire network, directly paralyzing the victim host and the network. Therefore, how to detect network anomalies and provide early warning information in a large-scale network environment is of great significance to ensure the normal operation of the network.
同时,随着网络带宽的不断提高,网络流量异常检测面临新的问题:一方面,网络传输速率大幅度提高,相同的网络攻击,在局域网表现非常明显,而在高速线路中可能并不容易发现,需要高准确性的网络流量异常检测模型;另一方面,网络带宽提高的同时也加快了网络攻击的速度,以网络蠕虫爆发为例,它能够在10min甚至更短的时间内感染互联网内大部分脆弱主机。这就要求异常检测系统在快速高效识别出异常流量的同时,还能够实时实施阻断策略。At the same time, with the continuous improvement of network bandwidth, network traffic anomaly detection is facing new problems: on the one hand, the network transmission rate has increased significantly, the same network attack is very obvious in the local area network, but it may not be easy to find in high-speed lines , requires a high-accuracy network traffic anomaly detection model; on the other hand, the increase in network bandwidth also speeds up the speed of network attacks. Taking the outbreak of network worms as an example, it can infect large Some vulnerable hosts. This requires the anomaly detection system to be able to implement blocking strategies in real time while quickly and efficiently identifying abnormal traffic.
因此,异常检测的关键是通过对网络流量正常行为的描述来分析和发现网络或系统中可能出现的异常行为,并向管理员提出警告,或主动作出反应。Therefore, the key to anomaly detection is to analyze and discover possible abnormal behaviors in the network or system by describing the normal behavior of network traffic, and to warn the administrator, or take the initiative to respond.
现有的异常检测方法主要有:统计异常检测法,基于域值的异常检测法,基于小波的异常检测法,基于免疫学的异常检测法,基于机器学习、数据挖掘和神经网络的异常检测法以及基于流量信息熵的异常检测法等,Existing anomaly detection methods mainly include: statistical anomaly detection, threshold-based anomaly detection, wavelet-based anomaly detection, immunology-based anomaly detection, machine learning, data mining and neural network-based anomaly detection And the anomaly detection method based on traffic information entropy, etc.,
但是这些方法主要存在以下问题:But these methods mainly have the following problems:
(1)报警意义不明确。由于上述异常检测方法只检测网络流量中的一种或几种特征向量,而且选取的特征向量没有特定的攻击含义,因而检测系统报警时只能知道网络中某些特征向量出现了异常,但是不能判断出现了什么样的攻击。(1) The meaning of the alarm is not clear. Since the above anomaly detection method only detects one or several eigenvectors in the network traffic, and the selected eigenvectors have no specific attack meaning, when the detection system alarms, it can only know that some eigenvectors in the network are abnormal, but cannot Determine what kind of attack has occurred.
(2)不能提供协同运行的共享数据。由于Internet是没有集中管理的多个管理域的互联网络,但是异常检测要求各个检测系统之间是协同运行的,因而作为协同运行的主要内容的共享数据的提供就显得非常重要。(2) Shared data for collaborative operation cannot be provided. Since the Internet is an interconnected network of multiple management domains without centralized management, anomaly detection requires that the various detection systems operate cooperatively, so the provision of shared data as the main content of collaborative operation is very important.
(3)可扩展性较差:由于现有的异常检测系统大多采用一种或几种单一的网络特征向量作为学习和判断的依据,对网络流量的异常描述较为单薄;在异常检测系统协同运行中网络特征向量选取得较少就可能会影响检测系统的可扩展性。(3) Poor scalability: Since most of the existing anomaly detection systems use one or several single network feature vectors as the basis for learning and judgment, the anomaly description of network traffic is relatively thin; The selection of less feature vectors in the medium network may affect the scalability of the detection system.
(4)检测精度、实时性、全面性和新异常行为识别能力不能满足异常检测的测试要求。(4) The detection accuracy, real-time performance, comprehensiveness and ability to identify new abnormal behaviors cannot meet the test requirements of abnormal detection.
检测精度、实时性、全面性和新异常行为识别能力是评价异常检测系统的四大关键指标。而目前的异常检测方法由于不能负荷高速网络流量的实时测量,尚不能实现实时异常检测;且一般采用分组抽样式处理,由于抽样不可避免地会丢失流量信息,导致检测精度和准确度降低,不能满足高速流量监控的精度需求;此外现有的异常检测手段单一,识别能力有限,其检测全面性和识别新异常行为的能力都比较差。Detection accuracy, real-time performance, comprehensiveness, and ability to identify new abnormal behaviors are the four key indicators for evaluating anomaly detection systems. However, the current anomaly detection method cannot realize real-time anomaly detection because it cannot load the real-time measurement of high-speed network traffic; and generally adopts the group sampling method to process, because the sampling will inevitably lose the flow information, resulting in a decrease in detection precision and accuracy. Meet the accuracy requirements of high-speed traffic monitoring; in addition, the existing anomaly detection methods are single, the recognition ability is limited, and its detection comprehensiveness and ability to identify new abnormal behaviors are relatively poor.
可见,现有流量异常检测实现方式,存在检测精度、实时性、全面性和报警意义不明确等诸多导致检测效果不佳的因素;同时,由于针对单一特征向量检测、控制策略单一,导致异常检测的可扩展性差,识别能力有限。且由于软件处理识别速率低,导致在异常检测时仅能给管理员发送报警,不能实现异常流量的阻断。It can be seen that the existing traffic anomaly detection implementation methods have many factors that lead to poor detection results, such as detection accuracy, real-time performance, comprehensiveness, and unclear alarm meanings; at the same time, due to the detection of a single eigenvector and the single control strategy, the anomaly detection The scalability is poor and the recognition ability is limited. Moreover, due to the low recognition rate of software processing, only an alarm can be sent to the administrator during abnormal detection, and abnormal traffic cannot be blocked.
发明内容Contents of the invention
本发明实施例提供一种网络流量异常检测方法和系统,用以解决现有技术中存在流量异常检测效果不佳、灵活性和可扩展性差的问题。Embodiments of the present invention provide a network traffic anomaly detection method and system to solve the problems of poor traffic anomaly detection effect, poor flexibility and scalability in the prior art.
一种网络流量异常检测方法,包括:A network traffic anomaly detection method, comprising:
监控网络中的流量,提取网络流量的基本特征数据;Monitor the traffic in the network and extract the basic characteristic data of the network traffic;
根据提取的基本特征数据,确定选定的攻击行为的组合特征数据,其中,所述组合特征数据为基本特征数据的子集;According to the extracted basic feature data, determine the combined feature data of the selected attack behavior, wherein the combined feature data is a subset of the basic feature data;
将确定的组合特征数据输入对应的所述选定的攻击行为的流量模型,得到输出结果;所述流量模型为根据样本特征库中的选定的攻击行为的样本数据预先建立的;Inputting the determined combined feature data into the corresponding traffic model of the selected attack behavior to obtain an output result; the traffic model is pre-established according to the sample data of the selected attack behavior in the sample feature library;
根据得到的输出结果,确定网络流量中是否存在选定的攻击行为。Based on the output obtained, determine whether the selected attack behavior is present in the network traffic.
一种网络流量异常检测系统,包括:流量统计过滤子系统和网管分析子系统;A network traffic anomaly detection system, comprising: a traffic statistics filtering subsystem and a network management analysis subsystem;
流量统计过滤子系统,用于监控网络中的流量,提取网络流量的基本特征数据;The traffic statistics and filtering subsystem is used to monitor the traffic in the network and extract the basic characteristic data of the network traffic;
网管分析子系统,用于根据提取的基本特征数据,确定选定的攻击行为的组合特征数据,其中,所述组合特征数据为基本特征数据的子集;将确定的组合特征数据输入对应的所述选定的攻击行为的流量模型,得到输出结果;所述流量模型为根据样本特征库中的选定的攻击行为的样本数据预先建立的;根据所述选定的攻击行为的流量模型的输出结果,确定网络流量中是否存在该选定的攻击行为。The network management analysis subsystem is used to determine the combined feature data of the selected attack behavior according to the extracted basic feature data, wherein the combined feature data is a subset of the basic feature data; input the determined combined feature data into the corresponding According to the traffic model of the selected attack behavior, the output result is obtained; the traffic model is pre-established according to the sample data of the selected attack behavior in the sample feature library; according to the output of the traffic model of the selected attack behavior As a result, it is determined whether the selected attack behavior is present in the network traffic.
本发明有益效果如下:The beneficial effects of the present invention are as follows:
本发明实施例提供的网络流量异常检测方法和系统,通过实时监控网络中的流量,提取网络流量的基本特征数据,确定选定的攻击行为的组合特征数据,将确定的组合特征数据依次输入对应的所述选定的攻击行为的流量模型,得到输出结果,从而确定网络流量中存在该选定的攻击行为。该方法针对不同的攻击行为分别建立模型,从而可以准确的检测出是哪种攻击行为,报警意义明确,检测精度高;该方法实时提取网络流量中的基本特征数据,且针对不同攻击行为有针对性的确定组合特征数据,从而可以全面的检测各种攻击行为,利于多个管理域的协作管理。该方法可以方便的扩展可检测的攻击行为,当有新的攻击行为时,可以建立其流量模型,对其进行检测,扩展方便。该方法能够获取良好的检测效果、较高的检测精度。The network traffic anomaly detection method and system provided by the embodiments of the present invention monitor the traffic in the network in real time, extract the basic feature data of the network traffic, determine the combined feature data of the selected attack behavior, and input the determined combined feature data into the corresponding The traffic model of the selected attack behavior is obtained to obtain an output result, so as to determine that the selected attack behavior exists in the network traffic. This method establishes models for different attack behaviors, so that the attack behavior can be accurately detected, the alarm meaning is clear, and the detection accuracy is high; the method extracts the basic characteristic data in the network traffic in real time, and is targeted for different attack behaviors. It can comprehensively detect various attack behaviors and facilitate the collaborative management of multiple management domains. This method can easily expand detectable attack behaviors. When there is a new attack behavior, its traffic model can be established to detect it, and the extension is convenient. This method can obtain good detection effect and high detection accuracy.
附图说明Description of drawings
图1为本发明实施例中网络流量异常检测系统的结构示意图;FIG. 1 is a schematic structural diagram of a network traffic anomaly detection system in an embodiment of the present invention;
图2为本发明实施例中网络流量异常检测方法的流程图;Fig. 2 is the flow chart of the network traffic anomaly detection method in the embodiment of the present invention;
图3为本发明实施例中建基于量子小波神经网络的流量模型的结构图;Fig. 3 is the structural diagram of the traffic model based on quantum wavelet neural network in the embodiment of the present invention;
图4为本发明实施例中网络流量异常检测系统在城域网络中的部署图;4 is a deployment diagram of a network traffic anomaly detection system in a metropolitan area network in an embodiment of the present invention;
图5为本发明实施例中网络流量异常检测系统的具体结构示意图;FIG. 5 is a schematic structural diagram of a network traffic anomaly detection system in an embodiment of the present invention;
图6为本发明实施例中网管分析子系统的具体结构示意图。FIG. 6 is a schematic structural diagram of a network management analysis subsystem in an embodiment of the present invention.
具体实施方式Detailed ways
针对现有技术中,网络流量异常检测效果不佳、灵活性和可扩展性差等若干问题,本发明实施例提供一种网络流量异常检测方法,基于实时提取的网络流量的特征数据实现流量异常检测,由于针对不同的异常攻击行为考虑了相应的多种特征数据的组合,使检测的实时性、准确性、全面性都获得提高,且检测灵活性和可扩展也比较好。Aiming at several problems in the prior art, such as poor network traffic anomaly detection effect, poor flexibility and scalability, the embodiment of the present invention provides a network traffic anomaly detection method, which implements traffic anomaly detection based on the characteristic data of network traffic extracted in real time , due to considering the combination of various characteristic data for different abnormal attack behaviors, the real-time performance, accuracy and comprehensiveness of detection are improved, and the detection flexibility and scalability are also relatively good.
本发明实施例提供的网络流量异常检测方法,通过如图1所示的网络流量异常检测系统实现。该系统包括:流量统计过滤子系统1和网管分析子系统2。The network traffic anomaly detection method provided by the embodiment of the present invention is implemented by the network traffic anomaly detection system shown in FIG. 1 . The system includes: flow statistics filtering subsystem 1 and network management analysis subsystem 2.
流量统计过滤子系统1,用于流量统计过滤子系统,用于监控网络中的流量,提取网络流量的基本特征数据。The traffic statistics filtering subsystem 1 is used for the traffic statistics filtering subsystem, which is used to monitor the traffic in the network and extract the basic characteristic data of the network traffic.
网管分析子系统2,用于根据提取的基本特征数据,确定选定的攻击行为的组合特征数据,其中,所述组合特征数据为基本特征数据的子集;将确定的组合特征数据输入对应的所述选定的攻击行为的流量模型,得到输出结果;所述流量模型为根据样本特征库中的选定的攻击行为的样本数据预先建立的;根据所述选定的攻击行为的流量模型的输出结果,确定网络流量中存在该选定的攻击行为。The network management analysis subsystem 2 is used to determine the combined feature data of the selected attack behavior according to the extracted basic feature data, wherein the combined feature data is a subset of the basic feature data; the determined combined feature data is input into the corresponding The traffic model of the selected attack behavior is obtained as an output result; the traffic model is pre-established according to the sample data of the selected attack behavior in the sample feature library; according to the traffic model of the selected attack behavior The output confirms that the selected attack behavior exists in the network traffic.
优选的,网管分析子系统2,还用于确定网络流量中存在该选定的攻击行为,根据该选定的攻击行为的属性信息,设置流量控制参数;Preferably, the network management analysis subsystem 2 is also used to determine that the selected attack behavior exists in the network traffic, and set flow control parameters according to the attribute information of the selected attack behavior;
流量统计过滤子系统1,还用于根据设置的流量控制参数对网络流量进行过滤控制。The traffic statistics filtering subsystem 1 is also used to filter and control network traffic according to the set traffic control parameters.
上述基于高速IP城域网的网络流量异常检测方法的流程如图2所示,包括如下步骤:The flow process of the above-mentioned network traffic anomaly detection method based on high-speed IP MAN is as shown in Figure 2, including the following steps:
步骤S11:监控网络中的流量,提取网络流量的基本特征数据。Step S11: monitor the traffic in the network, and extract the basic feature data of the network traffic.
实时监控网络中的流量,从网络流量中的下列至少一个信息中提取设定数量的特征信息,作为基本特征数据:流量相关信息、数据包相关信息、协议相关信息、端口相关信息、端口流量相关信息、地址相关信息、TCP标志位的相关信息。Monitor the traffic in the network in real time, and extract a set amount of feature information from at least one of the following information in the network traffic as basic feature data: traffic-related information, data packet-related information, protocol-related information, port-related information, port traffic-related Information, address-related information, and information about TCP flags.
具体的,从上述信息中提取设定数量特征数据,具体包括下列数据中的若干种:流报文数、流字节数、流开始时间、流结束时间、包长震荡频率、数据包平均间隔、平均包长、SYN包个数、协议类型、源端口、目的端口、每秒钟发送的数据包数量、源地址、目的地址。通过这些基本特征数据可以比较详细地描述了网络流量的运行状态。Specifically, the set quantity feature data is extracted from the above information, specifically including several of the following data: number of flow messages, number of flow bytes, flow start time, flow end time, packet length oscillation frequency, average interval of data packets , average packet length, number of SYN packets, protocol type, source port, destination port, number of data packets sent per second, source address, destination address. These basic feature data can describe the running status of the network traffic in more detail.
例如:统计到的基本特征数据可以记为包含n个基本特征变量的基本特征集X1,X2,……Xn。其中,n表示基本特征集中的基本特征变量的数量,优选的,n=256。For example: the statistical basic feature data can be recorded as a basic feature set X 1 , X 2 , ... X n including n basic feature variables. Wherein, n represents the number of basic feature variables in the basic feature set, preferably, n=256.
步骤S12:根据提取的基本特征数据,确定选定的攻击行为的组合特征数据,其中,组合特征数据为基本特征数据的子集。Step S12: According to the extracted basic feature data, determine combined feature data of the selected attack behavior, wherein the combined feature data is a subset of the basic feature data.
将提取的基本特征数据与选定的攻击行为进行类别互熵;根据互熵结果,确定各基本特征数据对选定的攻击行为的重要程度;根据各基本特征数据对选定的攻击行为的重要程度,从基本特征数据中确定选定的攻击行为的组合特征数据。Perform category cross-entropy between the extracted basic feature data and the selected attack behavior; determine the importance of each basic feature data to the selected attack behavior according to the cross-entropy results; according to the importance of each basic feature data to the selected attack behavior To determine the combined characteristic data of the selected attack behavior from the basic characteristic data.
即针对每种可能存在的攻击行为进行组合特征数据的选取。在进行特征数据的约简时,运用信息熵相关理论,通过计算基本特征集中的各基本特征变量X1,X2,……Xn与不同的攻击行为的互熵进行重要特征选取,根据互熵的大小确定基本特征变量Xi(i=1,2,……,n)的重要程度λi。That is, the selection of combined feature data is carried out for each possible attack behavior. When reducing feature data, use information entropy correlation theory to select important features by calculating the cross-entropy between each basic feature variable X 1 , X 2 ,...X n in the basic feature set and different attack behaviors. The size of the entropy determines the importance λ i of the basic characteristic variable X i (i=1, 2, . . . , n).
根据重要程度选取重要的基本特征变量组成组合特征集X1,X2,……Xm,其中m<n。进而得到组合特征数据 According to the degree of importance, select important basic feature variables to form a combined feature set X 1 , X 2 ,...X m , where m<n. Then get the combined feature data
运用信息熵相关理论,通过计算提取的基本特征数据与不同的攻击行为的互熵进行重要特征数据的选取,依据选取的重要特征数据建立能准确代表各攻击行为的组合特征集合,实现对统计网络流量的基本特征数据的有效约简。Using the theory of information entropy, the important feature data is selected by calculating the extracted basic feature data and the cross-entropy of different attack behaviors. Based on the selected important feature data, a combined feature set that can accurately represent each attack behavior is established to realize the statistical network. Effective reduction of basic characteristic data of traffic.
步骤S13:将确定的组合特征数据输入对应的选定的攻击行为的流量模型,得到输出结果。Step S13: Input the determined combined characteristic data into the traffic model of the corresponding selected attack behavior, and obtain the output result.
针对各种可能的攻击行为,提取组合特征数据后,将提取的组合特征数据输入对应的攻击行为的流量模型中。For various possible attack behaviors, after extracting the combined feature data, input the extracted combined feature data into the traffic model of the corresponding attack behavior.
流量模型为根据样本特征库中的选定的攻击行为的样本数据预先建立的。具体是基于量子小波神经网络对样本特征库中的攻击行为的特征数据进行学习训练之后得到的。The traffic model is pre-established according to the sample data of the selected attack behavior in the sample signature database. Specifically, it is obtained after learning and training the feature data of the attack behavior in the sample feature library based on the quantum wavelet neural network.
流量模型的建立以及使用流量模型对统计的组合特征数据进行处理得到输出结果的过程,在下面进行详细描述。The establishment of the traffic model and the process of using the traffic model to process the statistical combined characteristic data to obtain the output result will be described in detail below.
步骤S14:根据选定的攻击行为的流量模型的输出结果,确定网络流量中是否存在攻击行为和攻击行为的类型。Step S14: According to the output result of the traffic model of the selected attack behavior, determine whether there is an attack behavior and the type of the attack behavior in the network traffic.
根据得到的输出结果,确定网络流量中是否存在攻击行为并确定攻击行为的类型,具体包括:根据输出所述输出结果的攻击行为的流量模型,确定输出所述输出结果的流量模型对应的攻击行为的类型;以及根据输出的输出结果的输出值,确定该输出值对应的是存在攻击或不存在攻击,实现确定网络流量中是否存在攻击行为以及攻击行为的类型。According to the obtained output result, determine whether there is an attack behavior in the network traffic and determine the type of the attack behavior, specifically including: according to the traffic model of the attack behavior that outputs the output result, determine the attack behavior corresponding to the traffic model that outputs the output result type; and according to the output value of the output output result, it is determined whether the output value corresponds to an attack or no attack, so as to determine whether there is an attack behavior in the network traffic and the type of the attack behavior.
若输出所述输出结果的攻击行为的流量模型为DDoS攻击的流量模型,当输出结果为正常流量的输出值时,确认不存在DDoS攻击;当其输出结果为异常流量的输出值时,确定存在的攻击行为的类型为DDoS攻击;If the flow model of the attack behavior of outputting the output result is the flow model of DDoS attack, when the output result is the output value of normal flow, it is confirmed that there is no DDoS attack; when the output result is the output value of abnormal flow, it is determined that there is The type of attack behavior is a DDoS attack;
若输出所述输出结果的攻击行为的流量模型为木马病毒的流量模型,当输出结果为正常流量的输出值时,确认不存在木马病毒;当其输出结果为异常流量的输出值时,确定存在的攻击行为的类型为木马病毒;If the flow model of the attack behavior of outputting the output result is the flow model of Trojan horse virus, when the output result is the output value of normal flow, it is confirmed that there is no Trojan horse virus; when its output result is the output value of abnormal flow, it is determined that there is The type of attack behavior is a Trojan horse virus;
若输出所述输出结果的攻击行为的流量模型为恶意代码的流量模型,当输出结果为正常流量的输出值时,确认不存在恶意代码;当其输出结果为异常流量的输出值时,确定存在的攻击行为的类型为恶意代码;If the traffic model of the attack behavior of outputting the output result is the traffic model of malicious code, when the output result is the output value of normal traffic, it is confirmed that there is no malicious code; when the output result is the output value of abnormal traffic, it is determined that there is The type of attack behavior is malicious code;
若输出所述输出结果的攻击行为的流量模型为僵死病毒的流量模型,当输出结果为正常流量的输出值时,确认不存在僵死病毒;当其输出结果为异常流量的输出值时,确定存在的攻击行为的类型为僵死病毒。If the traffic model of the attack behavior of outputting the output result is the traffic model of zombie virus, when the output result is the output value of normal traffic, it is confirmed that there is no zombie virus; when its output result is the output value of abnormal traffic, it is determined that there is The type of attack behavior is a zombie virus.
即根据某个攻击行为的流量模型的输出结果,可以判断网络流量中是否存在该攻击行为。例如:某个选定的攻击行为-木马攻击的组合特征数据输入该攻击行为的流量模型后,输出结果为A,则表明不存在木马攻击,输出结果为B,则表明存在木马攻击。That is, according to the output result of the traffic model of an attack behavior, it can be judged whether the attack behavior exists in the network traffic. For example: after the combined feature data of a selected attack behavior-Trojan horse attack is input into the traffic model of the attack behavior, the output result is A, indicating that there is no Trojan horse attack, and the output result is B, indicating that there is a Trojan horse attack.
优选的,上述网络流量异常检测方法还包括:Preferably, the above-mentioned network traffic anomaly detection method also includes:
步骤S15:当确定网络流量中存在某个选定的攻击行为时,根据该选定的攻击行为的属性信息,设置流量控制参数。Step S15: When it is determined that there is a selected attack behavior in the network traffic, set flow control parameters according to the attribute information of the selected attack behavior.
一旦发现网络流量中存在某个攻击行为时,即根据从网络流量中提取的该攻击行为的组合特征数据,确定该攻击行为的流量控制参数,例如:端口、地址或其他控制参数等。Once an attack behavior is found in the network traffic, the traffic control parameters of the attack behavior, such as port, address or other control parameters, are determined according to the combined characteristic data of the attack behavior extracted from the network traffic.
优选的,在确定出网络流量中存在该选定的攻击行为时,向用户提供告警展示信息。Preferably, when it is determined that the selected attack behavior exists in the network traffic, alarm display information is provided to the user.
步骤S16:根据设置的流量控制参数对网络流量进行过滤控制。Step S16: Filtering and controlling network traffic according to the set traffic control parameters.
根据设置的端口、地址等流量控制参数,对网络流量进行过滤控制,实现拦截具有设置的流量控制参数的网络流量。从而实现对网络流量中的攻击行为的拦截和阻断过滤。According to the set flow control parameters such as ports and addresses, the network flow is filtered and controlled, and the network flow with the set flow control parameters is intercepted. In this way, the interception and blocking filtering of attack behaviors in network traffic can be realized.
上述方法中,建立流量模型的过程具体包括:In the above method, the process of establishing the traffic model specifically includes:
1)根据样本数据的期望输出和实际输出,确定基于量子小波神经网络的流量模型的模型参数权值。对模型参数权值的调整,使输入数据能对应到不同的类空间中。1) According to the expected output and actual output of the sample data, determine the model parameter weights of the traffic model based on the quantum wavelet neural network. The adjustment of the model parameter weights enables the input data to correspond to different class spaces.
根据样本数据的期望输出和实际输出,训练样本的均方误差函数,其中样本的均方误差函数为:According to the expected output and actual output of the sample data, the mean square error function of the training sample, where the mean square error function of the sample is:
其中,为第s批样本第k个输出神经元的期望输出,为第s批样本第k个输出神经元的实际输出。in, is the expected output of the kth output neuron of the sth batch of samples, is the actual output of the kth output neuron of the sth batch of samples.
假设量子小波神经网络的小波基采用二进正交小波函数,根据上述均方误差函数即可训练得到基于量子小波神经网络的流量模型的模型参数权值wij、vjk、aj、bj,在量子小波网络中,这些神经元权值wij、vjk、aj、bj均为可调参数。Assuming that the wavelet base of the quantum wavelet neural network adopts the binary orthogonal wavelet function, the model parameter weights w ij , v jk , a j , b j of the traffic model based on the quantum wavelet neural network can be obtained by training according to the above mean square error function , in the quantum wavelet network, these neuron weights w ij , v jk , a j , b j are all adjustable parameters.
具体的,可以基于样本数据基于快速牛顿(FN)算法实现均方误差函数Ek极小化,进行神经网络训练,获得wij、vjk、aj、bj的修正量。Specifically, the mean square error function E k can be minimized based on the fast Newton (FN) algorithm based on the sample data, and the neural network training can be performed to obtain the corrections of w ij , v jk , a j , and b j .
2)根据确定的基于量子小波神经网络的流量模型的模型参数权值,调整量子小波神经网络模型的量子间隔。2) Adjust the quantum interval of the quantum wavelet neural network model according to the determined model parameter weights of the traffic model based on the quantum wavelet neural network.
在每个训练周期,更新不同层间的连接权和隐层量子神经元的量子间隔,具体根据获得神经元权值wij、vjk、aj和bj,通过相应的算法对神经网络模型隐层量子小波神经元的量子间隔进行调整。量子间隔调整算法的思想是使量子小波神经网络中基于同一类流量行为样本数据的隐层神经元的输出变化最小。In each training cycle, the connection weights between different layers and the quantum interval of quantum neurons in the hidden layer are updated. Specifically, according to the obtained neuron weights w ij , v jk , a j and b j , the neural network model is updated by the corresponding algorithm Quantum Intervals of Quantum Wavelet Neurons in Hidden Layer Make adjustments. The idea of the quantum interval adjustment algorithm is to minimize the output change of the hidden layer neurons based on the same type of traffic behavior sample data in the quantum wavelet neural network.
3)根据确定的模型参数权值和调整后的量子间隔建立基于量子小波神经网络的流量模型。建立的基于量子小波神经网络的流量模型为:3) Establish a traffic model based on the quantum wavelet neural network according to the determined model parameter weights and the adjusted quantum interval. The established traffic model based on quantum wavelet neural network is:
其中:为第s批样本第k个输出神经元的期望输出;in: is the expected output of the kth output neuron of the sth batch of samples;
为第s批样本第k个输出神经元的实际输出; is the actual output of the kth output neuron of the sth batch of samples;
Xi为组合特征数据中的特征向量; Xi is the feature vector in the combined feature data;
λi表示特征向量Xi的重要程度;λ i represents the importance of feature vector Xi;
wij为输入层神经元Pi到隐含层神经元Sj的连接权;w ij is the connection weight of input layer neuron P i to hidden layer neuron S j ;
β为斜率因子;β is the slope factor;
为量子间隔; is the quantum interval;
隐含层激励函数是一个含有尺度因子aj和平移因子bj的小波函数;Hidden layer activation function is a wavelet function containing scale factor a j and translation factor b j ;
nq为量子间隔的数目;n q is the number of quantum intervals;
vjk隐含层神经元到输出层神经元间的连接权;The connection weight between v jk hidden layer neurons and output layer neurons;
通过采用量子小波神经网络对网络流量的样本数据进行训练学习,建立起基于多维度特征数据的流量模型,用于检测网络中异常流量。By using the quantum wavelet neural network to train and learn the sample data of network traffic, a traffic model based on multi-dimensional feature data is established to detect abnormal traffic in the network.
由于样本特征库支持在线远程升级,利用量子小波神经网络可以及时建立新异常行为的检测识别模型,便于实时更新异常流量检测识别模型。Since the sample feature library supports online remote upgrade, the detection and identification model of new abnormal behavior can be established in time by using the quantum wavelet neural network, which facilitates real-time update of the abnormal traffic detection and identification model.
建立的基于量子小波神经网络的流量模型的结构如图3所示。该流量模型中,输入层Lin有m个节点,分别对应组合特征集的m个向量;隐含层Lh的节点数目为u;输出层Lout有2个节点。每个节点对应一种输出结果,分别对应网络流量的正常和异常两种状态。相邻层节点全互连,而每层神经元之间无连接。The structure of the established traffic model based on quantum wavelet neural network is shown in Figure 3. In this traffic model, the input layer L in has m nodes, which correspond to the m vectors of the combined feature set; the number of nodes in the hidden layer L h is u; the output layer L out has 2 nodes. Each node corresponds to an output result, corresponding to two states of normal and abnormal network traffic. Adjacent layer nodes are fully interconnected, and there is no connection between neurons in each layer.
(1)输入层可以输入组合特征集X1,X2,……Xm,经输入层计算得到用于输入隐含层的输入层节点输出函数:(1) The input layer can input the combined feature set X 1 , X 2 ,...X m , and the output function of the input layer node used to input the hidden layer is obtained through the calculation of the input layer:
Pi=λiXi i=1,2,…,m;P i = λ i X i i = 1,2,...,m;
(2)将输入层节点输出函数输入隐含层,经隐含层计算得到用于输入输出层的隐含层节点输出函数:(2) Input the output function of the input layer node into the hidden layer, and calculate the hidden layer node output function for the input and output layer through the hidden layer:
也就是说,隐含层激励函数为这是一个含有尺度因子aj和平移因子bj的小波函数。β为斜率因子;WTP为量子小波神经元的输入激励;为量子间隔(s=1,2,…,nq);nq为量子间隔的数目。WT为包含wij的向量,为网络权向量;P为包含Pi=λiXi i=1,2,…,m的向量,为网络输入向量,从接入层获取。That is to say, the activation function of the hidden layer is This is a wavelet function with scaling factor aj and translation factor bj . β is the slope factor; W T P is the input excitation of the quantum wavelet neuron; is the quantum interval (s=1,2,...,n q ); n q is the number of quantum intervals. W T is a vector including w ij , which is a network weight vector; P is a vector including P i =λ i X i i=1,2,...,m, which is a network input vector, obtained from the access layer.
(3)隐含层节点输出函数输入输出层,经输出层计算后得到输出结果c1和c2。(3) The output function of the hidden layer node is input to the output layer, and the output results c1 and c2 are obtained after calculation by the output layer.
综合输入层、隐含层和输出层的计算,即经过下列流量模型的计算:The calculation of the comprehensive input layer, hidden layer and output layer is calculated through the following flow model:
其中,vjk为隐含层神经元Sj到输出层神经元Ck间的连接权。Among them, v jk is the connection weight between hidden layer neuron S j and output layer neuron C k .
上述模型采用量子小波神经网络,其中量子小波神经网络的隐层神经元借鉴了量子理论中的量子态叠加的思想,采用多个小波基函数的线性叠加作为激励函数,叠加的每一个小波函数有不同的量子间隔。The above model adopts the quantum wavelet neural network, in which the hidden layer neurons of the quantum wavelet neural network refer to the idea of quantum state superposition in quantum theory, and use the linear superposition of multiple wavelet basis functions as the excitation function. Each superimposed wavelet function has different quantum intervals.
步骤S13和步骤S14,将确定的组合特征数据依次输入对应的选定的攻击行为的流量模型,确定网络中是否存在选定的攻击行为的过程,即在上述的基于量子小波神经网络的流量模型中输入组合特征数据,最后的输出结果的过程。In steps S13 and S14, the determined combined feature data is sequentially input into the traffic model of the corresponding selected attack behavior, and the process of determining whether the selected attack behavior exists in the network, that is, in the above-mentioned traffic model based on the quantum wavelet neural network The process of inputting combined feature data and finally outputting the result.
将经过信息熵约简后得到的组合特征集X1,X2,……Xm作为描述选定的攻击行为的组合特征数据,输入该攻击行为的流量模型,即可确定网络流量中是否存在该攻击行为了。The combined feature set X 1 , X 2 ,...X m obtained after information entropy reduction is used as the combined feature data describing the selected attack behavior, and input into the traffic model of the attack behavior to determine whether there is It's time to attack.
量子小波神经网络在结构上是一种多层前馈神经网络,类似于BP网络。由于量子小波神经网络的实现形式多种多样,上述仅仅列举了一种实现形式,本发明的关键在于基于动态提取的针对各攻击行为的组合特征数据,确定是否存在该种攻击行为,实现攻击行为确定是可以考虑多元化的特征数据,因此本发明所建立的流量模型不限于上述基于量子小波神经网络所建立的流量模型,也可以是现有的基于小波理论的成熟的神经网络模型。Quantum wavelet neural network is a multi-layer feed-forward neural network in structure, similar to BP network. Due to the variety of implementation forms of quantum wavelet neural network, the above-mentioned only enumerates one implementation form. The key of the present invention is to determine whether there is such an attack behavior based on dynamically extracted combination feature data for each attack behavior, and to realize the attack behavior. It is determined that diversified characteristic data can be considered, so the traffic model established by the present invention is not limited to the above traffic model based on quantum wavelet neural network, and can also be an existing mature neural network model based on wavelet theory.
上述网络流量异常检测系统在城域网络中的部署情况如图4所示。其中高速IP城域网包括传输网、核心层、业务接入控制层和宽带接入网。从图4中可以看出,本发明实施例提供的网络流量异常检测系统可以部署在高速IP城域网的主干链路上,对高速的主干链路上的异常流量(木马、病毒等)进行实时识别和控制,达到对异常流量的实时告警和实时过滤的功能。例如部署在宽带接入网的接入路由器或二级以太网交换与业务接入控制层之间的链路上。The deployment of the above-mentioned network traffic anomaly detection system in the metropolitan area network is shown in Fig. 4 . Among them, the high-speed IP metropolitan area network includes transmission network, core layer, service access control layer and broadband access network. As can be seen from Fig. 4, the network traffic anomaly detection system provided by the embodiment of the present invention can be deployed on the backbone link of the high-speed IP metropolitan area network, and the abnormal traffic (trojan horse, virus, etc.) on the high-speed backbone link is detected. Real-time identification and control to achieve real-time alarm and real-time filtering of abnormal traffic. For example, it is deployed on the access router of the broadband access network or on the link between the secondary Ethernet switch and the service access control layer.
上述网络流量异常检测系统的具体结构如图5所示。该系统包括流量统计过滤子系统1和网管分析子系统2。其中:The specific structure of the above-mentioned network traffic anomaly detection system is shown in FIG. 5 . The system includes a traffic statistics filtering subsystem 1 and a network management analysis subsystem 2 . in:
流量统计过滤子系统1,具体包括:流量统计识别模块11和在线过滤模块12。优选的,流量统计识别模块11还连接有检测前流量测量模块13,在线过滤模块12之后还连接有检测后流量测量模块14。The traffic statistics filtering subsystem 1 specifically includes: a traffic statistics identification module 11 and an online filtering module 12 . Preferably, the traffic statistics identification module 11 is also connected with a pre-detection flow measurement module 13 , and after the online filtering module 12 is also connected with a post-detection flow measurement module 14 .
流量统计识别模块11,用于监控网络中的流量,提取网络流量的基本特征数据。The traffic statistics identification module 11 is used to monitor the traffic in the network and extract the basic characteristic data of the network traffic.
在线过滤模块12,用于获取网管分析子系统确定网络流量中存在选定的攻击行为时,根据该选定的攻击行为的属性信息,设置流量控制参数;以及根据获取的流量控制参数对网络流量进行过滤控制。The online filter module 12 is used to obtain the network management analysis subsystem when determining that there is a selected attack behavior in the network traffic, according to the attribute information of the selected attack behavior, set the flow control parameter; and according to the flow control parameter obtained. Perform filter control.
网管分析子系统2,具体包括:模型建立模块21、数据提取模块22、数据分析模块23和流量识别模块24。The network management analysis subsystem 2 specifically includes: a model building module 21 , a data extraction module 22 , a data analysis module 23 and a traffic identification module 24 .
模型建立模块21,用于根据样本特征库中的选定的攻击行为的样本数据预先建立选定的攻击行为的流量模型。The model building module 21 is configured to pre-establish a traffic model of the selected attack behavior according to the sample data of the selected attack behavior in the sample signature database.
优选的,上述模型建立模块21,具体用于:根据样本数据的期望输出和实际输出,确定基于量子小波神经网络的流量模型的模型参数权值;根据确定的基于量子小波神经网络的流量模型的模型参数权值,调整量子小波神经网络模型的量子间隔;根据确定的模型参数权值和调整后的量子间隔建立基于量子小波神经网络的流量模型。Preferably, the above-mentioned model building module 21 is specifically used to: determine the model parameter weights of the traffic model based on the quantum wavelet neural network according to the expected output and the actual output of the sample data; The model parameter weight value adjusts the quantum interval of the quantum wavelet neural network model; the flow model based on the quantum wavelet neural network is established according to the determined model parameter weight value and the adjusted quantum interval.
数据提取模块22,用于根据提取的基本特征数据,确定选定的攻击行为的组合特征数据,其中,所述组合特征数据为基本特征数据的子集。The data extraction module 22 is configured to determine combined feature data of the selected attack behavior according to the extracted basic feature data, wherein the combined feature data is a subset of the basic feature data.
优选的,上述数据提取模块22,具体用于:将提取的基本特征数据与选定的攻击行为进行类别互熵;根据互熵结果,确定各基本特征数据对所述选定的攻击行为的重要程度;根据各基本特征数据对所述选定的攻击行为的重要程度,从基本特征数据中确定所述选定的攻击行为的组合特征数据。Preferably, the above-mentioned data extraction module 22 is specifically used to perform category cross-entropy on the extracted basic feature data and the selected attack behavior; determine the importance of each basic feature data to the selected attack behavior according to the cross-entropy result. Degree: according to the importance of each basic feature data to the selected attack behavior, determine the combined feature data of the selected attack behavior from the basic feature data.
数据分析模块23,用于将确定的组合特征数据输入对应的所述选定的攻击行为的流量模型,得到输出结果。The data analysis module 23 is configured to input the determined combined feature data into the traffic model corresponding to the selected attack behavior, and obtain an output result.
优选的,上述数据分析模块23,具体用于:输入层输入组合特征集X1,X2,……Xm,经输入层计算得到用于输入隐含层的输入层节点输出函数;将得到的输入层节点输出函数输入隐含层,经隐含层计算得到隐含层节点输出函数;将得到的隐含层节点输出函数输入输出层,经输出层计算后得到输出结果。Preferably, the above-mentioned data analysis module 23 is specifically used for: the input layer input combination feature set X 1 , X 2 , ... X m , the input layer node output function used to input the hidden layer is calculated through the input layer; will be obtained The output function of the input layer node is input into the hidden layer, and the output function of the hidden layer node is obtained through the calculation of the hidden layer; the output function of the hidden layer node is input into the output layer, and the output result is obtained after the calculation of the output layer.
流量识别模块24,用于根据所述选定的攻击行为的流量模型的输出结果,确定网络流量中存在该选定的攻击行为。The traffic identification module 24 is configured to determine that the selected attack behavior exists in the network traffic according to the output result of the traffic model of the selected attack behavior.
优选的,上述网管分析子系统2,还包括:规则挖掘模块25,用于当确定出网络流量中存在该选定的攻击行为时,根据该选定的攻击行为的属性信息,设置流量控制参数。Preferably, the above-mentioned network management analysis subsystem 2 also includes: a rule mining module 25, which is used to set flow control parameters according to the attribute information of the selected attack behavior when it is determined that the selected attack behavior exists in the network traffic .
优选的,上述网管分析子系统2,还包括:信息输出模块,用于针对每种攻击行为,向用户展示流量识别模块24的确定结果,用户根据输出信息就可以知道存在哪种攻击行为和不存在哪种攻击行为,以及当存在选定的攻击行为时,向用户提供告警展示信息。Preferably, the above-mentioned network management analysis subsystem 2 also includes: an information output module, which is used to display the determination result of the traffic identification module 24 to the user for each attack behavior, and the user can know which kind of attack behavior exists and what kind of attack behavior there is according to the output information. Which kind of attack behavior exists, and when there is a selected attack behavior, provide alarm display information to the user.
上述流量统计过滤子系统1采用高速硬件线路转发引擎,可以对网络流量进行多业务并行识别,使该子系统的识别性能不会因业务的种类数量增多而下降,处理能力也不依赖于用户、业务和策略的复杂度。该子系统的网络流量统计识别基于网络流量的各中基本特征进行识别,实现了基于链路全局的网络流量特征统计,综合运用深度包检测(Deep Packet Inspection,DPI)、深度流检测(Deep Flow Inspection,DFI)等识别技术,针对报文逐一进行特征统计,例如:协议特征识别、流量行为分析、业务分析和统计,可以实现对网络流量多维特征数据(例如256种特征信息)的实时统计和报文分组级或流级别的智能识别。The above-mentioned traffic statistics and filtering subsystem 1 adopts a high-speed hardware line forwarding engine, which can carry out multi-service parallel identification of network traffic, so that the identification performance of this subsystem will not decrease due to the increase in the number of types of services, and the processing capacity does not depend on users, Complexity of business and strategy. The network traffic statistical identification of this subsystem is based on the identification of various basic characteristics of network traffic, and realizes the statistics of network traffic characteristics based on the overall link, and comprehensively uses Deep Packet Inspection (DPI), Deep Flow Inspection, DFI) and other identification technologies can perform feature statistics on packets one by one, such as: protocol feature identification, traffic behavior analysis, business analysis and statistics, which can realize real-time statistics and Intelligent identification at packet level or flow level.
上述流量统计过滤子系统1实现网络流量统计功能时,可以包括下列方面的功能:各类用户和业务的流量统计;各类流控策略的流量统计;灵活设定策略流的流量统计;指定IP地址或用户群组的流量统计;实时和历史的流量统计等等,在此不再一一列举。上述流量统计过滤子系统1实现网络流量业务识别功能时,可以包括下列方面的功能:能够实现对加密的、变种的和未知新出现的业务行为进行有效的识别和控制等等,此处也不再一一列举。When the above-mentioned traffic statistics filtering subsystem 1 realizes the network traffic statistics function, it may include the following functions: traffic statistics of various users and services; traffic statistics of various flow control strategies; flexible setting of traffic statistics of policy flows; specified IP Traffic statistics of addresses or user groups; real-time and historical traffic statistics, etc., will not be listed here. When the above traffic statistics and filtering subsystem 1 implements the network traffic service identification function, it may include the following functions: it can realize effective identification and control of encrypted, variant and unknown new business behaviors, etc. List them one by one.
上述流量统计过滤子系统1根据网管分析子系统2设置的流量控制参数,实现流量控制,设置的控制参数可以是端口、地址等,在对报文进行在线识别时,利用高校的字符串匹配引擎实现对网络流量中的数据内容的筛查,对于携带恶意代码的数据流进行过滤阻断,切断了木马病毒的传播途径。流量统计过滤子系统1可以根据网管分析子系统2的下发命令启动在线过滤识别。The above flow statistics and filtering subsystem 1 implements flow control according to the flow control parameters set by the network management analysis subsystem 2. The set control parameters can be port, address, etc. When identifying the message online, the string matching engine of the university is used Realize the screening of data content in network traffic, filter and block data streams carrying malicious codes, and cut off the transmission path of Trojan horse viruses. The traffic statistics and filtering subsystem 1 can start online filtering and identification according to the command issued by the network management analysis subsystem 2 .
上述网管分析子系统2,主要通过后端软件设计实现,根据流量统计过滤子系统1上报的统计数据,获取网络流量的基本特征数据,通过信息熵约简获取组合特征数据,然后基于量子小波神经网络流量模型进行数据分析,检测网络中的异常流量,并可以确定是哪种攻击行为,进而挖掘出过滤规则,设置流量控制参数,指示流量统计过滤子系统1实现在线病毒过滤功能。网管分析子系统2可以构建针对多种攻击行为的多个流量模型,在获取到基本特征数据后,依次针对多种攻击行为约简获取组合特征数据,依次分别输入相应的流量模型,从而确定是否存在相应的攻击行为。这种方法在有新的攻击行为需要分析时,可以建立流量模型进行分析即可,方便扩展应用。The above-mentioned network management analysis subsystem 2 is mainly implemented through back-end software design. According to the statistical data reported by the traffic statistics and filtering subsystem 1, the basic characteristic data of the network traffic is obtained, and the combined characteristic data is obtained through information entropy reduction, and then based on the quantum wavelet neural The network traffic model conducts data analysis, detects abnormal traffic in the network, and can determine the attack behavior, and then digs out filtering rules, sets traffic control parameters, and instructs the traffic statistics filtering subsystem 1 to realize the online virus filtering function. The network management analysis subsystem 2 can build multiple traffic models for various attack behaviors. After obtaining the basic characteristic data, it can sequentially reduce and obtain combined characteristic data for various attack behaviors, and input corresponding traffic models in turn to determine whether There are corresponding attacks. In this method, when there is a new attack behavior that needs to be analyzed, a traffic model can be established for analysis, which is convenient for expanding applications.
其中网管分析子系统的具体结构如图6所示,流量统计过滤子系统从因特网中获取数据后,传送给网管分析子系统。网管分析子系统在实际部署时,可以部署数据存储单元、数据分析单元和应用程序单元等几部分。The specific structure of the network management analysis subsystem is shown in Figure 6. After the traffic statistics and filtering subsystem obtains data from the Internet, it transmits the data to the network management analysis subsystem. When the network management analysis subsystem is actually deployed, several parts such as data storage unit, data analysis unit and application program unit can be deployed.
从因特网中获取的数据存储在数据存储单元中,数据存储单元中具体可以布设多个数据库,例如:流量统计数据库、链路层统计数据库、流量行为特征库、告警日志数据库、策略规则数据库和其他统计数据的数据库。分别用于存储从网络流量中提取的各种特征数据、建立的流量模型等,以及挖掘出的过滤规则,例如针对某个攻击行为设置的流量控制参数等信息,还可以存储检测到异常流量时的告警信息、告警展示信息等。其中,选定的攻击行为包括下列攻击行为中的一种或几种:DDoS攻击、木马病毒、恶意代码和僵死病毒等。告警展示信息包括下列展示信息中的一种或几种:各攻击行为的分类告警展示、攻击信息的图表展示和异常检测的策略规则展示。The data obtained from the Internet is stored in the data storage unit, and multiple databases can be arranged in the data storage unit, such as: traffic statistics database, link layer statistics database, traffic behavior characteristic database, alarm log database, policy rule database and others Database of statistical data. They are respectively used to store various feature data extracted from network traffic, established traffic models, etc., and mined filtering rules, such as traffic control parameters set for a certain attack behavior, and can also store abnormal traffic when abnormal traffic is detected. alarm information, alarm display information, etc. Wherein, the selected attack behavior includes one or more of the following attack behaviors: DDoS attack, Trojan horse virus, malicious code, zombie virus and so on. The alarm display information includes one or more of the following display information: classified alarm display of each attack behavior, graphic display of attack information, and policy rule display of anomaly detection.
该数据存储单元主要完成存储和处理所有的数据资源,通过统一的数据管理和维护标准,实现对数据资源管理。并按照资源的类型以及面向的不同应用,提供不同的存储、处理以及访问策略,为各类应用提供统一的数据视图。该单元的数据主要来自于底层硬件组成的流量统计过滤子系统上报的流量统计数据,该单元中的流量行为样本特征库支持在线远程升级,以支持对新异常行为的识别。The data storage unit mainly completes the storage and processing of all data resources, and realizes the management of data resources through unified data management and maintenance standards. And according to the types of resources and different applications, different storage, processing and access strategies are provided to provide a unified data view for various applications. The data of this unit mainly comes from the traffic statistical data reported by the traffic statistics and filtering subsystem composed of the underlying hardware. The traffic behavior sample feature library in this unit supports online remote upgrade to support the identification of new abnormal behaviors.
上述数据分析单元可以实现从基本特征数据中提取组合特征数据,具体采用基于信息熵的特征约简方式。以及将组合特征数据输入基于量子小波神经网络的流量模型,进行数据分析。具体可以数据提取模块22、数据分析模块23和流量识别模块24。The above-mentioned data analysis unit can realize the extraction of combined feature data from the basic feature data, specifically adopting a feature reduction method based on information entropy. And input the combined feature data into the traffic model based on the quantum wavelet neural network for data analysis. Specifically, the data extraction module 22 , the data analysis module 23 and the traffic identification module 24 can be used.
此外,数据分析单元还可以实现模型建立模块21的功能,建立流量模型并交由数据存储单元存储。In addition, the data analysis unit can also realize the function of the model building module 21, and establish a traffic model and store it in the data storage unit.
数据分析单元从大量的网络流量数据中分析检测隐含的异常流量时,为了避免选用一种或几种特征所导致的检测准确性差的问题,采用了流量特征数据的分层划分思想:先从网络流量中提取基本涵盖网络流量中的全部信息的基本特征数据,使提取的特征数据能够详细的反应网络流量的运行状态。但如果对所有的基本特征数据均进行实时存储、维护和分析检测的话,对于高速网络环境而言,其复杂度极高,实现难度极大。因此在分析时针对不同的攻击行为采用不同的约简后的特征数据进行分析。When the data analysis unit analyzes and detects hidden abnormal traffic from a large amount of network traffic data, in order to avoid the problem of poor detection accuracy caused by selecting one or several features, it adopts the idea of hierarchical division of traffic feature data: first The basic feature data that basically covers all the information in the network traffic is extracted from the network traffic, so that the extracted feature data can reflect the running status of the network traffic in detail. However, if all the basic feature data are stored, maintained, analyzed and detected in real time, the complexity is extremely high for a high-speed network environment, and it is extremely difficult to implement. Therefore, different reduced characteristic data are used for analysis according to different attack behaviors.
上述数据分析单元从基本特征数据中约简出组合特征数据,用于分析是否存在异常流量。组合特征数据的集合是可以根据实际需要实时改变设置的。针对某种特定的攻击行为,将涉及该攻击行为的基本特征的子集作为描述该种攻击行为的组合特征数据。通过信息熵有关理论对网络流量的基本特征数据的最优遴选与有效约简,计算选取基本特征数据中各基本特征与选定的攻击行为的互熵,实现重要特征选取,使选取的组合特征数据为能够准确代表选定的攻击行为的有效特征数据。将这些有效特征数据加载到针对相应攻击行为建立的基于量子小波神经网络的流量模型中,就可以根据输出结果确定是否存在异常流量,并能确定异常流量是由那种攻击行为造成的。The above-mentioned data analysis unit reduces the combined feature data from the basic feature data to analyze whether there is abnormal traffic. The set of combined feature data can be changed in real time according to actual needs. For a specific attack behavior, a subset of the basic features involved in the attack behavior is used as combined feature data describing the attack behavior. Through the optimal selection and effective reduction of the basic feature data of network traffic based on the theory of information entropy, the mutual entropy between each basic feature in the selected basic feature data and the selected attack behavior is calculated to realize the selection of important features, so that the selected combined features The data is effective characteristic data that can accurately represent the selected attack behavior. Load these effective feature data into the traffic model based on the quantum wavelet neural network established for the corresponding attack behavior, and then it can be determined whether there is abnormal traffic according to the output results, and it can be determined that the abnormal traffic is caused by the attack behavior.
上述数据分析单元在检测到攻击行为时,进行策略规则挖掘,确定过滤拦截该攻击行为的流量控制参数,并及时下发给流量统计过滤子系统,实现实时地阻断异常流量。When the above-mentioned data analysis unit detects an attack behavior, it conducts policy rule mining to determine the flow control parameters for filtering and intercepting the attack behavior, and sends it to the traffic statistics and filtering subsystem in time to block abnormal traffic in real time.
本发明实施例提供的网络流量异常检测系统还包括一个应用程序单元,用于实现信息输出模块的功能。例如:应用程序单元可以针对DDoS攻击、恶意代码攻击、木马病毒、僵死病毒进行检测并向用户展示检测结果,提供告警信息的多维展示以及异常流量的阻断等。其中:The network traffic anomaly detection system provided by the embodiment of the present invention further includes an application program unit for realizing the function of the information output module. For example, the application unit can detect DDoS attacks, malicious code attacks, Trojan horse viruses, and zombie viruses, and display the detection results to users, provide multi-dimensional display of alarm information, and block abnormal traffic. in:
分布式拒绝服务(Distributed Denial of service,DDoS)攻击利用合理的服务请求来占用过多的服务资源,致使服务超载,无法响应其他的请求。这些服务资源包括网络带宽,文件系统空间容量,开放的进程或者向内的连接。系统主要从以下方面监测DDoS:SYN洪水(SYN flooding)攻击、Smurf攻击(Smurfattack)、UDP洪水(Udp flooding)攻击、死亡之拼(Ping of death)攻击、泪滴(TearDrop)攻击、Land攻击(Land attack),其提取的组合特征数据可以是目的IP地址和源IP地址,根据网络流量的数据的相同目的IP地址和源IP地址来实现拦截和阻断。Distributed Denial of Service (DDoS) attacks use reasonable service requests to occupy too many service resources, causing the service to be overloaded and unable to respond to other requests. These service resources include network bandwidth, file system space capacity, open processes or incoming connections. The system mainly monitors DDoS from the following aspects: SYN flood attack, Smurf attack, UDP flood attack, Ping of death attack, TearDrop attack, Land attack ( Land attack), the combined feature data extracted by it can be the destination IP address and the source IP address, and interception and blocking are realized according to the same destination IP address and source IP address of the network traffic data.
木马病毒利用Windows的漏洞,侵入用户计算机,控制用户计算机,窃取用户资料,其危害面积非常广、危害程度非常深。可以以IP地址、端口号等作为组合特征数据检测过滤木马病毒。可以检测出的木马类型包括:挂马网址、盗号木马、远程控制木马、破坏型的木马、拒绝服务(Denial of Service,DoS)攻击型木马、反弹端口型木马、程序杀手型、代理木马、文件传输协议(File Transfer Protocol,FTP)木马等诸多类型。The Trojan horse virus exploits the loopholes of Windows to invade the user's computer, control the user's computer, and steal user data. Its harm area is very wide and the degree of harm is very deep. It can use IP address, port number, etc. as combined feature data to detect and filter Trojan horse viruses. The types of Trojans that can be detected include: Trojans linked to websites, hacking Trojans, remote control Trojans, destructive Trojans, denial of service (Denial of Service, DoS) attack Trojans, rebound port Trojans, program killers, proxy Trojans, file Transfer Protocol (File Transfer Protocol, FTP) Trojans and many other types.
恶意代码的检测通过对网络流量的海量数据进行求精和关联分析进行检测,通过对网络流量数据包进行数据分析,与特征码比较来实现检测。Malicious code detection is detected through refinement and correlation analysis of massive network traffic data, and detection is achieved by data analysis of network traffic data packets and comparison with signatures.
僵死病毒,即僵尸网络(英文名称叫Botnet)是互联网上在网络蠕虫、特洛伊木马、后门工具等传统恶意代码形态的基础上发展、融合而产生的一种新型攻击方法,往往被黑客用来发起大规模的网络攻击,如分布式拒绝服务攻击(DDoS)、海量垃圾邮件等,同时黑客控制的这些计算机所保存的信息也都可被黑客随意“取用”。因此,不论是对网络安全运行还是用户数据安全的保护来说,僵尸网络都是极具威胁的隐患。对僵死病毒的检测可以采用协议与结构相关的僵尸网络检测方式,可以以域名作为特征数据,并结合日志分析,确定出Botnet的位置及其规模、分布等。Zombie virus, that is, botnet (English name is called Botnet) is a new attack method developed and integrated on the basis of traditional malicious code forms such as network worms, Trojan horses, and backdoor tools on the Internet. It is often used by hackers to launch Large-scale network attacks, such as distributed denial of service attacks (DDoS), mass spam, etc., and the information stored in these computers controlled by hackers can also be "accessed" by hackers at will. Therefore, whether it is for the safe operation of the network or the protection of user data security, botnets are extremely threatening hidden dangers. Botnet detection methods related to protocols and structures can be used to detect botnets. Domain names can be used as characteristic data, combined with log analysis to determine the location, scale, and distribution of Botnets.
上述各种攻击行为的展示信息可以采用图表的形式形象地展示给用户,可以展示包括攻击来源、被攻击者、攻击时间、攻击事件、过滤的事件、成功的攻击数、失败的攻击数等等在内的展示内容,并与用户读取。The display information of the above-mentioned various attack behaviors can be graphically displayed to the user in the form of charts, including attack source, attacked person, attack time, attack event, filtered event, number of successful attacks, number of failed attacks, etc. Display content within and read with the user.
告警信息展示实现了在存在攻击时向用户告警的作用。实时地将告警日志按各种条件进行分类有助于帮助管理员迅速地发现某些特定攻击。一般可以按多种标准动态地切换告警日志的分类。对于每条攻击日志可按事先的定义以不同的颜色高亮显示。The alarm information display realizes the function of alerting the user when there is an attack. Classifying alarm logs by various conditions in real time helps administrators quickly discover certain attacks. Generally, the classification of alarm logs can be dynamically switched according to various standards. Each attack log can be highlighted in different colors according to the prior definition.
应用程序单元还可以支持策略规则的展示;支持以策略组的形式进行编辑修改,管理员可以快捷地修改整个组的规则属性,包括是否激活,以及各种系统动作等;提供了用户自定义策略规则功能,并支持正则表达式。The application unit can also support the display of policy rules; it supports editing and modification in the form of policy groups, and administrators can quickly modify the rule attributes of the entire group, including whether to activate or not, and various system actions; user-defined policies are provided Rule function, and supports regular expressions.
上述系统中流量统计过滤子系统,基于“全硬件化“的处理方式,对指定高速链路上的报文进行统计、识别和过滤;网管分析子系统将集成病毒规则挖掘、数据分析、信息展示、系统运维和策略维护等功能,以报表、曲线图等多种形式展示整个网络的安全状况和病毒信息的及时告警。这种分层的入侵检测系统架构能够有效地集成40Gbps链路线速病毒规则识别、异常流量过滤、报警信息统计和网络设备维护等功能,同时该体系结构采用前端硬件平台与后台软件系统相结合的方式,由前端硬件平台完成对业务流量信息的预处理,后台软件系统再对前端子系统上报信息进行集中处理,大大减少了处理时间,能够快速、高效、准确地识别出网络中存在的异常流量。The traffic statistics filtering subsystem in the above system, based on the "full hardware" processing method, counts, identifies and filters the packets on the designated high-speed link; the network management analysis subsystem will integrate virus rule mining, data analysis, and information display , system operation and maintenance, policy maintenance and other functions, display the security status of the entire network and timely alarm of virus information in various forms such as reports and graphs. This layered intrusion detection system architecture can effectively integrate functions such as 40Gbps line-speed virus rule identification, abnormal traffic filtering, alarm information statistics, and network equipment maintenance. In this way, the front-end hardware platform completes the preprocessing of business traffic information, and the background software system centrally processes the information reported by the front-end subsystems, which greatly reduces the processing time and can quickly, efficiently and accurately identify abnormal traffic in the network .
下面以DDoS攻击包含的SYN FLOOD攻击为例,说明通过本发明实施例提供的网络流量异常检测方法和系统实现网络流量异常检测的具体过程。Taking the SYN FLOOD attack included in the DDoS attack as an example, the specific process of realizing the abnormal detection of network traffic through the network traffic anomaly detection method and system provided by the embodiment of the present invention is described below.
对于SYN FLOOD攻击,经信息熵约简后得到的组合特征向量为6个,例如下表1所示的流报文数、流字节数、包长震荡频度、包平均间隔、SYN包个数、流开始/结束时间等6个组合特征向量。所建立的针对SYN FLOOD攻击的基于量子小波神经网络的流量模型如图3所示,其中该模型的网络拓扑结构为6—12—2。输入层神经元为6个,分别对应经有效约简后的的6个组合特征向量;隐层神经元为12个;输出层神经元为2个,分别对应流量正常、异常两种状态,即c1c2=10(c1为1,c2为0)时表示流量正常状态,c1c2=01(c1为0,c2为1)时表示流量异常状态,表示存在SYN FLOOD攻击。量子小波神经网络的权值和阈值的学习率选为0.02;量子间隔的学习率为0.02;选择多层小波激励函数的斜率因子等于0.95;选择量子神经元的量子层数为4。当所有参数设置相同,即学习误差精度不设定的情况下,直到设定的学习次数完成,学习速率为0.02,最大迭代次数分别设定为580、2500。训练580次,本专利的量子小波神经网络误差为8.8761×10-4;当训练2500次时,本专利的量子小波神经网络误差为1.8978×10-5。For the SYN FLOOD attack, the combined feature vectors obtained after information entropy reduction are 6, such as the number of flow messages, flow bytes, packet length oscillation frequency, average packet interval, and SYN packet number shown in Table 1 below. 6 combined feature vectors such as number, stream start/end time, etc. The established traffic model based on quantum wavelet neural network for SYN FLOOD attack is shown in Figure 3, where the network topology of the model is 6-12-2. There are 6 neurons in the input layer, corresponding to the 6 combined eigenvectors after effective reduction; 12 neurons in the hidden layer; 2 neurons in the output layer, corresponding to the two states of normal and abnormal traffic, namely When c 1 c 2 =10 (c1 is 1, c2 is 0), it indicates a normal flow state, and when c 1 c 2 =01 (c1 is 0, c2 is 1), it indicates an abnormal flow state, indicating that there is a SYN FLOOD attack. The learning rate of the weight and threshold of the quantum wavelet neural network is selected as 0.02; the learning rate of the quantum interval is 0.02; the slope factor of the multi-layer wavelet excitation function is selected to be equal to 0.95; When all parameters are set the same, that is, the learning error precision is not set, until the set learning times are completed, the learning rate is 0.02, and the maximum iteration times are set to 580 and 2500 respectively. After training 580 times, the error of the quantum wavelet neural network of this patent is 8.8761×10 -4 ; when training 2500 times, the error of the quantum wavelet neural network of this patent is 1.8978×10 -5 .
部分训练样本数据(未归一化前)如下表1所示。Part of the training sample data (before normalization) is shown in Table 1 below.
表1Table 1
本发明实施例提供的网络流量异常检测方法和系统,适用于高速IP城域网主干网络环境,其单通道处理能力不小于40Gbps,在支持40Gbps链路接口的同时也能顺利兼容10Gbps接口。能够有效的实现网络流量中的异常流量的识别和阻断,可以实时动态的监控到攻击行为,并动态更新数据库挖掘在线过滤规则,确定流量控制参数,对变化多端的攻击行为及时识别、告警、过滤、阻断。The network traffic anomaly detection method and system provided by the embodiments of the present invention are suitable for the backbone network environment of a high-speed IP metropolitan area network, and its single-channel processing capacity is not less than 40Gbps, and can be smoothly compatible with a 10Gbps interface while supporting a 40Gbps link interface. It can effectively realize the identification and blocking of abnormal traffic in the network traffic, monitor the attack behavior dynamically in real time, and dynamically update the database to mine online filtering rules, determine the flow control parameters, and timely identify, alarm, and Filter, block.
对不同的攻击行为分别建立流量模型来实现识别不同的攻击行为,从而可以准确的检测出网络流量中存在的是哪种攻击行为,报警意义明确,检测精度高。Establish traffic models for different attack behaviors to identify different attack behaviors, so that it can accurately detect what kind of attack behavior exists in the network traffic, the alarm meaning is clear, and the detection accuracy is high.
综合运用了DPI和DFI两种识别技术,能够逐个报文一一进行协议特征识别、流量行为分析、业务分析和统计,该方法实时提取网络流量中的基本特征数据,其提取的特征数据比较全面,且针对不同攻击行为有针对性的确定组合特征数据,从而可以全面的检测各种攻击行为,利于多个管理域的协作管理。且由于实时提取网络流量中的特征数据,能够较好地满足高速网络流量实时监测的要求、能够满足业务应用种类和应用规模增大速度较快、业务应用7×24小时的高可用和高实时性等要求。Using two identification technologies, DPI and DFI, it can perform protocol feature identification, traffic behavior analysis, business analysis and statistics one by one. This method extracts basic feature data in network traffic in real time, and the extracted feature data is relatively comprehensive. , and for different attack behaviors, the combined feature data is determined in a targeted manner, so that various attack behaviors can be detected comprehensively, which is beneficial to the collaborative management of multiple management domains. And because of the real-time extraction of characteristic data in network traffic, it can better meet the requirements of real-time monitoring of high-speed network traffic, and can meet the needs of business application types and application scale. sexual requirements.
可以方便的扩展可检测的攻击行为,当有新的攻击行为时,可以建立其流量模型,对其进行检测,扩展方便。该方法能够获取良好的检测效果、较高的检测精度。整个系统采用多个独立的功能模块和子系统实现,架构灵活,结构统一,具有良好的可扩展性和重构能力。The detectable attack behavior can be easily expanded. When there is a new attack behavior, its traffic model can be established and detected, which is convenient for expansion. This method can obtain good detection effect and high detection accuracy. The whole system is implemented by multiple independent functional modules and subsystems, with flexible architecture, unified structure, and good scalability and reconfiguration capabilities.
由于提取的特征数据可以比较全面的反应网络流量的实际情况,避免了现有方式采用单一的特征数据作为判断依据所导致的检测效果不佳的问题,且可以方便的添加和减少针对不同攻击行为的流量模型,获取全面、准确的检测效果。针对某种攻击行为,通过计算训练样本中各基本特征与该攻击行为的互熵选取其中重要特征建立组合特征集合,并利用量子小波神经网络模型对组合特征集合进行学习训练以实现对流量行为的分类,收敛速度快,涵盖的流量信息全面且准确,在高速网络环境下能够实时有效提高检测精度和识别新异常行为能力,使得在整个基于网络流量模型的异常检测框架下,能比较方便地实现对不同种类的异常攻击的检测,并能取得比较好的检测效果。Since the extracted feature data can fully reflect the actual situation of network traffic, it avoids the problem of poor detection effect caused by using a single feature data as the basis for judgment in the existing method, and can easily add and reduce attacks for different attack behaviors. traffic model to obtain comprehensive and accurate detection results. For a certain attack behavior, by calculating the cross-entropy between each basic feature in the training sample and the attack behavior, the important features are selected to establish a combined feature set, and the quantum wavelet neural network model is used to learn and train the combined feature set to realize traffic behavior. Classification, fast convergence speed, comprehensive and accurate traffic information covered, can effectively improve detection accuracy and identify new abnormal behaviors in real time in a high-speed network environment, making it easier to implement under the entire anomaly detection framework based on network traffic models The detection of different types of abnormal attacks can achieve better detection results.
显然,本领域的技术人员可以对本发明进行各种改动和变型而不脱离本发明的精神和范围。这样,倘若本发明的这些修改和变型属于本发明权利要求及其等同技术的范围之内,则本发明也意图包含这些改动和变型在内。Obviously, those skilled in the art can make various changes and modifications to the present invention without departing from the spirit and scope of the present invention. Thus, if these modifications and variations of the present invention fall within the scope of the claims of the present invention and their equivalent technologies, the present invention also intends to include these modifications and variations.
Claims (17)
Priority Applications (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
CN201110154226.3A CN102821002B (en) | 2011-06-09 | 2011-06-09 | Network flow abnormal detecting method and system |
Applications Claiming Priority (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
CN201110154226.3A CN102821002B (en) | 2011-06-09 | 2011-06-09 | Network flow abnormal detecting method and system |
Publications (2)
Publication Number | Publication Date |
---|---|
CN102821002A CN102821002A (en) | 2012-12-12 |
CN102821002B true CN102821002B (en) | 2015-08-26 |
Family
ID=47304873
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
CN201110154226.3A Active CN102821002B (en) | 2011-06-09 | 2011-06-09 | Network flow abnormal detecting method and system |
Country Status (1)
Country | Link |
---|---|
CN (1) | CN102821002B (en) |
Cited By (1)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN113747443B (en) * | 2021-02-26 | 2024-06-07 | 上海观安信息技术股份有限公司 | Safety detection method and device based on machine learning algorithm |
Families Citing this family (78)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
WO2014094254A1 (en) * | 2012-12-19 | 2014-06-26 | 华为技术有限公司 | Method, apparatus and network device for monitoring network |
CN103152442B (en) * | 2013-01-31 | 2016-06-01 | 中国科学院计算机网络信息中心 | A kind of detection and treatment method of corpse domain names and system |
CN103095728B (en) * | 2013-02-07 | 2016-04-27 | 重庆大学 | A kind of network security points-scoring system of Behavior-based control data fusion and method |
CN103117903B (en) * | 2013-02-07 | 2016-01-06 | 中国联合网络通信集团有限公司 | Surfing flow method for detecting abnormality and device |
CN104077524B (en) * | 2013-03-25 | 2018-01-09 | 腾讯科技(深圳)有限公司 | Training method and viruses indentification method and device for viruses indentification |
CN104079545A (en) * | 2013-03-29 | 2014-10-01 | 西门子公司 | Method, device and system for extracting data package filtering rules |
CN104519031B (en) * | 2013-09-30 | 2018-03-09 | 西门子公司 | A kind of method and apparatus for hostile network behavioral value |
CN103716180A (en) * | 2013-12-04 | 2014-04-09 | 国网上海市电力公司 | Network flow actual forecasting-based network abnormality pre-warning method |
CN103746987B (en) * | 2013-12-31 | 2017-02-01 | 东软集团股份有限公司 | Method and system for detecting DoS attack in semantic Web application |
CN103973684B (en) * | 2014-05-07 | 2017-05-24 | 北京神州绿盟信息安全科技股份有限公司 | Rule compiling and matching method and device |
CN104219221A (en) * | 2014-05-30 | 2014-12-17 | 郭瑞 | Network security flow generating method and network security flow generating system |
CN104202336A (en) * | 2014-09-22 | 2014-12-10 | 浪潮电子信息产业股份有限公司 | DDoS attack detection method based on information entropy |
CN104994056B (en) * | 2015-05-11 | 2018-01-19 | 中国电力科学研究院 | The dynamic updating method of flow identification model in a kind of Power Information Network |
CN105208026A (en) * | 2015-09-29 | 2015-12-30 | 努比亚技术有限公司 | Hostile attack preventing method and network system |
CN105975852A (en) * | 2015-12-31 | 2016-09-28 | 武汉安天信息技术有限责任公司 | Method and system for detecting sample relevance based on label propagation |
CN107196816B (en) * | 2016-03-14 | 2020-11-03 | 中国移动通信集团江西有限公司 | Abnormal flow detection method and system and network analysis equipment |
CN105847043A (en) * | 2016-03-21 | 2016-08-10 | 浪潮通信信息系统有限公司 | Mobile user internet surfing diagnosis method |
CN105873105B (en) * | 2016-04-22 | 2018-07-03 | 中国科学技术大学 | A kind of mobile radio communication abnormality detection and localization method based on network Quality of experience |
TWI599905B (en) * | 2016-05-23 | 2017-09-21 | 緯創資通股份有限公司 | Protecting method and system for malicious code, and monitor apparatus |
CN106130960B (en) * | 2016-06-12 | 2019-08-09 | 微梦创科网络科技(中国)有限公司 | System for judging hacking behavior, load scheduling method and device |
CN107707509B (en) * | 2016-08-08 | 2020-09-29 | 阿里巴巴集团控股有限公司 | Method, device and system for identifying and assisting in identifying fake traffic |
CN106447383A (en) * | 2016-08-30 | 2017-02-22 | 杭州启冠网络技术有限公司 | Cross-time multi-dimensional abnormal data monitoring method and system |
CN106411597A (en) * | 2016-10-14 | 2017-02-15 | 广东工业大学 | Network traffic abnormality detection method and system |
CN106685964B (en) * | 2016-12-29 | 2020-10-30 | 济南大学 | Malware detection method and system based on malicious network traffic thesaurus |
CN106850632B (en) * | 2017-02-10 | 2020-09-29 | 北京奇艺世纪科技有限公司 | Method and device for detecting abnormal combined data |
CN107104960A (en) * | 2017-04-20 | 2017-08-29 | 四川电科智造科技有限公司 | A kind of industrial control system intrusion detection method based on machine learning |
CN107196930B (en) * | 2017-05-12 | 2019-11-29 | 苏州优圣美智能系统有限公司 | The method of computer network abnormality detection |
CN108933706B (en) * | 2017-05-23 | 2022-02-25 | 华为技术有限公司 | A method, device and system for monitoring data flow |
CN108287782A (en) * | 2017-06-05 | 2018-07-17 | 中兴通讯股份有限公司 | A kind of multidimensional data method for detecting abnormality and device |
CN107508816A (en) * | 2017-08-31 | 2017-12-22 | 杭州迪普科技股份有限公司 | A kind of attack traffic means of defence and device |
CN107633058B (en) * | 2017-09-20 | 2021-06-01 | 武汉虹旭信息技术有限责任公司 | Deep learning-based data dynamic filtering system and method |
JP6835703B2 (en) * | 2017-12-01 | 2021-02-24 | 日本電信電話株式会社 | Cyber attack detection system, feature selection system, cyber attack detection method, and program |
CN109889470B (en) * | 2017-12-06 | 2020-06-26 | 中国科学院声学研究所 | A method and system for defending against DDoS attacks based on routers |
CN108171053B (en) * | 2017-12-28 | 2020-06-12 | 北京奇虎科技有限公司 | Rule discovery method and system |
CN108259494B (en) * | 2018-01-17 | 2020-10-02 | 北京邮电大学 | A network attack detection method and device |
CN108322463A (en) * | 2018-01-31 | 2018-07-24 | 平安科技(深圳)有限公司 | Ddos attack detection method, device, computer equipment and storage medium |
CN108494791A (en) * | 2018-04-08 | 2018-09-04 | 北京明朝万达科技股份有限公司 | A kind of DDOS attack detection method and device based on Netflow daily record datas |
CN108647292A (en) * | 2018-05-07 | 2018-10-12 | 前海梧桐(深圳)数据有限公司 | Enterprise's property sort computational methods based on neural network algorithm and system |
CN109039797B (en) * | 2018-06-11 | 2021-11-23 | 电子科技大学 | Strong learning based large flow detection method |
TWI684113B (en) * | 2018-08-28 | 2020-02-01 | 中華電信股份有限公司 | Gateway apparatus, detecting method of malicious domain and hacked host, and non-transitory computer readable medium thereof |
CN109120630B (en) * | 2018-09-03 | 2022-08-02 | 上海海事大学 | A DDoS attack detection method for SDN network based on optimized BP neural network |
CN109040113B (en) * | 2018-09-04 | 2021-03-19 | 海南大学 | Distributed denial of service attack detection method and device based on multi-core learning |
CN109067773B (en) * | 2018-09-10 | 2020-10-27 | 成都信息工程大学 | Vehicle-mounted CAN network intrusion detection method and system based on neural network |
CN110909753B (en) * | 2018-09-14 | 2022-07-01 | 中国移动通信集团广东有限公司 | Data classification method, system and equipment |
CN109327453B (en) * | 2018-10-31 | 2021-04-13 | 北斗智谷(北京)安全技术有限公司 | Specific threat identification method and electronic equipment |
CN109450895B (en) * | 2018-11-07 | 2021-07-02 | 北京锐安科技有限公司 | A kind of traffic identification method, device, server and storage medium |
CN110784435B (en) * | 2019-04-15 | 2020-11-06 | 北京嘀嘀无限科技发展有限公司 | Abnormal service identification method and device, electronic equipment and storage medium |
CN110602029B (en) * | 2019-05-15 | 2022-06-28 | 上海云盾信息技术有限公司 | Method and system for identifying network attack |
CN110149331B (en) * | 2019-05-22 | 2021-07-06 | 中国科学院长春光学精密机械与物理研究所 | A P2P botnet detection method, device and medium |
CN110287699B (en) * | 2019-06-12 | 2021-02-26 | 杭州迪普科技股份有限公司 | Application program feature extraction method and device |
CN110247911B (en) * | 2019-06-14 | 2021-06-08 | 曹严清 | Flow abnormity detection method and system |
CN110336830B (en) * | 2019-07-17 | 2021-08-31 | 山东大学 | A DDoS attack detection system based on software-defined network |
CN110569282A (en) * | 2019-09-04 | 2019-12-13 | 中国工商银行股份有限公司 | Data processing method, data processing device, computing equipment and computer readable storage medium |
CN111092852B (en) * | 2019-10-16 | 2023-04-07 | 平安科技(深圳)有限公司 | Network security monitoring method, device, equipment and storage medium based on big data |
CN112866185B (en) * | 2019-11-28 | 2021-11-02 | 海信集团有限公司 | Network traffic monitoring device and abnormal traffic detection method |
CN111131237B (en) * | 2019-12-23 | 2020-12-29 | 深圳供电局有限公司 | Microgrid attack identification method and grid-connected interface device based on BP neural network |
CN113746686A (en) * | 2020-05-27 | 2021-12-03 | 阿里巴巴集团控股有限公司 | Network flow state determination method, computing device and storage medium |
CN111756708B (en) * | 2020-06-09 | 2022-06-28 | 北京天空卫士网络安全技术有限公司 | Method and device for detecting directional threat attack |
CN113810207B (en) * | 2020-06-12 | 2024-11-12 | 天翼云科技有限公司 | Root cause server positioning method and root cause server positioning device |
CN111782472B (en) * | 2020-06-30 | 2022-04-26 | 平安科技(深圳)有限公司 | System abnormality detection method, device, equipment and storage medium |
CN111756584A (en) * | 2020-07-14 | 2020-10-09 | 济南浪潮高新科技投资发展有限公司 | Netflow protocol network flow analysis method and system based on deep learning |
CN111953695B (en) * | 2020-08-14 | 2022-10-28 | 中国工商银行股份有限公司 | Terminal behavior portrait construction method and device, electronic equipment and storage medium |
CN112001443A (en) * | 2020-08-24 | 2020-11-27 | 成都卫士通信息产业股份有限公司 | Monitoring method, device, storage medium and electronic device for network behavior data |
CN111818103B (en) * | 2020-09-09 | 2020-12-15 | 信联科技(南京)有限公司 | Traffic-based tracing attack path method in network target range |
CN112491806A (en) * | 2020-11-04 | 2021-03-12 | 深圳供电局有限公司 | Cloud platform flow security analysis system and method |
CN112333023A (en) * | 2020-11-06 | 2021-02-05 | 四川师范大学 | Intrusion detection system based on flow of Internet of things and detection method thereof |
CN112907321B (en) * | 2021-02-03 | 2021-08-27 | 珠海市鸿瑞信息技术股份有限公司 | Big data-based information security anomaly sensing platform for data mining and analysis |
CN113010884B (en) * | 2021-02-23 | 2022-08-26 | 重庆邮电大学 | Real-time feature filtering method in intrusion detection system |
CN115134099B (en) * | 2021-03-22 | 2024-05-03 | 中国移动通信集团江苏有限公司 | Network attack behavior analysis method and device based on full flow |
CN113625681B (en) * | 2021-07-19 | 2022-12-13 | 湖南大学 | CAN bus abnormality detection method, system and storage medium |
CN114666162B (en) * | 2022-04-29 | 2023-05-05 | 北京火山引擎科技有限公司 | Flow detection method, device, equipment and storage medium |
CN115174240A (en) * | 2022-07-13 | 2022-10-11 | 中国国家铁路集团有限公司 | Railway encrypted flow monitoring system and method |
CN115174254B (en) * | 2022-07-22 | 2023-10-31 | 科来网络技术股份有限公司 | Flow abnormality warning method and device, electronic equipment and storage medium |
CN116208356B (en) * | 2022-10-27 | 2023-09-29 | 浙江大学 | A virtual currency mining traffic detection method based on deep learning |
CN116192451A (en) * | 2022-12-26 | 2023-05-30 | 北京华清信安科技有限公司 | PTH attack detection method based on random tree algorithm |
CN116074215B (en) * | 2022-12-30 | 2024-04-19 | 中国联合网络通信集团有限公司 | Network quality detection method, device, equipment and storage medium |
CN116723138B (en) * | 2023-08-10 | 2023-10-20 | 杭银消费金融股份有限公司 | Abnormal flow monitoring method and system based on flow probe dyeing |
CN118555152B (en) * | 2024-07-30 | 2024-10-29 | 江西广投优炫信息技术有限公司 | Network application vulnerability anomaly detection method, system, electronic device and storage medium |
Citations (1)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN101221554A (en) * | 2008-01-25 | 2008-07-16 | 北京工业大学 | EEG Feature Extraction Method Based on Wavelet Transform and BP Neural Network |
Family Cites Families (1)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
EP2068516B1 (en) * | 2002-02-19 | 2018-09-05 | Google LLC | E-mail management services |
-
2011
- 2011-06-09 CN CN201110154226.3A patent/CN102821002B/en active Active
Patent Citations (1)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN101221554A (en) * | 2008-01-25 | 2008-07-16 | 北京工业大学 | EEG Feature Extraction Method Based on Wavelet Transform and BP Neural Network |
Non-Patent Citations (1)
Title |
---|
一种基于粗糙集属性约简的支持向量异常入侵检测方法;张义荣等;《计算机科学》;20061231;第33卷(第6期);第64-68页 * |
Cited By (1)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN113747443B (en) * | 2021-02-26 | 2024-06-07 | 上海观安信息技术股份有限公司 | Safety detection method and device based on machine learning algorithm |
Also Published As
Publication number | Publication date |
---|---|
CN102821002A (en) | 2012-12-12 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
CN102821002B (en) | Network flow abnormal detecting method and system | |
Zhang et al. | Real-time distributed-random-forest-based network intrusion detection system using Apache spark | |
Siddiqui et al. | Analysis of KDD CUP 99 dataset using clustering based data mining | |
KR101409563B1 (en) | Method and apparatus for identifying application protocol | |
US7424744B1 (en) | Signature based network intrusion detection system and method | |
Gao et al. | A dos resilient flow-level intrusion detection approach for high-speed networks | |
Alaidaros et al. | An overview of flow-based and packet-based intrusion detection performance in high speed networks | |
Wei et al. | Profiling and Clustering Internet Hosts. | |
Kim et al. | Deep reinforcement learning-based traffic sampling for multiple traffic analyzers on software-defined networks | |
Amza et al. | Hybrid network intrusion detection | |
Qiu et al. | Global Flow Table: A convincing mechanism for security operations in SDN | |
CN100377534C (en) | A network worm detection system and method | |
Ma et al. | Synchronizing DDoS detection and mitigation based graph learning with programmable data plane, SDN | |
Sukhni et al. | A systematic analysis for botnet detection using genetic algorithm | |
CN103501302A (en) | Method and system for automatically extracting worm features | |
Srinivasa | Application of genetic algorithms for detecting anomaly in network intrusion detection systems | |
Yange et al. | A data analytics system for network intrusion detection using decision tree | |
Siffer et al. | Netspot: A simple Intrusion Detection System with statistical learning | |
CN116436691A (en) | A system and method for detecting APT attacks based on the Industrial Internet of Things | |
TW202017337A (en) | Method and system for backbone network flow anomaly detection | |
Abudalfa et al. | Evaluating performance of supervised learning techniques for developing real-time intrusion detection system | |
Chopra et al. | Dynamic tracing of dos attack over software-defined networks using machine learning | |
Raju et al. | Statistical evaluation of network packets in an intrusion detection mechanism using ML and DL techniques | |
Nandhini et al. | Classification and analysis of distributed denial of service attacks using machine learning techniques | |
Labib et al. | Nsom: A tool to detect denial of service attacks using self-organizing maps |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
C06 | Publication | ||
PB01 | Publication | ||
C10 | Entry into substantive examination | ||
SE01 | Entry into force of request for substantive examination | ||
C14 | Grant of patent or utility model | ||
GR01 | Patent grant |