CN109033431A - A kind of efficient electric power networks data acquisition and intelligent analysis system and processing method - Google Patents
A kind of efficient electric power networks data acquisition and intelligent analysis system and processing method Download PDFInfo
- Publication number
- CN109033431A CN109033431A CN201810913614.7A CN201810913614A CN109033431A CN 109033431 A CN109033431 A CN 109033431A CN 201810913614 A CN201810913614 A CN 201810913614A CN 109033431 A CN109033431 A CN 109033431A
- Authority
- CN
- China
- Prior art keywords
- event
- merger
- information
- data
- acquisition
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
- G06Q50/00—Information and communication technology [ICT] specially adapted for implementation of business processes of specific business sectors, e.g. utilities or tourism
- G06Q50/06—Energy or water supply
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02E—REDUCTION OF GREENHOUSE GAS [GHG] EMISSIONS, RELATED TO ENERGY GENERATION, TRANSMISSION OR DISTRIBUTION
- Y02E40/00—Technologies for an efficient electrical power generation, transmission or distribution
- Y02E40/70—Smart grids as climate change mitigation technology in the energy generation sector
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y04—INFORMATION OR COMMUNICATION TECHNOLOGIES HAVING AN IMPACT ON OTHER TECHNOLOGY AREAS
- Y04S—SYSTEMS INTEGRATING TECHNOLOGIES RELATED TO POWER NETWORK OPERATION, COMMUNICATION OR INFORMATION TECHNOLOGIES FOR IMPROVING THE ELECTRICAL POWER GENERATION, TRANSMISSION, DISTRIBUTION, MANAGEMENT OR USAGE, i.e. SMART GRIDS
- Y04S10/00—Systems supporting electrical power generation, transmission or distribution
- Y04S10/50—Systems or methods supporting the power network operation or management, involving a certain degree of interaction with the load-side end user applications
Landscapes
- Business, Economics & Management (AREA)
- Health & Medical Sciences (AREA)
- Engineering & Computer Science (AREA)
- Economics (AREA)
- Public Health (AREA)
- Water Supply & Treatment (AREA)
- General Health & Medical Sciences (AREA)
- Human Resources & Organizations (AREA)
- Marketing (AREA)
- Primary Health Care (AREA)
- Strategic Management (AREA)
- Tourism & Hospitality (AREA)
- Physics & Mathematics (AREA)
- General Business, Economics & Management (AREA)
- General Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- Computer And Data Communications (AREA)
Abstract
The invention discloses a kind of efficient electric power networks data acquisition and intelligent analysis systems, including data acquisition module, are used for collecting data information；Intelligent data analysis module, for coming out the information filtering in data information containing event；Event merger processing module regularly sends all merger events to data memory module for counting the merger number of different event in a period of time, then by timer；Data memory module, for storing collection event and acquisition information.The present invention can improve the deficiencies in the prior art, enhance the safety of data dissemination.
Description
Technical field
The present invention relates to power monitoring technical field, especially a kind of efficient electric power networks data acquisition and intellectual analysis
System and processing method.
Background technique
In recent years, Ukraine's large-area power-cuts (2015), eastern United States Internet service paralysis (2016 occur in succession
Year), whole world outburst extort events such as viral (2017), electric system has become the important target of attack of international network war, electricity
Power monitoring system security protection bears immense pressure, and general safety product is generally basede on network flow and message analyzing technique,
Mainly internet generic service and agreement are monitored, analyzed, cyberspace isolation, equipment and user are determined relatively,
Network service it is privately owned it is controllable, be not optimal selection under normal circumstances for unattended electric power monitoring system.There is an urgent need to
It researches and develops and is suitble to electric power monitoring system, the network data acquisition and analysis system of equipment oriented event.
Summary of the invention
The technical problem to be solved in the present invention is to provide a kind of efficient electric power networks data acquisition and intelligent analysis systems
And processing method, the deficiencies in the prior art are able to solve, the safety of data dissemination is enhanced.
In order to solve the above technical problems, the technical solution used in the present invention is as follows.
A kind of efficient electric power networks data acquisition and intelligent analysis system, including,
Data acquisition module is used for collecting data information；
Intelligent data analysis module, for coming out the information filtering in data information containing event；
Event merger processing module is regularly sent for counting the merger number of different event in a period of time, then by timer
All merger events are to data memory module；
Data memory module, for storing collection event and acquisition information.
Preferably, the data acquisition module is set by server and/or work station and/or database and/or network
Standby and/or safety protection equipment carries out data acquisition.
Preferably, the message channel that data acquisition module receives data information see the table below,
The specific message of message format GB/T 31992 is,
<rank><space>date<space>time<space>equipment or system<space>behavior<space>reason；
The format of reason is,
<event type><space><event subtype><space><content>.
A kind of processing method of above-mentioned efficient electric power networks data acquisition and intelligent analysis system, including following step
It is rapid:
A, data acquisition module utilizes multithreading, libevent asynchronous mechanism, receives by acquiring equipment hair in electric power networks
The acquisition information brought improves the efficiency for receiving message, in the way of more queue processings, by the collected acquisition of different threads
Information is put into different queues；
B, intelligent data analysis module will acquire information in the information filtering containing event come out, by establish merger event base with
Event base will acquire " equipment or system " in information, " event type " and " subevent type " in the way of lookup interception
Content obtains out, and good by the format group of event base and merger event base, then using matching algorithm and merger event base and
Event base is matched；
C, event merger processing module uses conflation algorithm, counts the merger number of different event in a period of time, then passes through timing
Device regularly sends all merger events to data memory module, avoid in a period of time it is continuous and it is duplicate report similar events,
Memory space is saved, the efficiency of user's searched events is improved；
D, data memory module is using mysql database purchase collection event and acquisition information, and the acquisition information of storage is at least
It saves 3 months, event information preservation 1 year.
Preferably, information of the filtering containing event includes the following steps in step B,
B1, acquisition information is taken out from queue；
B2, judge to acquire whether information needs to do merger；
B3, merger is if desired carried out, is then sent to event merger processing module and carries out predetermined processing, step terminates；
If B4, not needing to carry out merger, acquisition information is sent to data memory module；
B5, judge to acquire whether information is event, if acquisition information is event, acquisition information is converted into event.
Preferably, statistics merger event times include the following steps in step C,
C1, creation hash table, calculate hash value based on IP；
C2, judge in hash table, whether IP value is equal, if unequal, create new hash node and parses if equal
Different type merger event；
C3, judge whether it is for the first time parse such merger event, if it is not, merger event times are then counted, if so, directly
Sending and receiving are sent to data memory module.
Include the following steps preferably, regularly sending merger event,
Poll hash table is removed by playing an individual thread timing, new event is judged whether there is and generates, if there is then updating
The number of repetition of event, sends an event to data memory module, and the value of now_count is assigned to pre_count after having sent；
Whether 30S is differed with current time value if the time value for checking IP node without if, if it is, thinking that the IP node exists
In 30S, new event is not generated, it is believed that the probability for occurring merger event behind the IP node is low probability, deletes IP section
Point reduces the number of traversal.
Brought beneficial effect is by adopting the above technical scheme: the present invention can support to acquire in a variety of electric power networks
Equipment receives the acquisition information of different acquisition equipment, and by intellectual analysis, analyzes warning information, timely convenient for user
Electric power networks field device operation conditions is solved, the equipment for accident occur can be found by warning information when occurring abnormal；This is
The system privately owned communication format proprietary using electric power with acquisition equipment room, carries out message communicating, enhances the safety of data dissemination.
Traditional conflation algorithm is handled for sequence, and conflation algorithm of the invention is duplicate removal.Traditional duplicate removal processing
It would generally traverse one by one and compare its inefficiency of item and cumbersome, the present invention is utilized except remaining hash algorithm is left and taken, and passes through creation hash
Table, the event node for needing duplicate removal can be quickly found out by calculating hash value.
Detailed description of the invention
Fig. 1 is the structure chart of a specific embodiment of the invention.
Fig. 2 is the flow chart that the information containing event is filtered in a specific embodiment of the invention.
Fig. 3 is statistics merger event times and the process for regularly sending merger event in a specific embodiment of the invention
Figure.
Specific embodiment
Referring to Fig.1-3, a specific embodiment of the invention includes
Data acquisition module 1 is used for collecting data information；
Intelligent data analysis module 2, for coming out the information filtering in data information containing event；
Event merger processing module 3 is regularly sent for counting the merger number of different event in a period of time, then by timer
All merger events are to data memory module 4；
Data memory module 4, for storing collection event and acquisition information.
The data acquisition module 1 passes through server and/or work station and/or database and/or the network equipment and/or peace
Full protection equipment carries out data acquisition.
The message channel that data acquisition module 1 receives data information see the table below,
The specific message of message format GB/T 31992 is,
<rank><space>date<space>time<space>equipment or system<space>behavior<space>reason；
The format of reason is,
<event type><space><event subtype><space><content>.
A kind of processing method of above-mentioned efficient electric power networks data acquisition and intelligent analysis system, including following step
It is rapid:
A, data acquisition module 1 utilizes multithreading, libevent asynchronous mechanism, receives by acquiring equipment hair in electric power networks
The acquisition information brought improves the efficiency for receiving message, in the way of more queue processings, by the collected acquisition of different threads
Information is put into different queues；
B, intelligent data analysis module 2 will acquire the information filtering containing event in information and come out, by establishing merger event base
" equipment or system " in information, " event type " and " subevent class will be acquired in the way of lookup interception with event base
Type " content obtains out, and good by the format group of event base and merger event base, then utilizes matching algorithm and merger event base
It is matched with event base；
C, event merger processing module 3 uses conflation algorithm, counts the merger number of different event in a period of time, then passes through timing
Device regularly sends all merger events to data memory module, avoid in a period of time it is continuous and it is duplicate report similar events,
Memory space is saved, the efficiency of user's searched events is improved；
D, data memory module 4 is using mysql database purchase collection event and acquisition information, and the acquisition information of storage is at least
It saves 3 months, event information preservation 1 year.
In step B, information of the filtering containing event includes the following steps,
B1, acquisition information is taken out from queue；
B2, judge to acquire whether information needs to do merger；
B3, merger is if desired carried out, is then sent to event merger processing module 3 and carries out predetermined processing, step terminates；
If B4, not needing to carry out merger, acquisition information is sent to data memory module 4；
B5, judge to acquire whether information is event, if acquisition information is event, acquisition information is converted into event.
In step C, statistics merger event times include the following steps,
C1, creation hash table, calculate hash value based on IP；
C2, judge in hash table, whether IP value is equal, if unequal, create new hash node and parses if equal
Different type merger event；
C3, judge whether it is for the first time parse such merger event, if it is not, merger event times are then counted, if so, directly
Sending and receiving are sent to data memory module.
Merger event is regularly sent to include the following steps,
Poll hash table is removed by playing an individual thread timing, new event is judged whether there is and generates, if there is then updating
The number of repetition of event, sends an event to data memory module, and the value of now_count is assigned to pre_count after having sent；
Whether 30S is differed with current time value if the time value for checking IP node without if, if it is, thinking that the IP node exists
In 30S, new event is not generated, it is believed that the probability for occurring merger event behind the IP node is low probability, deletes IP section
Point reduces the number of traversal.
The above shows and describes the basic principles and main features of the present invention and the advantages of the present invention.The technology of the industry
Personnel are it should be appreciated that the present invention is not limited to the above embodiments, and the above embodiments and description only describe this
The principle of invention, without departing from the spirit and scope of the present invention, various changes and improvements may be made to the invention, these changes
Change and improvement all fall within the protetion scope of the claimed invention.The claimed scope of the invention by appended claims and its
Equivalent thereof.
Claims (7)
1. a kind of efficient electric power networks data acquisition and intelligent analysis system, it is characterised in that: including,
Data acquisition module (1) is used for collecting data information；
Intelligent data analysis module (2), for coming out the information filtering in data information containing event；
Event merger processing module (3) is sent out for counting the merger number of different event in a period of time, then by timer timing
All merger events are sent to give data memory module (4)；
Data memory module (4), for storing collection event and acquisition information.
2. efficient electric power networks data acquisition according to claim 1 and intelligent analysis system, it is characterised in that: described
Data acquisition module (1) passes through server and/or work station and/or database and/or the network equipment and/or safety protection equipment
Carry out data acquisition.
3. efficient electric power networks data acquisition according to claim 2 and intelligent analysis system, it is characterised in that: data
The message channel that acquisition module (1) receives data information see the table below,
The specific message of message format GB/T 31992 is,
<rank><space>date<space>time<space>equipment or system<space>behavior<space>reason；
The format of reason is,
<event type><space><event subtype><space><content>.
4. the processing of efficient electric power networks data acquisition and intelligent analysis system described in a kind of claim 1-3 any one
Method, it is characterised in that the following steps are included:
A, data acquisition module (1) utilizes multithreading, libevent asynchronous mechanism, receives by acquiring equipment in electric power networks
The acquisition information sended over improves the efficiency for receiving message, in the way of more queue processings, adopts different threads are collected
Collection information is put into different queues；
B, intelligent data analysis module (2) will acquire the information filtering containing event in information and come out, by establishing merger event
Library and event base will acquire " equipment or system " in information, " event type " and " subevent in the way of lookup interception
Type " content obtains out, and good by the format group of event base and merger event base, then utilizes matching algorithm and merger event
Library and event base are matched；
C, event merger processing module (3) uses conflation algorithm, counts the merger number of different event in a period of time, then by fixed
When device regularly send all merger events to data memory module, avoid continuous in a period of time and duplicate report from mutually working together
Part saves memory space, improves the efficiency of user's searched events；
D, data memory module (4) is using mysql database purchase collection event and acquisition information, and the acquisition information of storage is extremely
It saves less 3 months, event information preservation 1 year.
5. the processing method of efficient electric power networks data acquisition and intelligent analysis system according to claim 4, special
Sign is: in step B, information of the filtering containing event includes the following steps,
B1, acquisition information is taken out from queue；
B2, judge to acquire whether information needs to do merger；
B3, merger is if desired carried out, is then sent to event merger processing module (3) and carries out predetermined processing, step terminates；
If B4, not needing to carry out merger, acquisition information is sent to data memory module (4)；
B5, judge to acquire whether information is event, if acquisition information is event, acquisition information is converted into event.
6. the processing method of efficient electric power networks data acquisition and intelligent analysis system according to claim 4, special
Sign is: in step C, statistics merger event times include the following steps,
C1, creation hash table, calculate hash value based on IP；
C2, judge in hash table, whether IP value is equal, if unequal, create new hash node and parses if equal
Different type merger event；
C3, judge whether it is for the first time parse such merger event, if it is not, merger event times are then counted, if so, directly
Sending and receiving are sent to data memory module.
7. the processing method of efficient electric power networks data acquisition and intelligent analysis system according to claim 4, special
Sign is: it regularly sends merger event and includes the following steps,
Poll hash table is removed by playing an individual thread timing, new event is judged whether there is and generates, if there is then updating
The number of repetition of event, sends an event to data memory module, and the value of now_count is assigned to pre_count after having sent；
Whether 30S is differed with current time value if the time value for checking IP node without if, if it is, thinking that the IP node exists
In 30S, new event is not generated, it is believed that the probability for occurring merger event behind the IP node is low probability, deletes IP section
Point reduces the number of traversal.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201810913614.7A CN109033431A (en) | 2018-08-13 | 2018-08-13 | A kind of efficient electric power networks data acquisition and intelligent analysis system and processing method |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201810913614.7A CN109033431A (en) | 2018-08-13 | 2018-08-13 | A kind of efficient electric power networks data acquisition and intelligent analysis system and processing method |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN109033431A true CN109033431A (en) | 2018-12-18 |
Family
ID=64632884
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201810913614.7A Pending CN109033431A (en) | 2018-08-13 | 2018-08-13 | A kind of efficient electric power networks data acquisition and intelligent analysis system and processing method |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN109033431A (en) |
Cited By (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110929896A (en) * | 2019-12-04 | 2020-03-27 | 全球能源互联网研究院有限公司 | A safety analysis method and device for system equipment |
| CN111092865A (en) * | 2019-12-04 | 2020-05-01 | 全球能源互联网研究院有限公司 | Security event analysis method and system |
| CN112510825A (en) * | 2020-11-18 | 2021-03-16 | 北京智芯微电子科技有限公司 | Real-time power failure active reporting and de-duplication method and system for high-speed power line carrier communication |
| CN113612641A (en) * | 2021-08-03 | 2021-11-05 | 中能融合智慧科技有限公司 | Efficient log flow acquisition and intelligent analysis system based on energy network |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20020019945A1 (en) * | 2000-04-28 | 2002-02-14 | Internet Security System, Inc. | System and method for managing security events on a network |
| CN104079430A (en) * | 2014-06-09 | 2014-10-01 | 汉柏科技有限公司 | Safety management platform, system and method based on information |
| CN104852927A (en) * | 2015-06-01 | 2015-08-19 | 国家电网公司 | Safety comprehensive management system based on multi-source heterogeneous information |
| CN108090186A (en) * | 2017-12-16 | 2018-05-29 | 国网信通亿力科技有限责任公司 | A kind of electric power data De-weight method on big data platform |
-
2018
- 2018-08-13 CN CN201810913614.7A patent/CN109033431A/en active Pending
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20020019945A1 (en) * | 2000-04-28 | 2002-02-14 | Internet Security System, Inc. | System and method for managing security events on a network |
| CN104079430A (en) * | 2014-06-09 | 2014-10-01 | 汉柏科技有限公司 | Safety management platform, system and method based on information |
| CN104852927A (en) * | 2015-06-01 | 2015-08-19 | 国家电网公司 | Safety comprehensive management system based on multi-source heterogeneous information |
| CN108090186A (en) * | 2017-12-16 | 2018-05-29 | 国网信通亿力科技有限责任公司 | A kind of electric power data De-weight method on big data platform |
Non-Patent Citations (1)
| Title |
|---|
| 周铁: "电力行业安全事件处理系统的设计与实现", 《中国优秀硕士学位论文全文数据库信息科技辑》 * |
Cited By (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110929896A (en) * | 2019-12-04 | 2020-03-27 | 全球能源互联网研究院有限公司 | A safety analysis method and device for system equipment |
| CN111092865A (en) * | 2019-12-04 | 2020-05-01 | 全球能源互联网研究院有限公司 | Security event analysis method and system |
| CN112510825A (en) * | 2020-11-18 | 2021-03-16 | 北京智芯微电子科技有限公司 | Real-time power failure active reporting and de-duplication method and system for high-speed power line carrier communication |
| CN112510825B (en) * | 2020-11-18 | 2022-01-14 | 北京智芯微电子科技有限公司 | Real-time power failure active reporting and de-duplication method and system for high-speed power line carrier communication |
| CN113612641A (en) * | 2021-08-03 | 2021-11-05 | 中能融合智慧科技有限公司 | Efficient log flow acquisition and intelligent analysis system based on energy network |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN109033431A (en) | A kind of efficient electric power networks data acquisition and intelligent analysis system and processing method | |
| CN108040074B (en) | A real-time network abnormal behavior detection system and method based on big data | |
| CN107835087A (en) | A kind of safety means alarm regulation extraction method based on Frequent Pattern Mining | |
| CN105491013B (en) | A kind of multiple-domain network Security Situation Awareness Systems and method based on SDN | |
| CN110401642A (en) | A method for collecting and analyzing industrial control traffic | |
| CN108055282A (en) | Industry control abnormal behaviour analysis method and system based on self study white list | |
| CN111935074A (en) | Integrated network security detection method and device | |
| CN102882881B (en) | Special data filtering method for eliminating denial-of-service attacks to DNS (domain name system) service | |
| CN104202336A (en) | DDoS attack detection method based on information entropy | |
| CN112350882A (en) | Distributed network traffic analysis system and method | |
| CN105790990B (en) | A kind of method and its system for supervising adapted telecommunication business | |
| CN102811162A (en) | Method and apparatus for detecting network attacks using a flow based technique | |
| CN112688822A (en) | Edge computing fault or security threat monitoring system and method based on multi-point cooperation | |
| CN111080500A (en) | Spark streaming based real-time rule deployment and control early warning method and device | |
| CN109067000A (en) | One kind matching electricity consumption intelligence load monitoring system | |
| CN110430159A (en) | A kind of excessive method for early warning of Platform Server firewall policy range of opening | |
| CN113271303A (en) | Botnet detection method and system based on behavior similarity analysis | |
| Wang et al. | Honeynet construction based on intrusion detection | |
| CN103915897A (en) | Method and device for monitoring digital substation sampled values | |
| CN101719847A (en) | High-performance monitoring method for DNS traffic | |
| CN115549953B (en) | Network security alarm method and system | |
| CN101692303A (en) | Method for acquiring power utilization data of large customer power utilization management terminals | |
| CN114003896B (en) | Internet of things big data analysis processing device and method | |
| CN112769847B (en) | Safety protection method, device, equipment and storage medium for Internet of things equipment | |
| CN110941836A (en) | Distributed vertical crawler method and terminal equipment |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| WD01 | Invention patent application deemed withdrawn after publication | ||
| WD01 | Invention patent application deemed withdrawn after publication |
Application publication date: 20181218 |