CN115632803A - Multi-dimensional Internet sensitive information detection method, system and equipment - Google Patents

Abstract

This application discloses a multi-dimensional Internet sensitive information detection method, system and equipment, which relate to the technical field of network information security. Information detection tasks are set according to sensitive information identification, and different identifications of different types of sensitive information are different; the information The detection task sets the detection frequency and allocates detection channels; performs task detection according to the detection frequency and detection channels to obtain the first detection data; screens the first detection data to obtain the second detection data, and the second detection The data is used to assess the security of sensitive information. Use the identification of sensitive information to detect sensitive information through multiple detection channels, and analyze the detected data to ensure the accuracy of the detected information. Finally, the target correlation is obtained by comparing the detected data with the original sensitive information identification, and then the security degree of sensitive information is evaluated, which improves the accuracy of external investigation of sensitive information compared with traditional methods.

Description

A multi-dimensional Internet sensitive information detection method, system and equipment

technical field

This application relates to the technical field of network information security, in particular to a multi-dimensional Internet sensitive information detection method, system and equipment.

Background technique

In the information age, in order to avoid frequent incidents of corporate sensitive information leakage, the state has successively promulgated data security laws and network security laws to strengthen management, further urging enterprises to increase security investment to prevent such risks. However, under the complex external attack situation, hackers steal sensitive data of enterprises and institutions and distribute and sell it to the network. Coupled with the poor internal management of the enterprise and the lack of security awareness of employees, uploading sensitive files and project core codes to the Internet can easily cause uncontrollable data leakage, and large-scale ones may also have a major security impact on the country.

Therefore, in today's security situation, how to avoid all kinds of losses caused by the leakage of sensitive information as much as possible has become an important part of enterprise security construction. In traditional technology, enterprises generally adopt internal defense and external investigation in order to prevent the leakage of sensitive information. Internal defense is to increase the strength of data security and try to avoid the leakage of sensitive data. External inspection is to screen the data disclosed to the outside world to determine whether the company's sensitive data has been leaked.

However, most of the external inspection methods in traditional technologies are to monitor designated network locations, and the channels are often single, so it is impossible to accurately screen external information.

Contents of the invention

In order to solve the above technical problems, the application proposes the following technical solutions:

In the first aspect, the embodiment of the present application provides a multi-dimensional Internet sensitive information detection method, the method includes: setting information detection tasks according to sensitive information identifiers, and different identifiers of different types of sensitive information are different; The detection task sets the detection frequency and allocates the detection channels; according to the detection frequency and detection channels, the task detection is carried out to obtain the first detection data; the first detection data is screened to obtain the second detection data, and the second detection The data is used to assess the security of sensitive information.

Using the above implementation method, sensitive information is detected through multiple detection channels by using the identification of sensitive information, and the detected data is analyzed to ensure the accuracy of the detected information. Finally, the target correlation is obtained by comparing the detected data with the original sensitive information, and then the security degree of sensitive information is evaluated. Compared with the traditional method, the accuracy of external investigation of sensitive information is improved.

With reference to the first aspect, in the first possible implementation manner of the first aspect, the setting of the information detection task according to the sensitive information identification includes: analyzing the sensitive information to determine the information identification of the sensitive information; The information identification determines the detection list of the sensitive information; and assigns a target ID to the detection list to establish a target task entry.

With reference to the first possible implementation of the first aspect, in the second possible implementation of the first aspect, the task detection and acquisition of the first detection data according to the detection frequency and detection channels includes: according to the target ID Generate a task ID associated with the target ID; establish an independent data table for each of the detection channels in the database, and use the same task ID for the data tables of the detection channels determined by the same detection task; Perform information detection according to the detection frequency according to the information identification, and store the detected information in the corresponding data table; after the detection is completed, read the data table with the same task ID in the database to obtain the temporary detection result of the same detection task.

With reference to the second possible implementation manner of the first aspect, in the third possible implementation manner of the first aspect, performing information detection according to the detection frequency according to the information identifier through the detection channel includes: Perform a preliminary sensitive information detection test on the first detection channel, the first detection channel is any detection channel in the same detection task; obtain the feedback response of the first detection channel; if the feedback response is to limit anti-crawling , the IP proxy pool is invoked to detect sensitive information through the first detection channel.

In combination with the third possible implementation of the first aspect, in the fourth possible implementation of the first aspect, calling the IP proxy pool to detect sensitive information through the first detection channel includes: The IP agent carries out state determination; If the first IP agent is an expired agent, then the first IP agent is removed from the IP agent pool; or, if the first IP agent is an invalid IP agent of the first detection channel, Then mark the first IP proxy, and screen the first IP proxy through other detection channels.

With reference to the fourth possible implementation of the first aspect, in the fifth possible implementation of the first aspect, the screening of the first IP proxy through other detection channels includes: calling the first IP The agent detects sensitive information through a second detection channel, and the second detection channel is any detection channel except the first detection channel; if the second detection channel can use the first IP agent to detect sensitive information detection, then retain the first IP proxy, otherwise, remove it from the IP proxy pool.

With reference to any of the second to fifth possible implementation manners of the first aspect, in a sixth possible implementation manner of the first aspect, the screening of the first detection data to obtain the second detection data includes: The temporary detection results are obtained by deduplicating the unique fields used by different detection channels to obtain the first data; the preset filter items filter the interference content of the first data to obtain the second data; according to the information of the detection task identifying and determining the target correlation of the second data; and scoring the second data according to the determined target correlation degree to obtain the second detection data.

In combination with the sixth possible implementation of the first aspect, in the seventh possible implementation of the first aspect, the detection data is selected from the second detection data according to the preset score interval and compared with the original sensitive information identifier, and the Sensitive information alerts are issued for comparison results.

In the second aspect, the embodiment of the present application provides a multi-dimensional Internet sensitive information detection system, the system includes: a task setting module, configured to set information detection tasks according to sensitive information identification, different identifications of different types of sensitive information Different; the detection channel allocation module is used to set the detection frequency of the information detection task and allocate detection channels; the information detection module is used to perform task detection according to the detection frequency and detection channels to obtain the first detection data; data processing A module, configured to screen the first detection data to obtain second detection data, where the second detection data is used to evaluate the security of sensitive information.

In the third aspect, the embodiment of the present application provides a device, including: a memory, a processor, and a computer program stored in the memory. When detecting sensitive information, the processor reads the computer program and executes the first aspect Or in the multi-dimensional Internet sensitive information detection method described in any possible implementation manner of the first aspect, the sensitive information detection is performed according to the information detection task.

Description of drawings

FIG. 1 is a schematic flow diagram of a multi-dimensional Internet sensitive information detection method provided by an embodiment of the present application;

FIG. 2 is a schematic diagram of a multi-dimensional Internet sensitive information detection system provided by an embodiment of the present application;

Fig. 3 is a schematic diagram of a device provided by an embodiment of the present application.

Detailed ways

The scheme will be described below in conjunction with the accompanying drawings and specific implementation methods.

Fig. 1 is a schematic flow diagram of a multi-dimensional Internet sensitive information detection method provided by the embodiment of the present application. Referring to Fig. 1, the multi-dimensional Internet sensitive information detection method in this embodiment includes:

S101. Set an information detection task according to sensitive information identifiers, where different identifiers of different types of sensitive information are different.

In this embodiment, firstly, the sensitive information is analyzed to determine the information identification of the sensitive information; the detection list of the sensitive information is determined according to the information identification; a target ID is assigned to the detection list to establish a target task entry.

Sensitive information to be detected at one time can be one piece or multiple pieces, and the types of sensitive information will also be different. For example, if some sensitive information has text content, its information identification is generally a keyword. If it is a source code package, the information identifier is the source code package name. There is also a website link within the enterprise, and if it is not disclosed to the outside world, its information is identified as a domain name. The above are only simple examples of several types, and the content of sensitive information may exist in other forms in specific situations.

In the embodiment of the present application, original sensitive information identifiers such as keywords and domain names of target units are provided, corresponding target IDs are generated in the database client table, and target entries are established. The fields that such a target entry should contain include target name, random number Add a time stamp to combine the target ID generated by the md5 algorithm (the format is a 32-bit string of mixed letters and numbers), the original sensitive information identification, and the list of executable tasks.

If keywords, source code package names, and domain names are taken as examples, the keywords, source code package names, and domain names corresponding to sensitive information will be parsed from different sensitive information according to the type of sensitive information. Then build a detection list, fill in the corresponding keywords, source code package name and domain name information in column or row units, and finally assign a target ID to each row or column. It should be pointed out that the information identification of sensitive information is not necessarily any one of a single keyword, source code package name or domain name, but may also be any combination of them. For example, the information identification of a sensitive information is a keyword and a domain name .

S102. Set a detection frequency for the information detection task and allocate detection channels.

The detection channels in the implementation of this application can use Internet search engines (Baidu search engine, Google search engine, fofa cyberspace search engine), library, cloud network disk, WeChat official account, mailbox, sub-domain name, source code hosting platform, third party Various monitoring channels such as cyberspace search engines and dark web resource sites. For different types of sensitive information, determine the scope of its possible leakage. For example, there is a high probability that source code packages will exist in cloud storage, source code hosting platforms, third-party cyberspace search engines, and dark web resource sites, but it is less likely to exist in archives. However, for text and website links, the possibility of leakage in the library is relatively high, so it is particularly important to determine the detection channels before detecting different types of sensitive information.

Of course, selective detection can also be carried out for detection channels that cannot exist, but that will cause unnecessary waste of detection cost and time. And when choosing a detection channel, if you are not sure about the detection channel, give priority to it to ensure a more comprehensive and accurate detection.

Set different detection frequencies for detection tasks, and set different detection frequencies according to specific needs. You can set one-time specified time execution tasks, interval time execution tasks (can be intervals of years/months/hours/minutes/seconds), and timing cycles Task execution (such as monthly/daily/a certain day of the month/a certain time of the day/even a certain minute and a second every hour). The setting of the specific detection frequency is also set according to the importance of sensitive information and the time point when it may be leaked.

S103. Perform mission detection according to the detection frequency and detection channels to obtain first detection data.

In this embodiment, the task ID associated with the target ID is generated according to the target ID; an independent data table is established for each of the detection channels in the database, and the data tables of the detection channels determined by the same detection task use the same The task ID. Carry out information detection according to the detection frequency according to the information identification through the detection channel, and store the detected information in the corresponding data table; after the detection is completed, read the data table with the same task ID in the database to obtain the same detection task provisional detection results.

When detecting sensitive information, the monitoring function modules of different channels are successively activated according to the detection channels preset by the detection task. For some detection channels, it may be necessary to use the ip proxy pool due to the limitation of request frequency and anti-climbing. Therefore, in this embodiment, performing information detection according to the detection frequency through the detection channel according to the information identifier specifically includes:

A preliminary sensitive information detection test is performed on the first detection channel according to the detection frequency, and the first detection channel is any detection channel in the same detection task. Obtain a feedback response from the first detection channel; if the feedback response is to limit anti-crawling, call the IP proxy pool to detect sensitive information through the first detection channel.

In this embodiment, in order to maintain the normality of the IP proxy pool, another independent process needs to be started to maintain the IP proxy pool, because the IP proxy does not need to be persistent, and it is stored in the redis database to facilitate multiple system calls, and it is necessary to continuously monitor the proxy status , Eliminate invalid agents and expired agents in time to ensure the availability of functions.

In an exemplary embodiment, before invoking the IP proxy pool to detect sensitive information through the first detection channel, the status of the IP proxy in the IP proxy pool is determined first. If the first IP proxy is an expired proxy, then remove the first IP proxy from the IP proxy pool. If the first IP proxy is an invalid IP proxy of the first detection channel, mark the first IP proxy, and screen the first IP proxy through other detection channels.

The main reason why the invalid IP agents determined through the first detection channel are directly eliminated here is that in the process of using these IP agents, there will often be agents that are invalid for a certain detection channel due to different detection channels. If another detection channel is still available, such an IP agent will only be marked as unavailable in the specific channel agent combination of the ip agent pool, and will not be directly removed from the entire agent pool, and will be completely deleted after it expires. Thereby maximizing the use of agency resources and reducing operating costs.

Therefore, further in this embodiment, it will be further verified whether the first IP proxy is a real invalid proxy. Specifically, the first IP agent is invoked to detect sensitive information through a second detection channel, and the second detection channel is any detection channel other than the first detection channel. If the second detection channel can use the first IP proxy to detect sensitive information, keep the first IP proxy; otherwise, remove it from the IP proxy pool.

S104. Screen the first detection data to obtain second detection data, where the second detection data is used to evaluate the security of sensitive information.

In this embodiment, the temporary detection results obtained through S103 are all stored in the data table marked with the task ID, and the temporary detection results are retrieved from the data table of the database, and the suspicious and sensitive information detected by all channels is cleaned and analyzed.

According to the unique field used by different detection channels, for example, in the channel of search engine, the URL (web address) is unique, and in the channel of cloud network disk, the title can be unique. The detected information inevitably has interference content, and the filter items can be preset to filter the interference part as much as possible according to the uniqueness field in the previous item. The correlation between the retrieved information and the target is different. Some channels, such as source code hosting platforms, will provide this correlation coefficient, but in more cases, you need to handle it yourself. A higher score should be given.

Since some of the above-mentioned processed data may be fuzzy matched, in order to further improve the accuracy of the data, in this embodiment, the detection data is further selected from the second detection data according to the preset score range and compared with the original sensitive information identification. Yes, after the analysis is completed, the results will be archived in the database again, and on this basis, the manual judgment of security experts will be used to provide as many sensitive information alerts as possible.

Corresponding to the multi-dimensional Internet sensitive information detection method provided in the foregoing embodiments, the present application also provides an embodiment of a multi-dimensional Internet sensitive information detection system.

Referring to FIG. 2 , the multi-dimensional Internet sensitive information detection system 20 provided by the embodiment of the present application includes: a task setting module 201 , a detection channel allocation module 202 , an information detection module 203 and a data processing module 204 .

The task setting module 201 is configured to set information detection tasks according to sensitive information identifiers, and different identifiers of different types of sensitive information are different. The detection channel allocation module 202 is configured to set the detection frequency for the information detection task and allocate detection channels. The information detection module 203 is configured to perform task detection according to the detection frequency and detection channel to obtain first detection data. The data processing module 204 is configured to screen the first detection data to obtain second detection data, and the second detection data is used to evaluate the security degree of sensitive information.

The present application also provides a device. Referring to FIG. 3 , the device 30 in this embodiment includes: a processor 301 , a memory 302 and a communication interface 303 .

In FIG. 3 , the processor 301 , the memory 302 and the communication interface 303 can be connected to each other through a bus; the bus can be divided into an address bus, a data bus, a control bus, and the like. For ease of representation, only one thick line is used in FIG. 3 , but it does not mean that there is only one bus or one type of bus.

The processor 301 usually controls the overall functions of the device 30, for example, when the device is started, and when sensitive information data to be detected is received after the device is started, the information detection task is set according to the sensitive information identification, and different identifications of different types of sensitive information are different. . The detection frequency is set for the information detection task and detection channels are allocated. The task detection is performed according to the detection frequency and the detection channel to obtain the first detection data. The first detection data is screened to obtain second detection data, and the second detection data is used to evaluate the security degree of sensitive information.

Also, the processor 301 may be a microprocessor (MCU). Processors may also include hardware chips. The aforementioned hardware chip may be an Application Specific Integrated Circuit (ASIC), a Programmable Logic Device (PLD) or a combination thereof. The above-mentioned PLD may be a complex programmable logic device (CPLD), a field programmable logic gate array (FPGA) or the like.

Memory 302 is configured to store computer-executable instructions to support the operation of device 30 data. The memory 301 can be implemented by any type of volatile or non-volatile storage devices or their combination, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable Programmable Read Only Memory (EPROM), Programmable Read Only Memory (PROM), Read Only Memory (ROM), Magnetic Memory, Flash Memory, Magnetic Disk or Optical Disk.

The communication interface 303 is used for the device 30 to transmit data, for example, to communicate with the database of the enterprise. The communication interface 303 includes a wired communication interface, and may also include a wireless communication interface. Wherein, the wired communication interface includes a USB interface, a MicroUSB interface, and may also include an Ethernet interface. The wireless communication interface may be a WLAN interface, a cellular network communication interface or a combination thereof.

In an exemplary embodiment, the device 30 provided in the embodiment of the present application further includes a power supply component, which provides power for various components of the device 30 . Power components may include a power management system, one or more power supplies, and other components associated with generating, managing, and distributing power to device 30 .

A communication component, the communication component is configured to facilitate wired or wireless communication between the device 30 and an external device. The device 30 can access wireless networks based on communication standards, such as WiFi, 4G or 5G, or a combination thereof. The communication component receives broadcast signals or broadcast related information from an external broadcast management system via a broadcast channel. The communication component also includes a Near Field Communication (NFC) module to facilitate short-range communication. For example, the NFC module can be implemented based on Radio Frequency Identification (RFID) technology, Infrared Data Association (IrDA) technology, Ultra Wideband (UWB) technology, Bluetooth (BT) technology and other technologies.

In an exemplary embodiment, device 30 may be implemented by one or more application specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field programmable Programmable gate array (FPGA), terminal, microterminal, processor or other electronic component implementation.

For the same and similar parts among the various embodiments in the specification of the present application, please refer to each other. In particular, for the system and device embodiments, since the methods therein are basically similar to the method embodiments, the description is relatively simple, and for relevant parts, refer to the description in the method embodiments.

It should be noted that in this article, relative terms such as "first" and "second" are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply these No such actual relationship or order exists between entities or operations. Furthermore, the term "comprises", "comprises" or any other variation thereof is intended to cover a non-exclusive inclusion such that a process, method, article or apparatus comprising a set of elements includes not only those elements, but also includes elements not expressly listed. other elements of or also include elements inherent in such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising a ..." does not exclude the presence of additional identical elements in the process, method, article or apparatus comprising said element.

Claims (10)

1. A multi-dimensional Internet sensitive information detection method is characterized by comprising the following steps:

setting an information detection task according to a sensitive information identifier, wherein different identifiers of different sensitive information types are different;

setting detection frequency of the information detection task and distributing detection channels;

performing task detection according to the detection frequency and the detection channel to obtain first detection data;

and screening the first detection data to obtain second detection data, wherein the second detection data is used for evaluating the security degree of sensitive information.

2. The method for detecting the sensitive information of the internet in the multi-dimension according to claim 1, wherein the setting of the information detection task according to the sensitive information identification comprises:

analyzing the sensitive information to determine an information identifier of the sensitive information;

determining a detection list of the sensitive information according to the information identifier;

and allocating target IDs to the detection lists to establish target task entries.

3. The method for detecting the internet sensitive information in multiple dimensions according to claim 2, wherein the task detection according to the detection frequency and the detection channel to obtain the first detection data comprises:

generating a task ID associated with the target ID according to the target ID;

establishing an independent data table for each detection channel in a database, wherein the data tables of the detection channels determined by the same detection task use the same task ID;

detecting information according to the detection frequency through the detection channel according to the information identification, and storing the detected information into a corresponding data table;

and after the detection is finished, reading the data tables with the same task ID in the database to obtain a temporary detection result of the same detection task.

4. The method for detecting the internet sensitive information in multiple dimensions according to claim 3, wherein the step of detecting the information according to the detection frequency through the detection channel according to the information identifier comprises the following steps:

performing preliminary sensitive information detection test on a first detection channel according to the detection frequency, wherein the first detection channel is any one detection channel in the same detection task;

acquiring a feedback response of the first detection channel;

and if the feedback response is limited to reverse crawling, calling an IP proxy pool to detect the sensitive information through the first detection channel.

5. The method for detecting the internet sensitive information in the multi-dimension according to claim 4, wherein invoking an IP proxy pool to perform the sensitive information detection through the first detection channel comprises:

determining the state of the IP agents in the IP agent pool;

if the first IP agent is an expired agent, removing the first IP agent from the IP agent pool;

or,

and if the first IP agent is an invalid IP agent of the first detection channel, marking the first IP agent, and screening the first IP agent through other detection channels.

6. The method as claimed in claim 5, wherein said screening the first IP agent through other detection channels comprises:

calling the first IP agent to detect the sensitive information through a second detection channel, wherein the second detection channel is any detection channel except the first detection channel;

if the second detection channel can utilize the first IP agent to detect the sensitive information, the first IP agent is reserved, otherwise, the second IP agent is removed from the IP agent pool.

7. The method for detecting internet sensitive information in multiple dimensions according to any one of claims 3 to 6, wherein the screening the first detection data to obtain the second detection data comprises:

the temporary detection result is subjected to de-duplication according to the unique fields used by different detection channels to obtain first data;

presetting a filtering item to filter the interference content of the first data to obtain second data;

determining the target relevance of the second data according to the information identification of the detection task;

and scoring the second data according to the determined target relevance degree to obtain the second detection data.

8. The method as claimed in claim 7, wherein the second detection data is selected from the second detection data according to a preset score interval to compare with the original sensitive information identifier, and the sensitive information is alarmed according to the comparison result.

9. A multidimensional internet-sensitive information detection system, the system comprising:

the task setting module is used for setting information detection tasks according to sensitive information identifiers, wherein different identifiers of different sensitive information types are different;

the detection channel distribution module is used for setting detection frequency of the information detection task and distributing detection channels;

the information detection module is used for carrying out task detection according to the detection frequency and the detection channel to obtain first detection data;

and the data processing module is used for screening the first detection data to obtain second detection data, and the second detection data is used for evaluating the security degree of the sensitive information.

10. An apparatus, comprising: the device comprises a memory, a processor and a computer program stored on the memory, and is characterized in that when sensitive information detection is carried out, the processor reads the computer program, executes the multi-dimensional internet sensitive information detection method according to any one of claims 1 to 8, and carries out sensitive information detection according to an information detection task.

Images