CN112364173A - IP address mechanism tracing method based on knowledge graph - Google Patents

Abstract

The invention relates to the technical field of information, in particular to an IP address mechanism tracing method based on a knowledge graph. Aiming at the problems of isolation and dispersion of safety information in network space, the invention solves the problem that IP and domain name mapping in network space safety have no affiliated organization information from practical application, eliminates the gap of network safety information such as IP, domain name and organization, and the like, and collects more and more comprehensive safety information for network defense, monitoring and the like in network space safety; the candidate organization names with higher probability and higher possibility are effectively screened from the disordered search results, and the optimal inference result of the organization to which the candidate organization names belong is efficiently obtained through the IP address; the accuracy of the result of the organization to which the organization belongs is deduced from the IP address in the network space security field is ensured.

Description

An IP address organization traceability method based on knowledge graph

technical field

The invention relates to the technical field of network information, in particular to a method for tracing the source of an IP address organization based on a knowledge graph.

Background technique

In today's era, information technology is developing faster and faster, and the sources of threats and attack methods of network security are constantly changing. It is urgent for network security personnel to improve the perception ability of network security information, perceive security information in an all-round way, and discover and grasp more security information in time. Seize the opportunity in the cyberspace security offensive and defensive warfare, and achieve "know yourself and the enemy".

At present, in the field of network security, most network security information is isolated and scattered, which makes it difficult for security personnel to effectively use security information to solve practical problems.

The existing invention patents are related to the field of cyberspace security. The methods for solving practical security problems based on knowledge graphs include: a method for detecting malicious domain names based on knowledge graphs (publication number: CN110290116A), which realizes the detection of malicious domain names based on knowledge graphs, It provides a new perspective for malicious domain name detection, but it only considers the dimension of domain name information in cyberspace, and does not comprehensively consider the interrelationship between domain name information and IP addresses, network assets and other dimensions of security information. There is often a wealth of hidden information and knowledge between IP, domain name and other security information. Digging this information deeper can better solve practical security problems; a malicious domain name matching method based on DNS mapping IP (public number: CN108737385A), this scheme maps IPs corresponding to known malicious domain names in real time through DNS, and matches the access behavior of malicious domain names based on full IP traffic, but this scheme only uses DNS to resolve domain names, and conducts malicious domain name access behavior based on IP address traffic If it is necessary to solve other security problems based on the mapping relationship between IP and domain name, it is necessary to further excavate the hidden information between IP and domain name; a distributed security event correlation analysis method based on knowledge graph (public number: CN108270785A), this scheme A network security knowledge graph is constructed in the field of cyberspace security, including basic dimension, vulnerability dimension, threat dimension, alarm event dimension, and attack rule dimension, and the correlation analysis of security events is realized. This solution combines security information from various dimensions of cyberspace security resources. The correlation analysis of security threat events is realized from a macro perspective, but the solution to actual security problems is not involved. In practical applications, different algorithm processes need to be designed on the network security knowledge map to solve actual security problems.

It is not difficult to see that the current cyberspace security field has the following shortcomings:

(1) The IP address information in the cyberspace cannot directly map the organization to which it belongs;

(2) The search results are complex and disordered, and it is difficult to obtain the optimal inference results;

(3) The clustering results are imprecise, and there is a huge clustering error.

Therefore, there is no particularly practical and effective method in the field of existing network information security, which can analyze and obtain effective security information from the network, and then use and solve practical problems. Sex has not been effectively improved.

SUMMARY OF THE INVENTION

In order to overcome the defects of the prior art mentioned in the above content, the present invention provides a method for tracing the source of IP address institutions based on knowledge graph, aiming at starting from reality, using cyberspace to detect IP addresses, based on searching, clustering and The combination of knowledge graphs further infers the organizations they belong to, and mines the deep value information hidden in IP addresses, which is of great significance to the attack, defense, and monitoring of cyberspace security.

In order to achieve the above object, the technical scheme specifically adopted in the present invention is:

A method for tracing the source of IP address institutions based on knowledge graph, comprising:

Obtain domain name information: For the effective IP address of the institution to be inferred, the domain name information corresponding to the IP address is obtained through DNS reverse resolution;

Obtaining the key information of the domain name: truncate and filter the domain name information to obtain the key information in the domain name information;

Obtain analysis samples: search according to the key information of the domain name and obtain several webpages corresponding to the key information of the domain name, sort and filter the webpages and reserve several webpages as analysis samples;

Sample processing: analyze the analysis sample and obtain the text content present in it;

Entity extraction: entity extraction is performed on the text content through the named entity recognition model, and all the institution names in the text content are identified;

Calculate the weight: Calculate and configure the weight of the institution name according to the order of the analysis samples and the label element of the webpage label where the text content is located;

Entity clustering: Clustering between entities is performed through the edit distance between institution names, and at the same time, the prior knowledge graph is used to constrain and guide the clustering results, and multiple categories of institution name sets are obtained;

Selected institution name: According to the weight of each institution name, calculate the weight of each category of institution name set by weighted average calculation method, select the institution name set with the largest weight, and use the institution name set with the largest weight in the institution name set. The name is used as the traceability inference result.

The above-disclosed institutional traceability method can extract valid information from the acquired web page tags, web page texts, etc. for in-depth analysis through anti-resolution of IP addresses and reprocessing of acquired domain name information, so that possible targets can be filtered out. Institutions, after performing various calculations, select the target institution with the highest possibility as the inferred result to complete the traceability.

Further, the domain name information can be processed in a variety of ways, and a feasible and more optimized way is adopted here, which is as follows: in the step of obtaining the key information of the domain name, the key information of the domain name is processed through a regular expression, Remove the prefix and suffix of the domain name information to obtain key information in the domain name.

Further, the number of analysis samples obtained is usually large, and there is a lot of noise. For this purpose, some webpages with relatively large correlation can be selected as analysis samples, generally about 20. After the analysis samples are selected and analyzed. During processing, the web page tags are removed through regular expressions and web page document processing rules to obtain text content.

Further, all analysis samples have an impact on the final inference result, but the proportion of influence of each analysis sample is not exactly the same. In the process of inference, the influence of each analysis sample is divided. The following feasible methods are adopted: the weight calculation and configuration are performed on the name of the institution according to the order of the analysis samples, the weight is calculated according to the order of the webpage, and one of the weight factors is calculated as follows:

Among them, ω _i is the weight of the analysis samples ranked i.

Further, the weight of the analysis samples includes two weight factors, one of which is the weight of the web page itself in all the analysis samples, which is directly related to the ranking of its influence in all the analysis samples; the other weight factor is related to each analysis sample. The label elements contained in each web page are related. The weight of the organization name is calculated and configured according to the label element of the web page label where the text content is located. Semantic weight ω _j ∈ [0, 10], and reflects the importance of text content to search results by the semantic weight of the label element of the web page.

Further, according to the weight of the webpage and the semantic weight corresponding to the label element of the webpage, calculate the weight of the organization name according to the following method

N_i为网页i中搜索到的所有机构名称条目，n_ij为网页i中含有机构名称j的词项数目。Among them, W _ij is the weight of the organization name j in the web page i, ω _i is the weight factor of the web page i, ω _j is the semantic weight of the organization name j, and the element label is used to construct a tree diagram, h _j is the place where the organization name j is located. and h _j ∈ [0, 4], tf _ij is the term frequency of the organization name j in web page i, idf _j is the inverse document frequency of the name term j,

N _i is all the organization name items searched in web page i, and n _ij is the number of terms in web page i that contain organization name j.

Further, before performing entity clustering, at least find out the alias of the candidate institution name in the prior knowledge graph and the subordinate institution affiliation to form the candidate institution entity pair and form a triple, and take the head entity and The tail entities form the entity set {(h _i ,t _i )}.

Further, a path satisfying the two entities ( _{hi,t i )} in the triplet is selected in the knowledge graph, each path is used as a feature, and the feature value of the path is calculated to form the feature vector of the triplet.

Specifically, the eigenvalues of the path can be calculated as follows

表示节点e′在关系类型R_l下，随机游走到尾节点e的概率，此概率代表着是否存在实体对(e′,e)间R_l关系。In the formula, P=R ₁ ... R _l represents a path, s is the start node, e is the tail node, e' is the middle node, h _s,p' (e') represents that under the relation type R _l , (s, e') The probability that the entity pair can be connected by the path p'. in,

Represents the probability that node e' randomly walks to the tail node e under the relationship type R _l , and this probability represents whether there is a R _l relationship between entity pairs (e', e).

After realizing the probability calculation of each feature vector, use the feature vector to train the logistic regression classifier, and judge whether there is a specified type of relationship between the two entities according to the predicted probability of the path between the two entities. When the probability of path prediction is relatively high, it means that the probability of the existence of aliases, subordinate institutions, etc. between the two entities is relatively high, that is, the two entities should be put into the same institution name set as an entity pair; When the probability of path prediction is low, it means that the probability of the existence of a specified relationship such as an alias, a subordinate organization, etc. between the two entities is low, that is, the two entities should be placed in two different entity pairs.

Compared with the prior art, the present invention has the following beneficial effects:

(1) Aiming at the isolation and dispersion of security information in cyberspace, starting from practical applications, use valid IP addresses to reasonably infer its organization, solve the dilemma of IP and domain name mapping without organization information in cyberspace security, and eliminate The gap between network security information such as IP, domain name and organization, collect more and more comprehensive security information for network attack, defense and monitoring in cyberspace security.

(2) By setting a reasonable weight factor for the webpage and its label elements, scientifically calculate the weight of the target word in the network information, and effectively screen out the candidate organizations with greater probability and possibility from the chaotic and disordered search results. Name, efficiently obtain the optimal inference result of the organization to which it belongs through the IP address.

(3) Take advantage of the relational interconnection of the network security knowledge graph to improve the clustering accuracy of search results and ensure the accuracy of the results of inferring the organization to which it belongs from the IP address in the field of cyberspace security.

Description of drawings

In order to illustrate the technical solutions of the embodiments of the present invention more clearly, the following briefly introduces the accompanying drawings used in the embodiments. It should be understood that the following drawings only show some embodiments of the present invention, and therefore should not be It is regarded as a limitation of the scope. For those of ordinary skill in the art, other related drawings can also be obtained according to these drawings without any creative effort.

FIG. 1 is a schematic flowchart of the implementation of the present invention according to an embodiment.

Figure 2 is a tree diagram of web page label elements.

Detailed ways

The present invention will be further explained below with reference to the accompanying drawings and specific embodiments.

It should be noted here that the descriptions of these embodiments are used to help the understanding of the present invention, but do not constitute a limitation of the present invention. Specific structural and functional details disclosed herein are merely illustrative of example embodiments of the present invention. The present invention, however, may be embodied in many alternative forms and should not be construed as limited to the embodiments set forth herein.

Example

Aiming at many problems of information security in the cyberspace in the prior art, especially the situation that it is difficult to trace back to the organization to which the security information is isolated and scattered, this embodiment proposes a traceability method, which can be based on the hidden information contained in the IP address. Find the available information in and calculate and infer the most likely said institutions.

Specifically, as shown in FIG. 1 , the present embodiment discloses a method for tracing the source of an IP address organization based on a knowledge graph, including:

S01. Obtain domain name information: for the effective IP address of the institution to be inferred, obtain the domain name information corresponding to the IP address through DNS reverse resolution;

S02. Obtain key information of the domain name: truncate and filter the domain name information to obtain the key information in the domain name information;

S03. Obtain analysis samples: search according to the key information of the domain name and obtain several web pages corresponding to the key information of the domain name. The web pages can be crawled through a web crawler, and after sorting and screening the web pages, several web pages are reserved as analysis samples;

S04, sample processing: analyze the analysis sample and obtain the text content existing in it;

S05, entity extraction: perform entity extraction on the text content through the named entity recognition model, and identify all the institution names in the text content;

S06. Calculating weights: According to the order of analysis samples and the label elements of the webpage labels where the text content is located, weights are calculated and configured for the name of the institution;

S07. Entity clustering: Clustering between entities is performed through the edit distance between institution names, and at the same time, the prior knowledge graph is used to constrain and guide the clustering results, and a collection of institution names of multiple categories is obtained;

S08. Select an institution name: According to the weight of each institution name, calculate the weight of each category of institution name sets by the weighted average calculation method, select the institution name set with the largest weight, and use the institution name set with the largest weight in the set of institution names. The name of the institution is used as the traceability inference result.

The above-disclosed institutional traceability method can extract valid information from the acquired web page tags, web page texts, etc. for in-depth analysis through anti-resolution of IP addresses and reprocessing of acquired domain name information, so that possible targets can be filtered out. Institutions, after performing various calculations, select the target institution with the highest possibility as the inferred result to complete the traceability.

When the method is actually used, various methods can be used to process the domain name information. This embodiment adopts a feasible and more optimized method. The details are as follows: in the step of obtaining the key information of the domain name, the key information of the domain name is processed by a regular expression. The information is processed to remove the prefix and suffix of the domain name information, so as to obtain the key information in the domain name.

When using this method to obtain analysis samples, the number of obtained analysis samples is usually large, and there is a lot of noise. For this, some webpages with relatively high correlation can be selected as analysis samples, generally about 20. When processing samples and analyzing samples, regular expressions and web page document processing rules are used to remove web page tags and obtain text content.

After obtaining the samples, all the analysis samples have an impact on the final inference result, but the proportion of the influence of each analysis sample is not exactly the same. In the process of inference, the influence of each analysis sample is divided. The following feasible methods can be adopted: the weight calculation and configuration of the institution name according to the order of the analysis samples, the weight calculation according to the order of the web pages, and the calculation of one of the weight factors in the following way.

Among them, ω _i is the weight of the analysis samples ranked i. That is, the selected analysis samples are sorted, and the weight of the analysis samples will be affected according to the order of sorting. Generally, according to the relevance between the content of the web document and the search term, sort from front to back. If the entry comes from a well-known web page such as wiki or baidu, the order of the web document is appropriately advanced; the higher the web page is sorted, the more content the web document is. With analytical value, the greater the weight ω _i of the web page document.

In the method adopted in this embodiment, the weight of the analysis samples includes two weight factors, one of which is the weight of the webpage itself in all the analysis samples, which is directly related to the ranking of its influence in all the analysis samples. The content has been described in detail; another weight factor is related to the label element contained in each web page. According to the label element of the page label where the text content is located, the weight of the organization name is calculated and configured. The contribution of the topic of the web page document gives the corresponding semantic weight ω _j ∈ [0, 10] to the web page tag element, and reflects the importance of the text content to the search results with the semantic weight of the web page tag element.

The terms contained in the tag elements that contribute significantly to the document topic in the web page can better reflect the document topic of the web page. For example, the text content contained in the <title> tag element in the <head> tag of a web document can better reflect the topic of the document than the terms appearing in <h1> and <p> in <body>, while <h1> in <body> The terms contained in the > and <h2> tags contribute more to the topic of the document than the terms in the <p>, <li> and other tags. By analogy, according to the importance of the tag element to the document, a tree diagram of web page tag elements is constructed, as shown in Figure 2. According to the division of the webpage label element tree diagram, the corresponding semantic weights corresponding to different webpage label elements are divided as follows

According to the semi-structured definition format of the XML document, the HTML tag element is modeled as a tree, and each tag element is represented as a node. Up and down levels, for example, HTML documents contain head tag elements and body tag elements, but generally the text content contained in the title and meta elements in the head contributes a lot to the theme of the web page, so the body element is lowered by one level and the title in the head tag. Labels are at the same level. And so on.

In the actual calculation, if a certain pending organization name A appears in the <title> tag element of the head, then this web page is more likely to be the home page of the A organization, and the value of h _j is 2 at this time.

According to the weight of the webpage and the semantic weight corresponding to the label elements of the webpage, for the term related to the name of the institution searched in webpage i, it can be used according to its position in different label elements and the frequency and number of times it appears in the webpage, using the following formula to calculate the weight of institution name

N_i为网页i中搜索到的所有机构名称条目，n_ij为网页i中含有机构名称j的词项数目。Among them, W _ij is the weight of the organization name j in the web page i, ω _i is the weight factor of the web page i, ω _j is the semantic weight of the organization name j, h _j is the label element tree diagram where the organization name j is located and h _j ∈ [0, 4], tf _ij is the term frequency of institution name j in web page i, idf _j is the inverse document frequency of name term j,

N _i is all the organization name items searched in web page i, and n _ij is the number of terms in web page i that contain organization name j.

The above disclosure includes the steps of entity clustering to help accurately determine the organization to which the security information belongs. Before performing entity clustering, at least find out the alias of the name of the candidate organization in the prior knowledge graph and the affiliation of the superior and inferior organizations to form the candidate organization. Organization entities are paired to form a triple, and the head entity and tail entity in the triple are taken to form an entity set {(h _i ,t _i )}. In this embodiment, the Path Ranking Algorithm (PRA algorithm) is used to search for entities with specified relationships. The main idea of the PRA algorithm is to judge the existence of a certain type of relationship through the different paths of connections between entities in the knowledge graph.

Specifically, a path that satisfies the two entities ( _{hi, t i )} in the triplet is selected in the knowledge graph, each path is used as a feature, and the feature value of the path is calculated to form the feature vector of the triplet.

Specifically, the eigenvalues of the path can be calculated as follows

表示节点e′在关系类型R_l下，随机游走到尾节点e的概率，此概率代表着是否存在实体对(e′,e)间R_l关系。In the formula, P=R ₁ ... R _l represents a path, s is the start node, e is the tail node, e' is the middle node, h _s,p' (e') represents that under the relation type R _l , (s, e') The probability that the entity pair can be connected by the path p'. in,

Represents the probability that node e' randomly walks to the tail node e under the relationship type R _l , and this probability represents whether there is a R _l relationship between entity pairs (e', e).

After realizing the probability calculation of each feature vector, use the feature vector to train the logistic regression classifier, and judge whether there is a specified type of relationship between the two entities according to the predicted probability of the path between the two entities. When the probability of path prediction is relatively high, it means that the probability of the existence of aliases, subordinate institutions, etc. between the two entities is relatively high, that is, the two entities should be put into the same institution name set as an entity pair; When the probability of path prediction is low, it means that the probability of the existence of a specified relationship such as an alias, a subordinate organization, etc. between the two entities is low, that is, the two entities should be placed in two different entity pairs.

Integrating the prior knowledge of network security knowledge graph to infer the similarity between candidate organizational entities is to avoid other similar candidate organizational entities with different aliases or subordinate relationships from being incorrectly clustered into different candidate organizational entities. category. For example, both the NBA and the American Basketball Association represent the same organization. If clustering is performed according to the edit distance of the organization name, the NBA and the American Basketball Association will be clustered into different categories, making the results ambiguous. At this time, the PRA algorithm is used to infer from the prior network security knowledge map that the NBA and the American Basketball Association have an alias relationship, which has a large similarity. As prior knowledge, it guides the clustering process and makes it cluster into the same category. .

After the seventh step of entity clustering, a set of candidate organization names of multiple categories is obtained, and each set contains multiple candidate organization names. The output result needs to take out the most reliable and most probable term. Organization name as final output. After entity clustering, the weight factor of the candidate organization name calculated in the sixth step is retained, and the weight of each category set is obtained by weighted average according to the weight of each candidate organization name. Select the set with the largest weight, and output the name of the organization with the largest weight in the set as the final inference result.

The above are the listed embodiments of the present invention, but the present invention is not limited to the above-mentioned optional embodiments. Those skilled in the art can arbitrarily combine the above-mentioned methods to obtain other various embodiments. Various other forms of implementation can be derived. The above specific embodiments should not be construed as limiting the protection scope of the present invention, which should be defined in the claims, and the description can be used to interpret the claims.

Claims (10)

1. an IP address mechanism tracing method based on knowledge graph, is characterized in that, comprises: Obtain domain name information: For the effective IP address of the institution to be inferred, the domain name information corresponding to the IP address is obtained through DNS reverse resolution; Obtaining the key information of the domain name: truncate and filter the domain name information to obtain the key information in the domain name information; Obtain analysis samples: search according to the key information of the domain name and obtain several webpages corresponding to the key information of the domain name, sort and filter the webpages and reserve several webpages as analysis samples; Sample processing: analyze the analysis sample and obtain the text content present in it; Entity extraction: entity extraction is performed on the text content through the named entity recognition model, and all the institution names in the text content are identified; Calculate the weight: Calculate and configure the weight of the institution name according to the order of the analysis samples and the label element of the webpage label where the text content is located; Entity clustering: Clustering between entities is performed through the edit distance between institution names, and at the same time, the prior knowledge graph is used to constrain and guide the clustering results, and multiple categories of institution name sets are obtained; Selected institution name: According to the weight of each institution name, calculate the weight of each category of institution name set by weighted average calculation method, select the institution name set with the largest weight, and use the institution name set with the largest weight in the institution name set. The name is used as the traceability inference result. 2. the IP address mechanism tracing method based on knowledge graph according to claim 1, is characterized in that: in the step of obtaining domain name key information, the domain name key information is processed by regular expression, and the prefix and suffix of domain name information are removed , to obtain key information in the domain name. 3. The IP address organization tracing method based on knowledge graph according to claim 1, characterized in that: when the analysis sample is processed, through regular expressions and webpage document processing rules, the webpage label is removed, and the text content is obtained. 4. the IP address institution traceability method based on knowledge graph according to claim 1, the described order according to the analysis sample carries out the weight calculation and configuration to the institution name successively, it is characterized in that: according to the order of the web page to calculate the weight, according to the following way to calculate one of the weighting factors,

Among them, ω _i is the weight of the analysis samples ranked i. 5. the IP address organization tracing method based on knowledge graph according to claim 4, described according to the label element of the web page label where the text content is located, weight calculation and configuration are carried out to the organization name, it is characterized in that: according to the web page label element to. The contribution of the topic of the web page document gives the corresponding semantic weight ω _j ∈ [0, 10] to the web page tag element, and reflects the importance of the text content to the search results with the semantic weight of the web page tag element. 6. the IP address institution tracing method based on knowledge graph according to claim 5, is characterized in that: according to the weight of webpage and the corresponding semantic weight of webpage label element, calculate the weight of institution name according to the following method

Among them, W _ij is the weight of the organization name j in the web page i, ω _i is the weight factor of the web page i, ω _j is the semantic weight of the organization name j, and the element label is used to construct a tree diagram, h _j is the place where the organization name j is located. and h _j ∈ [0, 4], tf _ij is the term frequency of the organization name j in web page i, idf _j is the inverse document frequency of the name term j,

N _i is all the organization name items searched in web page i, and n _ij is the number of terms in web page i that contain organization name j. 7. The IP address organization tracing method based on knowledge graph according to claim 1, it is characterized in that: before carrying out entity clustering, at least find out the alias of candidate organization name in the prior knowledge graph and the affiliation of superior and inferior organizations to A candidate institution entity pair is formed and a triplet is formed, and the head entity and the tail entity in the triplet are taken to form an entity set {(h _i ,t _i )}. 8. the IP address mechanism traceability method based on knowledge graph according to claim 7, is characterized in that: in knowledge graph, choose the path that satisfies two entities (h _i , t _i ) in triplet, each path As a feature and calculate the eigenvalues of the path to form the eigenvectors of the triples. 9. the IP address mechanism tracing method based on knowledge graph according to claim 8, is characterized in that: calculate the characteristic value of path according to following method

In the formula, P=R ₁ ... R _l represents a path, s is the start node, e is the tail node, e' is the middle node, h _s,p' (e') represents that under the relation type R _l , (s, e') The probability that the entity pair can be connected by the path p'. in,

Represents the probability that node e' randomly walks to the tail node e under the relationship type R _l , and this probability represents whether there is a R _l relationship between entity pairs (e', e). 10. The IP address mechanism tracing method based on knowledge graph according to claim 8, it is characterized in that: use feature vector training logistic regression classifier, according to the probability of path prediction between two entities, to judge the relationship between two entities. Whether there is a relationship of the specified type between them.

Images