CN111506706A - Relationship similarity based upper and lower meaning relationship forest construction method - Google Patents

Abstract

The invention relates to a method for constructing an upper-lower-sense relational forest based on relational similarity, and belongs to the fields of natural language processing and knowledge graphs. The method includes: inputting all triples, obtaining the entity pair probability set C of relations in all triples through a traditional multilayer perceptron, making a Cartesian for C, and calculating the similarity of the Cartesian product C×C with the formula The measurement value (matrix S) is filtered with the threshold value ThresholdA to filter out the relation similarity set; then combined with the open domain entity data set, the trained similarity relation is used to predict the relation correspondence in the open domain entity on the improved multi-layer perceptual model The number of entities is filtered by the threshold ThresholdB to obtain the Result matrix; finally, the relationship forest is constructed according to the Result matrix and the relationship similarity set. The invention can retrieve similar relationship entities in the knowledge map, and improve the accuracy of similar relationship extraction in the relationship extraction task.

Description

A method of constructing upper and lower relational forests based on relational similarity

technical field

The invention belongs to the field of natural language processing and knowledge graph, and relates to a method for constructing an upper-lower-sense relational forest based on relational similarity.

Background technique

At present, natural language processing and knowledge graphs in academia and industry are a hot research field, and the research on entity-to-relationship in natural language processing has made great progress.

Existing research on relational similarity mainly focuses on the following methods:

In 2006, Peter D.Turney proposed the concept of relational similarity in the article Similarity of Semantic Relation. As a pioneer of relational similarity research, Turney believes that measuring relational similarity is to measure the similarity of entity pairs corresponding to the relation, such as: mason and stone's The relationship (referred to as Relation A here), the relationship between carpenter and wood (referred to as Relation B here), the similarity of the relationship between A and B is based on their corresponding entity pair (mason: stone) and entity pair (carpenter: wood) to measure.

In 2013, Alisa Zhila proposed a method for measuring relational similarity at the NAACL-HLT conference, and Alisa's method for measuring relational similarity using the vector method in Combining Heterogeneous Models for Measuring Relational Similarity achieved a milestone in this research. meaning. In this paper, it is proposed to use the RNN recurrent neural network language model to learn the relationship word vector. The source corpus is composed of various entities corresponding to the relationship, and the entity pair is input into the RNN recurrent network to obtain an entity pair embedding matrix. The dimension of the matrix is: number of entities × dimension , in the experiment of this article, the dimension parameter is set to 1600, assuming that the entity pair Wi = (W _i1 , _Wi2 ), W _j = (W _j1 , W _j2 ) The corresponding word embeddings are (v _i1 , v _i2 ) and ₍ v _j1 , v _j2 ), then the direction vector corresponding to Wi _i is v _i =v _i2 -v _i1 , the direction vector corresponding to W _j is v _j =v _j2 -v _j1 , and the similarity measurement formula is:

In 2019, Weize Chen of Tsinghua University published a paper Quantifying Similarity between Relations with Fact Distribution at the ACL conference. In this paper, it is proposed to use the multilayer perceptron to find the relationship probability corresponding to the entity pair, that is, the probability of the head and tail entities corresponding to the relationship 1 is P _θ1 = ( h,t|r ₁ ), the head and tail entity probability corresponding to relation 2 is P _θ2 =(h,t|r ₂ ), then the similarity between relation 1 and relation 2 is measured by KL divergence:

After finding the KL divergence, use the function g(x) to find the similarity in both directions to obtain the similarity value S(r ₁ , r ₂ ) of the final relation 1 and relation 2:

S(r ₁ ,r ₂ )=g(D _KL (P _θ1 (h,t|r ₁ )||(P _θ2 (h,t|r ₂ )),D _KL ((P _θ1 (h,t| r ₂ )||(P _θ2 (h,t|r ₁ ))) (3)

It can be seen from the above solutions that relational similarity is currently widely used in relation extraction tasks. The current technology is to directly apply relational similarity to tasks, such as: using relational similarity to deduplicate relationships and build knowledge map etc. However, such applications lack application structure, and direct application to the corresponding tasks will lead to insignificant effects, or the relational similarity technology cannot be effectively applied according to the corresponding tasks.

SUMMARY OF THE INVENTION

In view of this, the purpose of the present invention is to provide a method for constructing a relational forest of upper and lower sense based on the relational similarity, and constructs the relational forest according to the relational similarity and certain rules. After building the relational forest, each task retrieves the relational forest to obtain the corresponding relational objects when applying relational similarity technology. The relational similarity technology is decoupled from its downstream tasks, and the technology is highly cohesive. ① This alleviates the problem of low accuracy of relation extraction in the current relation extraction task due to the similarity of relations. ②In the construction of knowledge graph, the constructed relation forest can be used to retrieve entities with similar relations. (3) In the relational reasoning task, the relational forest is used to obtain similar relations to reason to obtain the next object.

To achieve the above object, the present invention provides the following technical solutions:

A method for constructing upper and lower relational forests based on relational similarity, comprising the following steps:

S1: Input all triples, and obtain the entity pair probability set C=(P _θ1 , P _θ2 , P _θ3 ,..., P _θn ) of relations in all triples through traditional multilayer perceptron, and n is the number of relations;

S2: Do the Cartesian product of the set C with itself to obtain a set of size n×n, delete the same pair of two elements in the Cartesian product, and leave n ² -n pairs of elements, and then calculate each The relationship similarity measure value S(r _i ,r _j ) for the elements, where i=1,2,...,n, j=1,2,...,n;

S3: Set the threshold value ThresholdA, and filter out the relationship metric value S(r _i , r _j ) that is greater than or equal to ThresholdA as the similar relationship set F(r _i , r _j );

S4: Improve the multi-layer perceptron model, respectively use the relation word vector or the entity word vector as the input of the improved multi-layer perceptron model, and the output is a Tmp_rel matrix or a Tmp_ent matrix;

S5: Multiply the output result after inputting the relation word vector and the entity word vector into the multi-layer perceptron, that is, Tmp_rel*Tmp_ent to obtain the Result_tmp matrix, and the dimension is the number of relations × the number of entities;

S6: Calculate Softmax for each row of the Result_tmp matrix, indicating that a probability is obtained for the entity corresponding to each relationship; and set a threshold value ThresholdB, screen out the entities whose probability value is greater than ThresholdB in the Result_tmp matrix, and obtain the Result matrix, that is, Result= (soft max(Result_tmp)>=TresholdB);

S7: Construct a polytree according to the similarity relationship set F(r _i ,r _j ) and the Result matrix;

S8: The relational forest is composed of multiple multi-fork trees.

Further, in the step S4, the perceptron of the improved multi-layer perceptron model is 5 layers, and the hidden value is 1024, that is, each layer has 1024 neural units; wherein, the dimension of the relation word vector is the number of relation words × the number of neural units The dimension of the Tmp_rel matrix is the number of neural units × the number of relations, and the dimension of the Tmp_ent matrix is the number of neural units × the number of entity words.

Further, in the step S7, constructing a polytree specifically includes: counting the number r _n _num of each row in the Result matrix, and combining the similarity relationship set F(r _i ,r _j ); in the similarity relationship, it is assumed that the number of corresponding entities is large If the number of corresponding entities is small, it will be regarded as a hyponymous relationship;

Case 1: If there is a similarity relationship S(r ₁ , r ₂ ), S(r ₁ , r ₃ ) in F(r _i , r _j ), then find out in the Result matrix according to r ₁ , r ₂ , r ₃ corresponding r ₁ _num, r ₂ _num, r ₃ _num;

Case 2: If there is a similarity relationship S(r ₁ , r ₂ ), S(r ₁ , r ₃ ), S(r ₁ , r ₄ ), S(r ₃ , r ₅ ) in F(r _i , r _j ) ), then find out the corresponding r ₁ _num, r ₂ _num, r ₃ _num, r ₄ _num, r ₅ _num in the Result matrix according to r ₁ , r ₂ , r ₃ ;

By analogy, a polytree is constructed according to the similarity relation set F(r _i ,r _j ) and the Result matrix.

Further, in case 1, R1=softmax(r ₁ _num), R2=softmax(r ₂ _num), R3=softmax(r ₃ _num), take the root node Root=max(R1, R2, R3), and the rest are child node.

Furthermore, in case 2, R1=softmax(r ₁ _num), R2=softmax(r ₂ _num), R3=softmax(r ₃ _num), R4=softmax(r ₄ _num), R5=softmax(r ₅ _num) ), take the root node Root=max(R1, R2, R3, R4, R5), and the rest are child nodes.

The beneficial effects of the present invention are:

1) In the existing relationship extraction technology, due to the problem of similar relationships, the relationship extraction accuracy is not high. Using the relationship forest of the present invention to remove redundant relationships can effectively alleviate the problem of low relationship extraction accuracy.

2) In the task of the recommendation system, the relationship forest of the present invention is used to effectively discover the recommended content of the similar relationship and improve the recommendation accuracy.

3) In relational reasoning tasks, similar relations are retrieved in relational forests to improve the depth and breadth of relational reasoning.

Other advantages, objects, and features of the present invention will be set forth in the description that follows, and will be apparent to those skilled in the art based on a study of the following, to the extent that is taught in the practice of the present invention. The objectives and other advantages of the present invention may be realized and attained by the following description.

Description of drawings

In order to make the purpose, technical scheme and advantages of the present invention clearer, the present invention will be preferably described in detail below in conjunction with the accompanying drawings, wherein:

Figure 1 is a schematic structural diagram of a traditional multi-layer original perceptron model;

Fig. 2 is the flow chart of the construction method of the upper and lower sense relation forest of the present invention;

Fig. 3 is the improved multilayer perceptron model structure schematic diagram of the present invention, (a) is the internal structure schematic diagram that input is relation word vector, (b) is the internal structure schematic diagram that input is entity word vector;

Fig. 4 is the node relation schematic diagram of situation 1 in the present invention constructs multi-fork tree;

5 is a schematic diagram of the node relationship of the situation 2 in the construction of a multi-node tree according to the present invention;

Fig. 6 is the relational forest schematic diagram that is formed by a plurality of multi-fork trees;

FIG. 7 is a flow chart of applying the relational forest constructed by the present invention to a search engine in a recommendation system.

Detailed ways

The embodiments of the present invention are described below through specific specific examples, and those skilled in the art can easily understand other advantages and effects of the present invention from the contents disclosed in this specification. The present invention can also be implemented or applied through other different specific embodiments, and various details in this specification can also be modified or changed based on different viewpoints and applications without departing from the spirit of the present invention. It should be noted that the drawings provided in the following embodiments are only used to illustrate the basic idea of the present invention in a schematic manner, and the following embodiments and features in the embodiments can be combined with each other without conflict.

Please refer to Fig. 1 to Fig. 6, Fig. 2 is a flowchart of a method for constructing an upper-lower relationship forest based on relationship similarity, and the method specifically includes the following steps:

Step 1: The corpus of the embodiment of the present invention comes from triples (entity 1, relationship, entity 2) extracted from the knowledge base, and the entity pair probability P _θ1 =( h,t|r ₁ ) and relation 2 entity pair probability P _θ2 =(h,t|r ₂ ), where h: (head) entity 1, t: (tail) entity 2, r: (relation) relation Then, the relationship similarity Sim _r1,r2 of the relationship 1 and the relationship 2 is obtained through the formula (2) and the formula (3).

Step 2: Obtain the entity pair probability set C=(P _θ1 ,P _θ2 ,P _θ3 ,...,P _θn ) (n is the number of relations) in all triples through the multi-layer perceptron, and compare the set C with itself Do a Cartesian product to get a set of size n×n.

Delete the same pair of two elements in the Cartesian product in Table 1, leaving n ² -n pairs of elements. Each pair of elements uses formula (2) and formula (3) to obtain a relationship similarity measure S(r _i , r _j ) (i=1,2,...,n, j=1,2,...,n).

Table 1 C×C (Cartesian product)

P_θ1 P_θ1 P_θ1 P_θ2 P_θ1 P_θ3 ... P_θ1 P_θn P_θ2 P_θ1 P_θ2 P_θ2 P_θ2 P_θ3 ... P_θ2 P_θn P_θ3 P_θ1 P_θ3 P_θ2 P_θ3 P_θ3 ... P_θ3 P_θn ... ... ... ... ... P_θn P_θ1 P_θn P_θ2 P_θn P_θ3 ... P_θn P_θn

Step 3: Take the threshold value ThresholdA=0.7, take all the relational metric values of S(r _i ,r _j )>=ThresholdA in Table 2, the S(r _i ,r _j ) value filtered by this condition is called similar relation. The relation set after the similarity relation is extracted from Table 2 is F(r _i ,r _j ).

Table 2 Relationship similarity measure matrix

S(r₁,r₁) S(r₁,r₂) S(r₁,r₃) ... S(r₁,r_n) S(r₂,r₁) S(r₂,r₂) S(r₂,r₃) ... S(r₂,r_n) S(r₃,r₁) S(r₃,r₂) S(r₃,r₃) ... S(r₃,r_n) ... ... ... ... ... S(r_n,r₁) S(r_n,r₂) S(r_n,r₃) ... S(r_n,r_n)

Step 4: Design a new model multi-layer perceptron, the perceptron is 5 layers, and the hidden value is 1024, that is, each layer has 1024 neural units. As shown in Figure 3(a), the relation word vector is used as the input of the multilayer perceptron model, and the dimension of the relation word vector is (the number of relation words × the number of neural units); the output is the Tmp_rel matrix, and the dimension is (the number of neural units) number × number of relationships). As shown in Figure 3(b), the entity word vector is used as the input of the multi-layer perceptron model, and the dimension of the entity word vector is (the number of entity words × the number of neural units); the output is the Tmp_ent matrix, and the dimension is (the number of neural units) number × number of entity words).

Step 5: Multiply the result of the relationship word vector and the entity word vector into the multi-layer perceptron, that is, Tmp_rel*Tmp_ent to obtain the Result_tmp matrix, and the matrix dimension is (the number of relations × the number of entities).

Step 6: Calculate Softmax for each row of Result_tmp. It means to obtain the probability for the entity corresponding to each relation, and set the threshold value ThresholdB=0.8. Filter the matrix Result_tmp to retain entities with probability values greater than 0.8 in the Result matrix.

Result_tmp=Tmp_rel*Tmp_ent (4)

Result=(soft max(Result_tmp)>=TresholdB) (5)

The assumption is made here: in the similarity relationship, a relationship with a large number of entities corresponding to the relationship is regarded as a superordinate relationship, and a relationship with a small number of corresponding entities is regarded as a hyponymous relationship.

Step 7: Construct a polytree according to the similarity relationship set F(r _i ,r _j ) and the Result matrix;

Count the number of each row in the Result matrix, and combine the similarity relationship set F(r _i ,r _j ). In the similarity relationship, the one with a large number of corresponding entities is regarded as a superordinate relationship, and the one with a small number of corresponding entities is regarded as a subordinate relationship.

Case 1: As shown in Figure 4, if _there is a similarity relationship S(r ₁ , r ₂ ), S(r ₁ , r ₃ ) in F(ri , r _j ), then according to r ₁ , r ₂ , r ₃ Find the corresponding r ₁ _num, r ₂ _num, r ₃ _num in the Result matrix.

R1=softmax(r ₁ _num) (6)

R2=softmax(r ₂ _num) (7)

R3=softmax(r ₃ _num) (8)

Take the root node as:

Root=max(R1,R2,R3) (9)

The left and right child nodes are R2, R3.

Case 2: As shown in Figure 5, if there is a similarity relationship S(r ₁ , r ₂ ), S(r ₁ , r ₃ ), S(r ₁ , r ₄ ), S(r 1 , r 3 ), S(r 1 , r 4 ) in F(r _i , r _j ) (r ₃ , r ₅ ), then find out the corresponding r ₁ _num, r ₂ _num, r ₃ _num, r ₄ _num, r ₅ _num in the Result matrix according to r ₁ , r ₂ , r ₃ .

R1=softmax(r ₁ _num) (10)

R2=softmax(r ₂ _num) (11)

R3=softmax(r ₃ _num) (12)

R4=softmax(r ₄ _num) (13)

R5=softmax(r ₅ _num) (14)

Take the root node as:

Root=max(R1,R2,R3,R4,R5) (15)

The child nodes are R2, R3, R4, and R5 is a child node of R3.

And so on, according to F(r _i , r _j ) similar relation set and Result matrix to construct multi-fork tree.

Step 8: Constitute a relational forest from multiple polytrees. In Fig. 6, the gray node is the root node, which is obtained by root=max(R1, R2, . . . , Rn), the white node is the child node, and the child node is obtained by formulas (6) to (15). Eight sets of similarity relationships are illustrated in Figure 6.

As shown in Figure 7, the application of the relational forest constructed by the present invention in practical applications, such as applying it to a recommendation system for a search engine, can quickly and accurately obtain the required recommendation content.

Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and not to limit them. Although the present invention has been described in detail with reference to the preferred embodiments, those of ordinary skill in the art should understand that the technical solutions of the present invention can be Modifications or equivalent replacements, without departing from the spirit and scope of the technical solution, should all be included in the scope of the claims of the present invention.

Claims (5)

1. a method for constructing a relational forest based on relational similarity, characterized in that the method comprises the following steps: S1: Input all triples, and obtain the entity pair probability set C=(P _θ1 , P _θ2 , P _θ3 ,..., P _θn ) through the traditional multilayer perceptron, where the probability P _θi is (entity i ₁ , entity i ₂ | relationship), n is the number of relationships; S2: Do the Cartesian product of the set C with itself to obtain a set of size n×n, delete the same pair of two elements in the Cartesian product, and leave n ² -n pairs of elements, and then calculate each The relationship similarity measure value S(r _i ,r _j ) for the elements, where i=1,2,...,n, j=1,2,...,n; S3: Set the threshold value ThresholdA, and filter out the relationship metric value S(r _i , r _j ) that is greater than or equal to ThresholdA as the similar relationship set F(r _i , r _j ); S4: Improve the multi-layer perceptron model, respectively use the relation word vector or the entity word vector as the input of the improved multi-layer perceptron model, and the output is a Tmp_rel matrix or a Tmp_ent matrix; S5: Multiply the output result after inputting the relation word vector and the entity word vector into the multi-layer perceptron, that is, Tmp_rel*Tmp_ent to obtain the Result_tmp matrix, and the dimension is the number of relations × the number of entities; S6: Calculate Softmax for each row of the Result_tmp matrix, indicating that the probability is obtained for the entity corresponding to each relationship; and set the threshold value ThresholdB, filter out the entities whose probability value is greater than ThresholdB in the Result_tmp matrix, and obtain the Result matrix, that is, Result= (softmax(Result_tmp)>=TresholdB); S7: Construct a polytree according to the similarity relationship set F(r _i ,r _j ) and the Result matrix; S8: The relational forest is composed of multiple multi-fork trees. 2. a kind of upper and lower relation forest construction method based on relation similarity according to claim 1, is characterized in that, in described step S4, the perceptron of the improved multilayer perceptron model is 5 layers, and hidden is 1024 That is, each layer has 1024 neural units; among them, the dimension of the relation word vector is the number of relation words × the number of neural units, the dimension of the Tmp_rel matrix is the number of neural units × the number of relations, and the dimension of the Tmp_ent matrix is the number of neural units × the number of entities number of words. 3. a kind of upper and lower sense relation forest construction method based on relation similarity according to claim 1, is characterized in that, in described step S7, constructing polynomial tree specifically comprises: the quantity r _n of each row in statistical Result matrix _num, combined with the similarity relationship set F(r _i , r _j ); in the similarity relationship, it is assumed that the number of corresponding entities is large as a superordinate relationship, and the number of corresponding entities is small as a hyponymous relationship; Case 1: If there is a similarity relationship S(r ₁ , r ₂ ), S(r ₁ , r ₃ ) in F(r _i , r _j ), then find out in the Result matrix according to r ₁ , r ₂ , r ₃ corresponding r ₁ _num, r ₂ _num, r ₃ _num; Case 2: If there is a similarity relationship S(r ₁ , r ₂ ), S(r ₁ , r ₃ ), S(r ₁ , r ₄ ), S(r ₃ , r ₅ ) in F(r _i , r _j ) ), then find out the corresponding r ₁ _num, r ₂ _num, r ₃ _num, r ₄ _num, r ₅ _num in the Result matrix according to r ₁ , r ₂ , r ₃ ; By analogy, a polytree is constructed according to the similarity relation set F(r _i ,r _j ) and the Result matrix. 4. The method for constructing a relational forest based on relational similarity according to claim 3, characterized in that, in case ₁ , R1=softmax(r1_num), R2=softmax( _{r2_num} ), R3 =softmax(r ₃ _num), take the root node Root=max(R1, R2, R3), and the rest are child nodes. 5. A method for constructing a relational similarity forest based on relational similarity according to claim 3, wherein in case 2, R1=softmax( _{r1_num} ), R2=softmax( _{r2_num} ), R3 =softmax(r ₃ _num), R4=softmax(r ₄ _num), R5=softmax(r ₅ _num), take the root node Root=max(R1, R2, R3, R4, R5), and the rest are child nodes.

Images