CN111897924A - A Text Retrieval Method Based on Fusion and Extension of Association Rules and Word Vectors - Google Patents

Abstract

The invention proposes a text retrieval method based on the fusion and expansion of association rules and word vectors. The initial inspection documents obtained by querying and retrieving the original Chinese document set are used to construct the initial inspection document set, and deep learning tools are used to carry out the initial inspection document set. The word vector semantic learning training is performed to obtain the feature word vector set, and then the first m documents are extracted from the initial inspection document set as the pseudo-related feedback document set, and the support and confidence based on the Copulas function are used to mine the pseudo-related feedback document set. Candidate extension words , establish a candidate expansion word set, finally calculate the vector cosine similarity between the candidate expansion word and the original query, extract the final expansion word, combine the final expansion word with the original query into a new query, and retrieve the original document set again to obtain the final retrieval result. The experimental results show that the retrieval performance of the method of the present invention is better than the existing methods, can effectively reduce the problem of query subject drift and word mismatch, improve the information retrieval performance, and has good application value and promotion prospect.

Description

A Text Retrieval Method Based on Fusion and Extension of Association Rules and Word Vectors

technical field

The invention relates to a text retrieval method based on the fusion and expansion of association rules and word vectors, and belongs to the technical field of information retrieval.

Background technique

With the development of network technology and the rapid growth of digital resources, how network users can quickly and accurately find the information resources they need, and how to reduce query subject drift and word mismatch to meet the information needs of network users is an important issue that needs to be solved urgently in the field of information retrieval. question. The use of query expansion technology in information retrieval can solve the above problems. Query expansion refers to transforming the original query weight, or adding other feature words related to the original query semantics, to make up for the lack of semantic information caused by the original query being too simple, and to improve Purpose of Information Retrieval Performance. In the past ten years, scholars have studied information retrieval methods based on query expansion from different perspectives, and some effective information retrieval methods have been produced. For example, an information retrieval method based on query expansion and classification proposed by Yue Wen et al. (see Literature: Yue Wen, Chen Zhiping, Lin Yaping. Information Retrieval Algorithm Based on Query Expansion and Classification [J]. Journal of System Simulation, 2006, 018(007): 1926-1929, 1934.), Vaidyanathan et al. (Vaidyanathan R, Das S, Srivastava N. Query Expansion Strategy based on Pseudo Relevance Feedback and Term Weight Scheme for Monolingual Retrieval [J]. International Journal of Computer Applications, 2015, 105(8): 1-6.) proposes a pseudo-relevant feedback extension information retrieval method, Etc., these methods have verified the effectiveness of the retrieval method through experiments, but have not finally completely solved the technical problems such as query subject drift and word mismatch in information retrieval.

In order to solve the technical problems such as query subject drift and word mismatch in the current information retrieval system, and improve the retrieval performance of the information system, the present invention uses the Copulas function (see document: Sklar A.Fonctions de repartitionàndimensions et leurs marges[J].Publication de l' Institut de Statistique l'Universite Paris, 1959, 8(1):229-231.) introduced the field of information retrieval, and integrated association pattern mining and word vector semantic learning in information retrieval to achieve query expansion, and proposed a new method based on association rules. The text retrieval method fused and extended with the word vector, the experimental results show that the method of the invention can improve and improve the cross-information retrieval performance, and has good application value and promotion prospect.

SUMMARY OF THE INVENTION

The purpose of the present invention is to propose a text retrieval method based on the fusion and expansion of association rules and word vectors, which can be used in the field of information retrieval, such as web information retrieval systems and search engines, and can reduce query topic drift and word inconsistency in information retrieval. Matching questions to improve and enhance the query performance of information retrieval systems.

The concrete technical scheme adopted in the present invention is as follows:

A text retrieval method based on the fusion extension of association rules and word vectors, comprising the following steps:

Step 1. Chinese users query and retrieve the original Chinese document set to obtain a preliminary inspection document, and construct a preliminary inspection document set.

Step 2. Use deep learning tools to perform word vector semantic learning training on the initial inspection document set to obtain a feature word word vector set.

The deep learning tool in the present invention refers to the Skip-gram model of Google's open source word vector tool word2vec (see: https://code.google.com/p/word2vec/ for details).

Step 3. Extract the first m initial inspection documents from the initial inspection document set as pseudo-relevant feedback documents, construct a pseudo-relevant feedback document set, and perform Chinese word segmentation, Chinese stop words removal, and feature word extraction on the initial inspection pseudo-relevant feedback document set. Preprocessing, and calculating feature word weights, and finally constructing pseudo-relevant feedback Chinese document database and Chinese feature word database.

The present invention adopts TF-IDF (term frequency-inverse document frequency) weighting technology (see document: Ricardo Baeza-Yates Berthier Ribeiro-Neto et al., translated by Wang Zhijin et al., "Modern Information Retrieval", Machinery Industry Press, 2005: 21-22.) Calculate feature word weights.

Step 4. Use the support and confidence based on the Copulas function to mine the candidate extension words from the pseudo-related feedback document set, and establish the candidate extension word set. The specific steps are as follows:

(4.1) Generate 1_candidate item set C ₁ : extract feature words from the Chinese feature lexicon as 1_candidate item set C ₁ .

(4.2) Generate 1_frequent itemset L ₁ : calculate the support Cop_Sup(C ₁ ) of C ₁ based on the Copulas function, and extract C ₁ whose Cop_Sup(C ₁ ) is not lower than the minimum support threshold ms as 1_ frequent itemsets L ₁ , and added to the frequent itemset set FIS (Frequent ItemSet).

The Cop_Sup (Copulas basedSupport) represents the support degree based on the Copulas function. The calculation of Cop_Sup(C ₁ ) of the 1_candidate item set C ₁ is shown in formula (1):

表示1_候选项集C₁在伪相关反馈中文文档库中出现的频度，DocNum表示伪相关反馈中文文档库总文档数量，

表示1_候选项集C₁在伪相关反馈中文文档库中的项集权重，ItemsWeight表示伪相关反馈中文文档库中全体中文特征词的权重累加和。In formula (1),

Represents the frequency of 1_candidate item set C ₁ appearing in the pseudo-relevant feedback Chinese document library, DocNum indicates the total number of documents in the pseudo-relevant feedback Chinese document library,

Represents the item set weight of 1_candidate item set C ₁ in the pseudo-relevant feedback Chinese document database, and ItemsWeight represents the weight accumulation of all Chinese feature words in the pseudo-relevant feedback Chinese document database.

(4.3) Generate k_candidate item set C _k : self-connect (k-1)_frequent itemset L _k-1 to generate k_ candidate item set C _k , where k≥2.

The self-connection method adopts Apriori algorithm (see document: Agrawal R, Imielinski T, Swami A. Mining association rules between sets of items in large database [C]//Proceedings of the 1993 ACM SIGMOD International Conference on Management of Data, Washington DC, USA , 1993:207-216.) The candidate item set connection method given in.

(4.4) 2_candidate item set C ₂ pruning: when mining to 2_ candidate item set C ₂ , if the C ₂ does not contain the original query term, delete the C ₂ , if the C ₂ contains the original query term item, then leave the C ₂ , and then, go to step (4.5) with the remaining C ₂ ; when the k_candidate item set C _k is mined, and the k ≥ 3, then C _k directly goes to step (4.5) .

(4.5) Generating k_frequent itemsets L _k : Calculate Cop_Sup(C _k ), extract C _k whose Cop_Sup(C _k ) is not lower than ms as k_ frequent itemsets L _k , and add them to FIS.

The calculation of the Cop_Sup(C _k ) is shown in formula (2):

表示k_候选项集C_k在伪相关反馈中文文档库中出现的频度，

表示k_候选项集C_k在伪相关反馈中文文档库中的项集权重；DocNum和ItemsWeight的定义与式(1)相同。In formula (2),

represents the frequency of k_candidate item set C _k appearing in the pseudo-relevant feedback Chinese document library,

Represents the item set weight of k_candidate item set C _k in the pseudo-relevant feedback Chinese document library; the definitions of DocNum and ItemsWeight are the same as formula (1).

(4.6) After adding 1 to k, go to step (4.3) and continue to execute subsequent steps in sequence until the L _k is an empty set, then the frequent itemset mining ends, and go to step (4.7).

(4.7) Take L _k arbitrarily from the FIS, where k ≥ 2.

Q_i∪Ret_j＝L_k，

所述Ret_j为不含查询词项的真子集项集，所述Q_i为含查询词项的真子集项集，所述Q为原查询词项集合。(4.8) Extract the association rule Q _i →Ret _j from L _k , calculate the confidence level Cop_Con(Q _i →Ret _j ) of the association rule Q _i →Ret _j based on the Copulas function, the i≥1, j≥1, and

Q _i ∪Ret _j =L _k ,

The Ret _j is the proper subset itemset without query terms, the Q _i is the proper subset itemsets with query terms, and the Q is the original query term set.

The Cop_Con (Copulas based Confidence) represents the confidence based on the Copulas function, and the calculation formula of the Cop_Con (Q _i →Ret _j ) is shown in formula (3):

表示k_频繁项集L_k在伪相关反馈中文文档库中出现的频度，

表示k_频繁项集L_k在伪相关反馈中文文档库中的项集权重，num_Qi表示k_频繁项集L_k的真子集项集Q_i在伪相关反馈中文文档库中出现的频度，weight_Qi表示k_频繁项集L_k的真子集项集Q_i在伪相关反馈中文文档库中的项集权重；exp表示以自然常数e为底的指数函数。In formula (3),

represents the frequency of k_frequent itemsets L _k appearing in the pseudo-relevant feedback Chinese document library,

Represents the itemset weight of _{k_frequent} itemset L _k in the pseudo-relevant feedback Chinese document database, and num _Qi represents the frequency of the proper subset itemset Qi of k_ frequent itemset L _k in the pseudo-relevant feedback Chinese document database , weight _Qi represents the item set weight of the proper subset itemset Qi of _{k_frequent} itemset L _k in the pseudo-relevant feedback Chinese document database; exp represents the exponential function with the natural constant e as the base.

(4.9) Generate association rules Qi → Ret _j : extract the association rules Qi → Ret _{j whose Cop_Con (Q i → Ret j} ) is not less than the minimum confidence threshold mc _and add them to the association rule set AR (Association Rule), then turn to Go to step (4.8), re-extract other proper subset itemsets Ret _j and Qi from L _{k ,} and then perform subsequent steps in sequence, and so on, until all proper subset itemsets of L _k are taken out if and only if all At this time, go to step (4.7), carry out a new round of association rule pattern mining, take out any other L _k from the FIS, and then perform the subsequent steps in sequence, and so on, until all k_frequent items in the FIS are If and only if the set L _k is taken out once, then the association rule pattern mining is completed, and the following step (4.10) is performed.

(4.10) Generate candidate expansion words: extract the association rule consequent Ret _j from the association rule set AR as a candidate expansion word, obtain the candidate expansion word set CETS (Candidate Expansion Term Set), calculate the candidate expansion word weight w _Ret , and then, Go to step 5.

The CETS is shown in formula (4):

In formula (4), Ret _i represents the ith candidate extension word.

The calculation formula of the candidate extended word weight w _Ret is shown in formula (5):

In formula (5), max() represents the maximum value of association rule confidence. When the same rule expansion word appears in multiple association rule patterns at the same time, the one with the largest confidence value is taken as the weight of the rule expansion word.

Step 5. Calculate the vector cosine similarity between the candidate expansion word and the original query word, and extract the candidate expansion word that is not lower than the similarity threshold as the final expansion word. The specific steps are as follows:

(5.1) Calculate the vector similarity VecSim(Ret _i , Q): In the feature word vector set, calculate the candidate extension word Ret _i and the original query term set Q (the Q=(q ₁ ,q ₂ ,...,q _r )) vector cosine similarity VecSim(Ret _i ,q _j ) of each query word {q ₁ ,q ₂ ,...,q _r }, the 1≤j≤r, the candidate expansion word and each query are accumulated The word vector similarity is taken as the total vector similarity VecSim(Ret _i ,Q) of the candidate expanded word.

The VecSim(Ret _i ,q _j ) is shown in formula (6):

In formula (6), the vRet _i represents the word vector value of the i-th candidate expansion word Ret _i , and vq _j represents the word vector value of the j-th query term q _j .

The VecSim(Ret _i , Q) calculation formula is shown in formula (7):

(5.2) Generate the final expanded word: extract the candidate expanded word whose VecSim(Ret _i , Q) is not lower than the vector similarity threshold minVSim as the final expanded word of the original query term set Q, and obtain the final expanded word set FETS (FinalExpansionTerm Set) , as shown in formula (8):

In formula (8), Ret _i is the ith final extended word.

Then the weight w _Fet of the final extended word Ret is calculated, and the weight w _Fet of the final extended word Ret is composed of the candidate extended word weight w _Ret and the vector similarity VecSim(Ret _i , Q). The w _Fet calculation formula is shown in formula (9):

Step 6. The expanded word is combined with the original query into a new query, and the original Chinese document set is retrieved again to obtain the final retrieved document.

Step 7. Return the final retrieved document set to the user.

Compared with the prior art, the present invention has the following beneficial effects:

(1) The present invention proposes a text retrieval method based on the fusion and expansion of association rules and word vectors. The inventive method uses deep learning tools to perform word vector semantic learning training on the initial inspection document set to obtain a feature word vector set. The support and confidence of the Copulas function are used to mine the candidate expansion words from the pseudo-relevant feedback document set, calculate the vector cosine similarity between the candidate expansion words and the original query, and extract the candidate expansion words that are not lower than the similarity threshold as the final expansion words. The word and the original query are combined into a new query, the original document set is retrieved again to obtain the final retrieval result, and the final retrieved document set is returned to the user. The experimental results show that the retrieval performance of the method of the present invention is better than the existing methods, can effectively reduce the problem of query subject drift and word mismatch, improve the information retrieval performance, and has good application value and promotion prospect.

(2) Four similar query expansion methods that have appeared in recent years are selected as the comparison methods of the method of the present invention, and the experimental data is the Chinese corpus of the national standard data set NTCIR-5CLIR. The experimental results show that the P@5 value of the experimental result of the method of the present invention is higher than that of the benchmark retrieval. Compared with the four comparison methods, the P@5 value of the method of the present invention is mostly improved, indicating that the retrieval performance of the method of the present invention is excellent. Based on benchmark retrieval and comparison methods, it can improve information retrieval performance, reduce query drift and word mismatch problems in information retrieval, and has high application value and broad promotion prospects.

Description of drawings

FIG. 1 is a schematic overall flow chart of the text retrieval method based on the fusion extension of association rules and word vectors according to the present invention.

Detailed ways

1. In order to better illustrate the technical solution of the present invention, the related concepts involved in the present invention are introduced as follows:

1. Itemset

In text mining, a text document is regarded as a transaction, each feature word in the document is called an item, the collection of feature word items is called an itemset, and the number of all items in an itemset is called the itemset length. k_itemsets refer to itemsets containing k items, where k is the length of the itemsets.

2. Antecedents and Consequences of Association Rules

Let x and y be an arbitrary set of feature terms, and the implication in the form of x→y is called an association rule, where x is called the antecedent of the rule, and y is called the consequent of the rule.

3. Support based on Copulas function

Copulas based Support is represented as Cop_Sup().

The calculation of the support degree Cop_Sup(T ₁ ∪T ₂ ) of the feature word item set (T ₁ ∪ T ₂ ) based on the Copulas function is shown in formula (10):

表示项集(T₁∪T₂)在伪相关反馈中文文档库中出现的频度，

表示项集(T₁∪T₂)在伪相关反馈中文文档库中的项集权重。DocNum表示伪相关反馈中文文档库总文档数量，ItemsWeight表示伪相关反馈中文文档库中全体中文特征词的权重累加和。In formula (10),

represents the frequency of itemsets (T ₁ ∪ T ₂ ) appearing in the pseudo-relevant feedback Chinese document database,

Represents the itemset weight of the itemset (T ₁ ∪ T ₂ ) in the pseudo-relevant feedback Chinese document database. DocNum represents the total number of documents in the pseudo-relevant feedback Chinese document database, and ItemsWeight represents the weight accumulation of all Chinese feature words in the pseudo-relevant feedback Chinese document database.

4. Confidence based on Copulas function

The confidence based on the Copulas function ((Copulas based Confidence) is represented as Cop_Con().

The calculation formula of the feature word association rule T ₁ →T ₂ confidence Cop_Con(T ₁ →T ₂ ) based on the Copulas function is shown in formula (11):

表示项集(T₁∪T₂)在伪相关反馈中文文档库中出现的频度，

表示项集(T₁∪T₂)在伪相关反馈中文文档库中的项集权重，

表示项集T₁在伪相关反馈中文文档库中出现的频度，

表示项集T₁在伪相关反馈中文文档库中的项集权重。exp表示以自然常数e为底的指数函数。In formula (11),

represents the frequency of itemsets (T ₁ ∪ T ₂ ) appearing in the pseudo-relevant feedback Chinese document database,

represents the itemset weight of the itemset (T ₁ ∪ T ₂ ) in the pseudo-relevant feedback Chinese document database,

represents the frequency of itemset T ₁ appearing in the pseudo-relevant feedback Chinese document database,

Represents the itemset weight of the itemset T1 in the pseudo _- relevant feedback Chinese document database. exp represents the exponential function with the natural constant e as the base.

2. The present invention will be further described below in conjunction with the accompanying drawings and specific comparative experiments.

As shown in Figure 1, the text retrieval method based on the fusion extension of association rules and word vectors of the present invention comprises the following steps:

Step 1. Chinese users query and retrieve the original Chinese document set to obtain a preliminary inspection document, and construct a preliminary inspection document set.

Step 2. Use deep learning tools to perform word vector semantic learning training on the initial inspection document set to obtain a feature word word vector set.

The deep learning tool in the present invention refers to the Skip-gram model of Google's open source word vector tool word2vec.

Step 3. Extract the first m initial inspection documents from the initial inspection document set as pseudo-relevant feedback documents, construct a pseudo-relevant feedback document set, and perform Chinese word segmentation, Chinese stop words removal, and feature word extraction on the initial inspection pseudo-relevant feedback document set. After preprocessing, the TF-IDF weighting technology is used to calculate the weights of feature words, and finally the pseudo-relevant feedback Chinese document database and Chinese feature word database are constructed.

Step 4. Use the support and confidence based on the Copulas function to mine the candidate extension words from the pseudo-related feedback document set, and establish the candidate extension word set. The specific steps are as follows:

(4.1) Generate 1_candidate item set C ₁ : extract feature words from the Chinese feature lexicon as 1_candidate item set C ₁ .

(4.2) Generate 1_frequent itemset L ₁ : calculate the support Cop_Sup(C ₁ ) of C ₁ based on the Copulas function, and extract C ₁ whose Cop_Sup(C ₁ ) is not lower than the minimum support threshold ms as 1_ frequent itemsets L ₁ , and added to the frequent itemset set FIS (Frequent ItemSet).

The Cop_Sup (Copulas basedSupport) represents the support degree based on the Copulas function. The calculation of Cop_Sup(C ₁ ) of the 1_candidate item set C ₁ is shown in formula (1):

表示1_候选项集C₁在伪相关反馈中文文档库中出现的频度，DocNum表示伪相关反馈中文文档库总文档数量，

表示1_候选项集C₁在伪相关反馈中文文档库中的项集权重，ItemsWeight表示伪相关反馈中文文档库中全体中文特征词的权重累加和。In formula (1),

Represents the frequency of 1_candidate item set C ₁ appearing in the pseudo-relevant feedback Chinese document library, DocNum indicates the total number of documents in the pseudo-relevant feedback Chinese document library,

Represents the item set weight of 1_candidate item set C ₁ in the pseudo-relevant feedback Chinese document database, and ItemsWeight represents the weight accumulation of all Chinese feature words in the pseudo-relevant feedback Chinese document database.

(4.3) Generate k_candidate item set C _k : self-connect (k-1)_frequent itemset L _k-1 to generate k_ candidate item set C _k , where k≥2.

The self-connection method adopts the candidate itemset connection method given in the Apriori algorithm.

(4.4) 2_candidate item set C ₂ pruning: when mining to 2_ candidate item set C ₂ , if the C ₂ does not contain the original query term, delete the C ₂ , if the C ₂ contains the original query term item, then leave the C ₂ , and then, go to step (4.5) with the remaining C ₂ ; when the k_candidate item set C _k is mined, and the k ≥ 3, then C _k directly goes to step (4.5) .

(4.5) Generating k_frequent itemsets L _k : Calculate Cop_Sup(C _k ), extract C _k whose Cop_Sup(C _k ) is not lower than ms as k_ frequent itemsets L _k , and add them to FIS.

The calculation of the Cop_Sup(C _k ) is shown in formula (2):

表示k_候选项集C_k在伪相关反馈中文文档库中出现的频度，

表示k_候选项集C_k在伪相关反馈中文文档库中的项集权重；DocNum和ItemsWeight的定义与式(1)相同。In formula (2),

represents the frequency of k_candidate item set C _k appearing in the pseudo-relevant feedback Chinese document library,

Represents the item set weight of k_candidate item set C _k in the pseudo-relevant feedback Chinese document library; the definitions of DocNum and ItemsWeight are the same as formula (1).

(4.6) After adding 1 to k, go to step (4.3) and continue to execute subsequent steps in sequence until the L _k is an empty set, then the frequent itemset mining ends, and go to step (4.7).

(4.7) Take L _k arbitrarily from the FIS, where k ≥ 2.

Q_i∪Ret_j＝L_k，

所述Ret_j为不含查询词项的真子集项集，所述Q_i为含查询词项的真子集项集，所述Q为原查询词项集合。(4.8) Extract the association rule Q _i →Ret _j from L _k , calculate the confidence level Cop_Con(Q _i →Ret _j ) of the association rule Q _i →Ret _j based on the Copulas function, the i≥1, j≥1, and

Q _i ∪Ret _j =L _k ,

The Ret _j is the proper subset itemset without query terms, the Q _i is the proper subset itemsets with query terms, and the Q is the original query term set.

The Cop_Con (Copulas based Confidence) represents the confidence based on the Copulas function, and the calculation formula of the Cop_Con (Q _i →Ret _j ) is shown in formula (3):

表示k_频繁项集L_k在伪相关反馈中文文档库中出现的频度，

表示k_频繁项集L_k在伪相关反馈中文文档库中的项集权重，num_Qi表示k_频繁项集L_k的真子集项集Q_i在伪相关反馈中文文档库中出现的频度，weight_Qi表示k_频繁项集L_k的真子集项集Q_i在伪相关反馈中文文档库中的项集权重；exp表示以自然常数e为底的指数函数。In formula (3),

represents the frequency of k_frequent itemsets L _k appearing in the pseudo-relevant feedback Chinese document library,

Represents the itemset weight of _{k_frequent} itemset L _k in the pseudo-relevant feedback Chinese document database, and num _Qi represents the frequency of the proper subset itemset Qi of k_ frequent itemset L _k in the pseudo-relevant feedback Chinese document database , weight _Qi represents the item set weight of the proper subset itemset Qi of _{k_frequent} itemset L _k in the pseudo-relevant feedback Chinese document database; exp represents the exponential function with the natural constant e as the base.

(4.9) Generate association rules Qi → Ret _j : extract the association rules Qi → Ret _{j whose Cop_Con (Q i → Ret j} ) is not less than the minimum confidence threshold mc _and add them to the association rule set AR (Association Rule), then turn to Go to step (4.8), re-extract other proper subset itemsets Ret _j and Qi from L _{k ,} and then perform subsequent steps in sequence, and so on, until all proper subset itemsets of L _k are taken out if and only if all At this time, go to step (4.7), carry out a new round of association rule pattern mining, take out any other L _k from the FIS, and then perform the subsequent steps in sequence, and so on, until all k_frequent items in the FIS are If and only if the set L _k is taken out once, then the association rule pattern mining is completed, and the following step (4.10) is performed.

(4.10) Generate candidate expansion words: extract the association rule consequent Ret _j from the association rule set AR as a candidate expansion word, obtain the candidate expansion word set CETS (Candidate Expansion Term Set), calculate the candidate expansion word weight w _Ret , and then, Go to step 5.

The CETS is shown in formula (4):

In formula (4), Ret _i represents the ith candidate extension word.

The calculation formula of the candidate extended word weight w _Ret is shown in formula (5):

In formula (5), max() represents the maximum value of association rule confidence. When the same rule expansion word appears in multiple association rule patterns at the same time, the one with the largest confidence value is taken as the weight of the rule expansion word.

Step 5. Calculate the vector cosine similarity between the candidate expansion word and the original query word, and extract the candidate expansion word that is not lower than the similarity threshold as the final expansion word. The specific steps are as follows:

(5.1) Calculate the vector similarity VecSim(Ret _i , Q): In the feature word vector set, calculate the candidate extension word Ret _i and the original query term set Q (the Q=(q ₁ ,q ₂ ,...,q _r )) vector cosine similarity VecSim(Ret _i ,q _j ) of each query word {q ₁ ,q ₂ ,...,q _r }, the 1≤j≤r, the candidate expansion word and each query are accumulated The word vector similarity is taken as the total vector similarity VecSim(Ret _i ,Q) of the candidate expanded word.

The VecSim(Ret _i ,q _j ) is shown in formula (6):

In formula (6), the vRet _i represents the word vector value of the i-th candidate expansion word Ret _i , and vq _j represents the word vector value of the j-th query term q _j .

The VecSim(Ret _i , Q) calculation formula is shown in formula (7):

(5.2) Generate the final expanded word: extract the candidate expanded word whose VecSim(Ret _i , Q) is not lower than the vector similarity threshold minVSim as the final expanded word of the original query term set Q, and obtain the final expanded word set FETS (FinalExpansionTerm Set) , as shown in formula (8):

In formula (8), Ret _i is the ith final extended word.

Then the weight w _Fet of the final extended word Ret is calculated, and the weight w _Fet of the final extended word Ret is composed of the candidate extended word weight w _Ret and the vector similarity VecSim(Ret _i , Q). The w _Fet calculation formula is shown in formula (9):

Step 6. The expanded word is combined with the original query into a new query, and the original Chinese document set is retrieved again to obtain the final retrieved document.

Step 7. Return the final retrieved document set to the user.

Experimental design and results:

We carry out experimental comparison with existing similar methods to illustrate the effectiveness of the method of the present invention.

1. Experimental environment and experimental data:

The experimental data of the present invention is NTCIR-5CLIR (see: http://research.nii.ac.jp/ntcir/data/data-en.html.) Chinese text corpus, a total of 8 data sets, a total of 901446 Chinese documents , and the specific information is shown in Table 1. The corpus is an international standard corpus, including document set, query set and result set, that is, 50 Chinese queries, 4 types of query topics and 2 evaluation criteria result sets. This paper uses Description (Desc) and Title query subjects to complete the retrieval experiment. Title query is a short query, which briefly describes the query subject in terms of nouns and noun phrases, while Desc query is a long query, which briefly describes the query subject in sentence form; the result set includes Rigid (highly relevant, relevant to the query) and Relax (highly relevant, relevant and partially relevant to the query) evaluation criteria.

The experimental data preprocessing is: word segmentation and removal of stop words.

The experimental result retrieval evaluation index is MAP (Mean Average Precision)

Table 1 The original corpus of the experiment of the present invention and its quantity

2. Benchmark retrieval and comparison methods:

The basic retrieval environment of the experiment is built with Lucene.Net (see: http://lucenenet.apache.org/).

The benchmark retrieval is the retrieval result obtained by submitting the original query to Lucene.Net for initial retrieval.

The comparison method is described as follows:

Comparison method 1: Use the weighted association pattern mining technology of literature (Huang Mingxuan. Vietnamese-English cross-language query expansion based on weighted association pattern mining [J]. Journal of Information Science, 2017, 36(3): 307-318.) to mine rule expansion word, the parameters are mc=0.1, mi=0.0001, and the experimental result is the average value when ms is 0.004, 0.005, 0.006, and 0.007 respectively.

Comparison method 2: Using literature (Huang Mingxuan, Jiang Caoqing. Post-translation expansion of Vietnamese-English cross-language query based on fully weighted positive and negative association pattern mining [J]. Journal of Electronic Engineering, 2018, 46(12): 3029-3036.) The fully weighted positive and negative association pattern mining technique mines regular expansion words. The parameters are mc=0.1, α=0.3, minPR=0.1, minNR=0.01, and ms are 0.10, 0.11, 0.12, 0.13, respectively, and the experimental results are averaged.

Comparison method 3: Using the literature (Zhang H R, Zhang J W, Wei X Y, et al. A new frequent pattern mining algorithm with weighted multiple minimum supports[J]. Intelligent Automation & Soft Computing, 2017, 23(4): 605-612.) The weighted frequent pattern mining technology based on multiple support thresholds mines rule expansion words, the parameters are mc=0.1, LMS=0.2, HMS=0.25, WT=0.1, ms are 0.1, 0.15, 0.2, 0.25, respectively, and the experimental results are averaged .

Comparison method 4: Using the literature (Xu Kan, Lin Yuan, Qu Chen, et al. Research on word vector method for patent query expansion [J]. Computer Science and Exploration, 2018, 12(6): 972-980.) Based on word vector query extension method. Experimental parameters: k=60, α=0.1.

The skip-gram model word vector semantic learning training parameters used in the present invention: batch_size=128, embedding_size=300, embedding_size=300, skip_window=2, num_skips=4, num_sampled=64.

3. The experimental methods and results are as follows:

The NTCIR-5CLIR corpus 50 Chinese queries were used to conduct the experiments of the present invention on 8 data sets, and the average value of the P@5 retrieval results of the benchmark retrieval BLR results, the comparison method and the method of the present invention was obtained, as shown in Tables 2 and 3.

Table 2 The retrieval performance P@5 value of the method of the present invention and the benchmark retrieval and comparison method (Title query)

Table 3 The retrieval performance P@5 value of the method of the present invention and the benchmark retrieval and comparison method (Desc query)

The experimental results show that the P@5 values of the experimental results of the method of the present invention are all higher than the benchmark retrieval. Compared with the four comparison methods, most of the P@5 values of the method of the present invention have been improved and improved, indicating that the method of the present invention expands the retrieval. The performance is higher than that of benchmark retrieval and similar comparison methods. The experimental results show that the method of the invention is effective, can indeed improve the information retrieval performance, and has high application value and broad promotion prospects.

Claims (1)

1. a text retrieval method based on association rule and word vector fusion expansion, is characterized in that, comprises the following steps: Step 1. Chinese users query and retrieve the original Chinese document set to obtain the initial inspection document, and construct the initial inspection document set; Step 2. Use deep learning tools to perform word vector semantic learning training on the initial inspection document set to obtain a feature word word vector set; The deep learning tool of the present invention refers to the Skip-gram model of Google's open source word vector tool word2vec; Step 3. Extract the first m initial inspection documents from the initial inspection document set as pseudo-relevant feedback documents, construct a pseudo-relevant feedback document set, and perform Chinese word segmentation, Chinese stop words removal, and feature word extraction on the initial inspection pseudo-relevant feedback document set. Preprocess, and use TF-IDF weighting technology to calculate the weight of feature words, and finally build a pseudo-relevant feedback Chinese document database and Chinese feature word database; Step 4. Use the support and confidence based on the Copulas function to mine the candidate extension words from the pseudo-related feedback document set, and establish the candidate extension word set. The specific steps are as follows: (4.1) Generate 1_candidate item set C ₁ : extract feature words from Chinese feature vocabulary as 1_ candidate item set C ₁ ; (4.2) Generate 1_frequent itemset L ₁ : calculate the support Cop_Sup(C ₁ ) of C ₁ based on the Copulas function, and extract C ₁ whose Cop_Sup(C ₁ ) is not lower than the minimum support threshold ms as 1_ frequent itemsets L ₁ , and add it to the frequent itemset set FIS (Frequent ItemSet); The calculation of Cop_Sup(C ₁ ) of the 1_candidate item set C ₁ is shown in formula (1):

In formula (1),

Represents the frequency of 1_candidate item set C ₁ appearing in the pseudo-relevant feedback Chinese document library, DocNum indicates the total number of documents in the pseudo-relevant feedback Chinese document library,

Represents the item set weight of 1_candidate item set C ₁ in the pseudo-relevant feedback Chinese document database, and ItemsWeight represents the weight accumulation of all Chinese feature words in the pseudo-relevant feedback Chinese document database; (4.3) Generate k_candidate item set C _k : self-connect (k-1)_frequent itemset L _k-1 to generate k_ candidate item set C _k , where k≥2; The self-connection method adopts the candidate itemset connection method given in the Apriori algorithm; (4.4) 2_candidate item set C ₂ pruning: when mining to 2_ candidate item set C ₂ , if the C ₂ does not contain the original query term, delete the C ₂ , if the C ₂ contains the original query term item, then leave the C ₂ , and then, go to step (4.5) with the remaining C ₂ ; when the k_candidate item set C _k is mined, and the k ≥ 3, then C _k directly goes to step (4.5) ; (4.5) Generate k_frequent itemset L _k : calculate Cop_Sup(C _k ), extract C _k whose Cop_Sup(C _k ) is not lower than ms as k_ frequent itemset L _k , and add it to FIS; The calculation of the Cop_Sup(C _k ) is shown in formula (2):

In formula (2),

represents the frequency of k_candidate item set C _k appearing in the pseudo-relevant feedback Chinese document library,

Represents the item set weight of k_candidate item set C _k in the pseudo-relevant feedback Chinese document library; the definitions of DocNum and ItemsWeight are the same as formula (1); (4.6) After adding 1 to k, go to step (4.3) and continue to execute subsequent steps in sequence until the L _k is an empty set, then the frequent itemset mining ends, and go to step (4.7); (4.7) arbitrarily take out L _k from the FIS, the k ≥ 2; (4.8) Extract the association rule Q _i →Ret _j from L _k , calculate the confidence level Cop_Con(Q _i →Ret _j ) of the association rule Q _i →Ret _j based on the Copulas function, the i≥1, j≥1, and

Q _i ∪Ret _j =L _k ,

The Ret _j is the proper subset itemset without query terms, the Q _i is the proper subset itemsets with query terms, and the Q is the original query term set; The calculation formula of Cop_Con(Q _i →Ret _j ) is shown in formula (3):

In formula (3),

represents the frequency of k_frequent itemsets L _k appearing in the pseudo-relevant feedback Chinese document library,

represents the itemset weight of k_frequent itemset L _k in the pseudo-relevant feedback Chinese document database,

Represents the frequency of the proper subset itemset Q _i of _{k_frequent} itemset L _k appearing in the pseudo-relevant feedback Chinese document database, and weight _Qi represents the proper subset itemset Qi of k_ frequent itemset L _k in the pseudo-relevant feedback Chinese The itemset weight in the document library; exp represents the exponential function with the natural constant e as the base; (4.9) Generate association rules Qi → Ret _j : extract the association rules Qi → Ret _{j whose Cop_Con(Q i → Ret j} ) is not less than the minimum confidence threshold mc _and add them to the association rule set AR, and then go to step (4.8 ), re-extract other proper subset itemsets Ret _j and Q _i from L _k , and then perform subsequent steps in sequence, and so on, until all proper subset itemsets of L _k are taken out once and only if they are taken out once, this Then go to step (4.7), carry out a new round of association rule pattern mining, take out any other L _k from the FIS, and then perform the subsequent steps in sequence, and so on, until all k_frequent itemsets L _k in the FIS are when And only when all are taken out once, then the association rule pattern mining is over, and then go to the following step (4.10); (4.10) generate candidate extension words: extract association rule consequent Ret _j as candidate extension word from association rule set AR, obtain candidate extension word set CETS, calculate candidate extension word weight w _Ret , then, go to step 5; The CETS is shown in formula (4):

In formula (4), Ret _i represents the ith candidate extension word; The calculation formula of the candidate extended word weight w _Ret is shown in formula (5):

In formula (5), max( ) represents the maximum value of the association rule confidence. When the same rule expansion word appears in multiple association rule patterns at the same time, the one with the largest confidence value is taken as the weight of the rule expansion word; Step 5. Calculate the vector cosine similarity between the candidate expansion word and the original query word, and extract the candidate expansion word that is not lower than the similarity threshold as the final expansion word. The specific steps are as follows: (5.1) Calculate the vector similarity VecSim(Ret _i , Q): In the feature word vector set, calculate the candidate extension word Ret _i and the original query term set Q (the Q=(q ₁ ,q ₂ ,...,q _r )) vector cosine similarity VecSim(Ret _i ,q _j ) of each query word {q ₁ ,q ₂ ,...,q _r }, the 1≤j≤r, the candidate expansion word and each query are accumulated The vector similarity of the word is used as the total vector similarity of the candidate expanded word VecSim(Ret _i ,Q); The VecSim(Ret _i ,q _j ) is shown in formula (6):

In formula (6), the vRet _i represents the word vector value of the ith candidate expansion word Ret _i , and vq _j represents the word vector value of the jth query term q _j ; The VecSim(Ret _i , Q) calculation formula is shown in formula (7):

(5.2) Generate the final expanded word: extract the candidate expanded word whose VecSim(Ret _i , Q) is not lower than the vector similarity threshold minVSim as the final expanded word of the original query term set Q, and obtain the final expanded word set FETS, such as formula ( 8) shown:

In formula (8), Ret _i is the ith final extended word; Then the weight w _Fet of the final extended word Ret is calculated, and the weight w _Fet of the final extended word Ret is composed of the candidate extended word weight w _Ret and the vector similarity VecSim(Ret _i , Q); the w _Fet calculation formula is as follows (9) shows:

Step 6. The expanded word and the original query are combined into a new query to retrieve the original Chinese document set again to obtain the final retrieval document; Step 7. Return the final retrieved document set to the user.

Images