CN110909217A - Method and device for realizing search, electronic equipment and storage medium - Google Patents

Abstract

The invention discloses a method and a device for realizing search, electronic equipment and a storage medium. The method comprises the following steps: acquiring a search query statement; extracting a plurality of candidate words from the search query sentence, and generating a plurality of candidate words according to the search query sentence; further generating a rewriting sentence according to the extracted candidate words and the generated candidate words; and searching and inquiring according to the rewriting sentences to obtain a search result. The technical scheme combines two modes of extraction and generation to generate candidate words and further generates rewriting sentences, so that reasonable rewriting of search query sentences input by a user is realized according to semantics and scenes, and the results returned by search are further close to the actual requirements of the user.

Description

Search implementation method, device, electronic device and storage medium

technical field

The present invention relates to the technical field of search, and in particular to a method, apparatus, electronic device and storage medium for realizing search.

Background technique

Generally, search engines are more suitable for entering queries consisting of precise keywords, and queries described by natural language will result in poor returned results. For example, Figure 1 shows a schematic diagram of query results corresponding to different query sentences. As shown in Figure 1, a user may input natural language such as "I want to know how much an iPhone X costs" when searching, especially when This is especially true in the context of language search. But obviously, the search results obtained in this way are unsatisfactory, and if the search terms are changed according to semantics, such as "iPhone X price", the search results are relatively more in line with user needs. Therefore, how to replace the search term is a problem that needs to be solved.

SUMMARY OF THE INVENTION

In view of the above problems, the present invention is proposed to provide a search implementation method, apparatus, electronic device and storage medium that overcome the above problems or at least partially solve the above problems.

According to one aspect of the present invention, a method for realizing search is provided, including: obtaining a search query statement; extracting several candidate words from the search query statement, and generating several candidate words according to the search query statement; According to the extracted candidate words and the generated candidate words, a rewritten sentence is further generated; a search query is performed according to the rewritten sentence, and a search result is obtained.

Optionally, the search query sentence is generated according to the voice input by the user.

Optionally, extracting several candidate words from the search query statement, and generating several candidate words according to the search query statement includes: encoding the search query statement to obtain encoded data; The encoded data is decoded to output a first candidate vocabulary, and the encoded data is decoded in a generative mode to output a second candidate vocabulary.

Optionally, the encoding the search query statement to obtain the encoded data includes: performing word embedding processing on the search query statement to obtain a word vector corresponding to each word included in the search query statement; The vector is encoded to obtain the input hidden vector.

Optionally, the encoding according to each word vector to obtain a latent vector includes: performing the encoding based on a layer of bidirectional long short-term memory network LSTM.

Optionally, the decoding the encoded data in the extraction mode, and outputting the first candidate vocabulary includes: calculating an attention weight at according to the input ^latent vector; calculating the The extraction weight of each word in the search query:

Among them, P _extract (w) is the extraction weight of the target word w, p _w is the adjustment factor, f _w is the number of times the target word w appears in the search query statement, N is the number of all queries in the corpus, |w| is the number of queries containing the target word w in the corpus, and t is time t; the first candidate word list includes one or more words and their corresponding extraction weights.

Optionally, the decoding the encoded data in the generation mode, and outputting the second candidate vocabulary includes: calculating an attention weight at according to the input ^latent vector; according to the attention weight ^at and the input The hidden vector calculates the context weight C ^t ; calculates the distribution probability P _vocab of the second candidate vocabulary according to the attention weight at , the context weight C ^t and the target hidden vector h ^t at the current moment _.

Optionally, the calculating the attention weight at according to the input ^latent vector includes: calculating the attention weight at based on ^formulas (3) and (4):

a ^t =softmax(e ^t ) (4);

的相似程度，

为t时刻的覆盖向量，v、W₁、W₂、W_c和b_atten为预置参数；

为输入隐向量，h_t为输出隐向量。Among them, the function score is used to compare the target hidden vector h _t with the input hidden vector

the degree of similarity,

is the coverage vector at time t, and v, W ₁ , W ₂ , W _c and _batten are preset parameters;

is the input hidden vector, and h _t is the output hidden vector.

Optionally, calculating the context weight C ^t according to the attention weight at and the input latent vector includes: calculating the context weight C ^t based on formulas (5) and (6 ⁾ :

where cov ^t is the coverage matrix at time t.

Optionally, calculating the distribution probability P _vocab of the second candidate vocabulary according to the attention weight a ^t , the context weight C ^t and the target latent vector h _t at the current moment includes: calculating based on formula (7) P _vocab :

P _vocab =f(C ^t ,h _t )=softmax(V'(V[h _t ,C ^t ]+b)+b') (7);

Among them, V, b and V', b' are two-step linear transformation parameter matrix and bias vector.

Optionally, the decoding of the encoded data in the extraction mode, the output of the first candidate vocabulary, and the decoding of the encoded data in the generation mode, and the output of the second candidate vocabulary include: based on a layer of one-way LSTM implement decoding.

Optionally, further generating the rewritten statement according to the extracted candidate words and the generated candidate words includes: according to the extraction weight P _extract of the first candidate vocabulary and the distribution probability P _vocab of the second candidate vocabulary and the adjustment factor p _gen A third candidate vocabulary is determined, and a rewritten sentence is generated according to the third candidate vocabulary.

Optionally, determining the third candidate vocabulary according to the extraction weight P _extract of the first candidate vocabulary, the distribution probability P _vocab of the second candidate vocabulary, and the adjustment factor p _gen includes: calculating the adjustment factor p based on formula (8). _gen :

Wherein, w _h , w _s , w _x and b are preset parameters, x _t is the input search query statement, and σ is the sigmoid function;

Calculate the probability of each candidate word in the third candidate word list based on formula (9):

P(w)=p _gen P _vocab (w)+(1-p _gen )P _extract (w) (9).

According to another aspect of the present invention, a search implementation device is provided, comprising: an obtaining unit, adapted to obtain a search query; a rewriting unit, adapted to extract several candidate words from the search query; The search query statement generates several candidate words; according to the extracted candidate words and the generated candidate words, a rewritten statement is further generated; the search unit is adapted to perform a search query according to the rewritten statement to obtain a search result.

Optionally, the search query sentence is generated according to the voice input by the user.

Optionally, the rewriting unit is adapted to encode the search query statement to obtain encoded data; decode the encoded data in an extraction mode, output a first candidate vocabulary, and encode the encoded data in a generation mode. The data is decoded and the second candidate vocabulary is output.

Optionally, the rewriting unit is adapted to perform word embedding processing on the search query sentence to obtain word vectors corresponding to each word included in the search query sentence; and to obtain an input latent vector by encoding each word vector.

Optionally, the rewriting unit is adapted to perform the encoding based on a one-layer bidirectional long short-term memory network LSTM.

Optionally, the rewriting unit is adapted to calculate the attention weight at according to the input ^latent vector; calculate the extraction weight of each word in the search query sentence based on formulas (1) and (2):

Among them, P _extract (w) is the extraction weight of the target word w, p _w is the adjustment factor, f _w is the number of times the target word w appears in the search query statement, N is the number of all queries in the corpus, |w| is the number of queries containing the target word w in the corpus, and t is time t; the first candidate word list includes one or more words and their corresponding extraction weights.

Optionally, the rewriting unit ^{is adapted to calculate an attention weight at according to the input latent vector; calculate a context weight C t according} to the attention weight ^at and the input latent vector; according to the attention weight at ^t , the context weight C ^t and the target latent vector h _t at the current moment to calculate the distribution probability P _vocab of the second candidate vocabulary.

Optionally, the rewriting unit is adapted to calculate the attention weight at based on ^formulas (3) and (4):

a ^t =softmax(e ^t ) (4);

的相似程度，

为t时刻的覆盖向量，v、W₁、W₂、W_c和b_atten为预置参数；

为输入隐向量，h_t为输出隐向量。Among them, the function score is used to compare the target hidden vector h _t with the input hidden vector

the degree of similarity,

is the coverage vector at time t, and v, W ₁ , W ₂ , W _c and _batten are preset parameters;

is the input hidden vector, and h _t is the output hidden vector.

Optionally, the rewriting unit is adapted to calculate the context weight C ^t based on formulas (5) and (6):

where cov ^t is the coverage matrix at time t.

Optionally, the rewriting unit is adapted to calculate P _vocab based on formula (7):

P _vocab =f(C ^t ,h _t )=softmax(V'(V[h _t ,C ^t ]+b)+b') (7);

Among them, V, b and V', b' are two-step linear transformation parameter matrix and bias vector.

Optionally, the rewriting unit is adapted to implement decoding based on a layer of unidirectional LSTM.

Optionally, the rewriting unit is adapted to determine the third candidate vocabulary according to the extraction weight P _extract of the first candidate vocabulary, the distribution probability P _vocab of the second candidate vocabulary and the adjustment factor p _gen , and according to the third candidate vocabulary Table generation rewrite statement.

Optionally, the rewriting unit is adapted to calculate the adjustment factor p _gen based on formula (8):

Among them, w _h , w _s , w _x and b are preset parameters, x _t is the input search query statement, and σ is the sigmoid function; the probability of each candidate word in the third candidate word list is calculated based on formula (9):

P(w)=p _gen P _vocab (w)+(1-p _gen )P _extract (w) (9).

According to yet another aspect of the present invention, there is provided an electronic device comprising: a processor; and a memory arranged to store computer-executable instructions that, when executed, cause the processor to perform any of the above a described method.

According to yet another aspect of the present invention, a computer-readable storage medium is provided, wherein the computer-readable storage medium stores one or more programs, the one or more programs, when executed by a processor, implements the following: Any of the methods described above.

It can be seen from the above that the technical scheme of the present invention extracts several candidate words from the search query statement after acquiring the search query statement, generates several candidate words according to the search query statement, and generates several candidate words according to the extracted candidate words and the generated candidate words. word, further generate a rewritten sentence, perform a search query according to the rewritten sentence, and obtain a search result. The technical solution combines two methods of extraction and generation to generate candidate words, and further generates rewriting sentences, which realizes reasonable rewriting of search query sentences input by users according to semantics and scenarios, and further makes the results returned by the search closer to the actual needs of users. .

The above description is only an overview of the technical solutions of the present invention, in order to be able to understand the technical means of the present invention more clearly, it can be implemented according to the content of the description, and in order to make the above and other purposes, features and advantages of the present invention more obvious and easy to understand , the following specific embodiments of the present invention are given.

Description of drawings

Various other advantages and benefits will become apparent to those of ordinary skill in the art upon reading the following detailed description of the preferred embodiments. The drawings are for the purpose of illustrating preferred embodiments only and are not to be considered limiting of the invention. Also, the same components are denoted by the same reference numerals throughout the drawings. In the attached image:

1 shows a schematic diagram of query results corresponding to different query statements;

2 shows a schematic flowchart of a method for implementing a search according to an embodiment of the present invention;

3 shows a schematic structural diagram of a query rewriting model according to an embodiment of the present invention;

4 shows a schematic structural diagram of an apparatus for implementing a search according to an embodiment of the present invention;

FIG. 5 shows a schematic structural diagram of an electronic device according to an embodiment of the present invention;

FIG. 6 shows a schematic structural diagram of a computer-readable storage medium according to an embodiment of the present invention.

Detailed ways

Exemplary embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. While exemplary embodiments of the present disclosure are shown in the drawings, it should be understood that the present disclosure may be embodied in various forms and should not be limited by the embodiments set forth herein. Rather, these embodiments are provided so that the present disclosure will be more thoroughly understood, and will fully convey the scope of the present disclosure to those skilled in the art.

FIG. 2 shows a schematic flowchart of a method for implementing a search according to an embodiment of the present invention. As shown in Figure 2, the method includes:

Step S210, obtaining a search query statement.

Step S220, extracting several candidate words from the search query statement, and generating several candidate words according to the search query statement.

Extractive rewriting can use a specific calculation rule to calculate the weight of each word in the search query sentence input by the user, and select a word with a significant weight as a keyword. This method is simple and convenient, but all keywords are limited to the input word set, and words with high frequency tend to be extracted, which is less effective in some cases.

Generative rewriting can "understand" the user's input and then generate some keywords based on the user's intent. This method can generate new words, but the generation process is often uncontrollable, and it also generates some completely wrong words.

Taking the search scenario shown in Figure 1 as an example, the extraction method will extract "iPhone X" and "how many", both of which come from the search query sentences entered by the user and are insufficient to summarize their intent. The method of generating patterns will produce different results depending on the training corpus. For example, "iPhone 8" and "Price" are generated. Although new words can be generated, the model will calculate the probability based on the vocabulary made by the training corpus when generating the words. If "iPhone X" is not in the training corpus, it can only be replaced by a wrong synonym. Such results can lead to search page errors.

Therefore, the above two methods have their own advantages and disadvantages, and only one method to rewrite the query often does not get better results. In the embodiment of the present invention, the combination of these two methods is of great significance for query rewriting, and the finally generated rewriting statement is also more accurate.

Step S230, further generating a rewritten sentence according to the extracted candidate words and the generated candidate words.

In step S240, a search query is performed according to the rewritten sentence, and a search result is obtained.

It can be seen that in the method shown in FIG. 2, after the search query statement is obtained, several candidate words are extracted from the search query statement, and several candidate words are generated according to the search query statement, and according to the extracted candidate words and the generated candidate words , further generate a rewritten statement, perform a search query according to the rewritten statement, and obtain a search result. The technical solution combines two methods of extraction and generation to generate candidate words, and further generates rewriting sentences, which realizes reasonable rewriting of search query sentences input by users according to semantics and scenarios, and further makes the results returned by the search closer to the actual needs of users. .

In an embodiment of the present invention, in the above method, the search query sentence is generated according to the voice input by the user.

Voice search has been supported by many search engines. In fact, it can be implemented by using existing search technology by converting input voice into text. In this case, the search query statement is more colloquial, and it is more difficult to obtain the result desired by the user according to the search query statement.

In an embodiment of the present invention, in the above method, extracting several candidate words from the search query statement, and generating several candidate words according to the search query statement includes: encoding the search query statement to obtain encoded data; extracting The mode decodes the encoded data and outputs the first candidate vocabulary, and the generation mode decodes the encoded data and outputs the second candidate vocabulary.

Here, a query rewriting model can be used to implement, specifically, the query rewriting model can be implemented with reference to the sequence-to-sequence model (seq2seq). seq2seq is a network with an "Encoder-Decoder" structure. Its input is a sequence, and its output is also a sequence. The encoder converts a variable-length signal sequence into a fixed-length vector representation, decoding The processor turns this fixed-length vector into a variable-length target signal sequence.

In the model structure of seq2seq, when each word is output, the attention to the words in the input is inconsistent, and its weight is calculated according to specific rules. This can make the generated sequence more reasonable and retain most of the information in the input, which is also called the attention mechanism. Generally, in natural language processing applications, the attention model is regarded as the alignment model of a word in the output sentence and each word of the input sentence.

Under the seq2seq model, when the user inputs a query x={x ₁ ,...,x _n } (x _i represents the ith word of the input sentence), the goal of the model is to convert this query into semantically similar keywords Query y={y ₁ ,...,y _m } (y _i represents the ith word of the output). In this model, each word of the query is fed into the "encoder" in turn, and then the "decoder" receives the previously generated words {y ₁ ,...,y _t-1 } and a context vector C to predict The next word y _t . The formula is as follows:

where p(y _t |{y ₁ ,...,y _t-1 },C) represents the premise that the previously generated words {y ₁ ,...,y _t-1 } and a context vector C are known below, the conditional probability of y _t . The context vector C can be calculated by the attention mechanism, and a score method is used to measure the weight of each word in the input and calculate its weighted sum.

When calculating attention weights, the present invention also uses a coverage mechanism to prevent the problem of duplication of generated results. The specific implementations of the encoder and decoder of the model of the present invention are described below. FIG. 3 shows a schematic structural diagram of a query rewriting model according to an embodiment of the present invention. It can be seen that the decoder of the query rewriting model is divided into two modes, and the distribution of the output words is determined by an adjustment factor. The example in Fig. 3 is that the query sentence is "Tell meiPhone X cost", that is, tell me how much the iPhone X is, and two generated candidate words "iPhone X" and "price (price)" are also shown.

Referring to Figure 3, the first candidate vocabulary actually shows the distribution of each word in the input sentence, that is, the extraction weight (marked as input distribution in the figure); the second candidate vocabulary actually shows the generated distribution of each word , that is, the distribution probability of the second candidate vocabulary (marked as vocabulary distribution in the figure). In an embodiment of the present invention, the decoder of the query rewriting model may include an extraction mode and a generation mode.

In an embodiment of the present invention, in the above method, encoding the search query statement to obtain encoded data includes: performing word embedding processing on the search query statement to obtain a word vector corresponding to each word contained in the search query statement; The word vector is encoded to obtain the input latent vector.

Word embedding can form the word vector representation of the sentence, that is, the words in the sentence are represented as a vector. Feeding each word into the encoder one by one can generate a hidden vector h _s . The latent vector serves as a high-level representation of the input sentence and is used in the decoding stage to generate new sequences. In an embodiment of the present invention, in the above method, encoding according to each word vector to obtain a latent vector includes: encoding based on a layer of bidirectional long short-term memory network LSTM. LSTM (Long Short-Term Memory) is a temporal recurrent neural network suitable for processing and predicting important events with relatively long intervals and delays in time series.

In an embodiment of the present invention, in the above method, decoding the encoded data in the extraction mode, and outputting the first candidate vocabulary includes: calculating the attention weight at according to the input ^latent vector; based on formulas (1) and (2) Calculate the extraction weight of each word in the search query:

Among them, P _extract (w) is the extraction weight of the target word w, p _w is the adjustment factor, f _w is the number of times the target word w appears in the search query sentence, N is the number of all queries in the corpus, and |w| is the corpus contains the number of queries of the target word w, and t is time t; the first candidate word list includes one or more words and their corresponding extraction weights.

TF-IDF is the product of two statistics, word frequency tf(w) and inverse document frequency idf(w). The high TF-IDF is jointly determined by the high frequency of the word and the low frequency of the word in the whole corpus, so this method can be used to exclude common terms. For natural language queries, this method can effectively remove some common colloquial descriptions, such as "how", "what", and retain important information.

TF-IDF values and attention weights have different emphases in measuring word importance. Attention weights focus on the semantic matching of input and output, using the hidden state to calculate their similarity value. In this way, it focuses on the "meaning" of the word. TF-IDF focuses on the statistical features of words, it counts the importance of the word in the entire corpus, and these two values describe the importance of the input word from different perspectives. By combining them with weighting factors, better keywords can be extracted from the input.

In an embodiment of the present invention, in the above method, decoding the encoded data in the generation mode, and outputting the second candidate vocabulary includes: calculating the attention weight at according to the input ^latent vector; according to the attention weight ^at and the input hidden vector The vector calculates the context weight C ^t ; calculates the distribution probability P _vocab of the second candidate vocabulary according to the attention weight a ^t , the context weight C ^t and the target latent vector h _t at the current moment.

Specifically, in an embodiment of the present invention, in the above method, calculating the attention weight at according to the input ^latent vector includes: calculating the attention weight at based on ^formulas (3) and (4):

a ^t =softmax(e ^t ) (4);

的相似程度，cov_i ^t为t时刻的覆盖向量，v、W₁、W₂、W_c和b_atten为预置参数；

为输入隐向量，h_t为输出隐向量。其中，cov⁰是一个全零矩阵。还需要说明的是Softmax函数的意义是将K维实数向量z映射到一个新的K维实数向量σ(z)，使得向量的每一个元素值都在0-1之间，且所有元素和为1。Among them, the function score is used to compare the target hidden vector h _t with the input hidden vector

The similarity degree of , cov _i ^t is the coverage vector at time t, v, W ₁ , W ₂ , W _c and _batten are preset parameters;

is the input hidden vector, and h _t is the output hidden vector. where cov ⁰ is an all-zero matrix. It should also be noted that the meaning of the Softmax function is to map the K-dimensional real vector z to a new K-dimensional real vector σ(z), so that the value of each element of the vector is between 0-1, and the sum of all elements is 1.

In an embodiment of the present invention, in the above method, calculating the context weight C ^t according to the attention weight at and the input latent vector includes: calculating the context weight C ^t based on formulas (5) and (6 ⁾ :

where cov ^t is the coverage matrix at time t. That is, at time t, a coverage matrix cov ^t is maintained to record the coverage of the words in the input. It is the sum of the attention distributions of all previous moments, and the context vector C is obtained by the weighted summation of the input latent vector through the attention weight at ^t .

In an embodiment of the present invention, in the above method, calculating the distribution probability P _vocab of the second candidate vocabulary according to the attention weight a ^t , the context weight C ^t and the target latent vector h _t at the current moment includes: based on formula (7) Calculate P _vocab :

P _vocab =f(C ^t ,h _t )=softmax(V'(V[h _t ,C ^t ]+b)+b') (7);

Among them, V, b and V', b' are two-step linear transformation parameter matrix and bias vector.

That is, after obtaining the context vector C, combine it with the target latent vector h _t at the current moment to obtain the distribution probability P _vocab of the vocabulary through two fully connected layers.

In an embodiment of the present invention, in the above method, decoding the encoded data in the extraction mode, outputting the first candidate vocabulary, and decoding the encoded data in the generation mode, and outputting the second candidate vocabulary includes: based on a layer of One-way LSTM implements decoding.

To sum up, the decoder receives the input word vector representation and the decoder's hidden vector h _t , and calculates the probability of each word in the vocabulary through the attention mechanism, and selects the word with the highest probability as the output. This method corresponds to generating Mode; the weight of each word in the input sentence is calculated by the attention matrix and the extraction method, and the word with the most weight is selected as the output, which corresponds to the extraction mode.

In an embodiment of the present invention, in the above method, further generating a rewritten sentence according to the extracted candidate words and the generated candidate words includes: according to the extraction weight P _extract of the first candidate vocabulary and the distribution probability of the second candidate vocabulary P _vocab and the adjustment factor p _gen determine a third candidate vocabulary, and generate a rewritten sentence according to the third candidate vocabulary.

In an embodiment of the present invention, in the above method, determining the third candidate vocabulary according to the extraction weight P _extract of the first candidate vocabulary, the distribution probability P _vocab of the second candidate vocabulary, and the adjustment factor p _gen includes: based on the formula (8) Calculate the adjustment factor p _gen :

Wherein, w _h , w _s , w _x and b are preset parameters, x _t is the input search query statement, and σ is the sigmoid function;

Calculate the probability of each candidate word in the third candidate word list based on formula (9):

P(w)=p _gen P _vocab (w)+(1-p _gen )P _extract (w) (9).

This results in the final distribution shown in Figure 3.

The following briefly introduces the training process of the query rewriting model used in the embodiment of the present invention.

The construction method of the query rewrite model can include the following steps:

Generate training data based on search click data. Here, the search click data in the commercial search engine can be used as the data source, and the search records of high-quality users are preferably selected as the initial training corpus. The training data can be obtained after cleaning the initial training corpus.

The target model is trained according to the training data to obtain an intermediate model; the target model includes an encoder and a decoder, and the decoder includes an extraction mode and a generation mode. The extraction mode and the generation mode have been introduced above, and the specific implementation process may refer to the foregoing embodiments.

It is judged whether the intermediate model satisfies the preset conditions. If yes, the intermediate model is used as the query to rewrite the model and the training is stopped. Otherwise, the intermediate model is used as the target model for iterative training.

In an embodiment of the present invention, in the above method, generating training data according to the search click data includes: extracting several groups of sentence pairs from the search click data; the sentence pairs include search query sentences and headline sentences corresponding to the click search results.

The sentence pair (query-title) describes the search query sentence input by the user, that is, the displayed demand, and describes the content of the actual click, that is, the actual demand. Taking Figure 1 as an example, when the input search query is "iPhone X price" as an example, when the user actually clicks the first search result, the sentence pair is "iPhone X price-Apple iPhoneX full Netcom quotation parameter picture forum Zhongguancun Online".

In fact, the search query sentence and the title sentence here are not necessarily complete sentences that read fluently, but may only contain several isolated words, which are collectively described as "sentences" in the present invention for the convenience of introduction.

In an embodiment of the present invention, the above method further includes: removing noise from the extracted sentence pairs. These noises are mainly caused by the user's misoperation or by coincidentally being interested in a certain page. Specifically, sentences that generally appear in sentence pairs do not match semantically, and these noises can seriously affect the training process of the model.

In an embodiment of the present invention, in the above method, removing noise from the extracted sentence pairs includes: calculating the subject similarity between the search query sentence and the title sentence in each sentence pair, and/or calculating the search query in each sentence pair Semantic similarity between query sentences and title sentences; remove noise according to a preset similarity threshold. In the specific implementation, reference may be made to the following two embodiments, but it should be understood that it is not limited to the following calculation methods of similarity: The topic similarity of query sentences and headline sentences includes: semantic representation of search query sentences and headline sentences, training a latent Dirichlet distribution LDA topic model and calculating the topic distribution of search query sentences and the topic distribution of headline sentences, based on JS divergence The distribution similarity of the topic distribution of the search query sentence and the title sentence in each sentence pair is calculated. In an embodiment of the present invention, in the above method, calculating the semantic similarity between the search query statement and the title sentence in each sentence pair includes: determining the word vector of the words in the search query statement and the title sentence, and combining the search query sentence and the title sentence They are respectively expressed as the mean of the word vectors, and the similarity between the search query sentence and the title sentence in each sentence pair is calculated based on the cosine similarity.

Among them, topic similarity starts from the topic distribution of sentences, and calculates the similarity between distributions. The sentence is first semantically represented, the LDA model is trained and the topic distribution of a sentence is calculated. The similarity between the two distributions is then calculated using JS (Jensen-Shannon) divergence. Semantic similarity starts with the word vector of the words in the sentence, expresses a sentence as the mean of the word vectors of the words in the sentence, and then uses the cosine similarity to calculate the similarity of the two sentences. By setting a reasonable threshold, the purpose of removing noise is achieved. LDA (Latent Dirichlet Allocation) is a document topic generation model, also known as a three-layer Bayesian probability model, which includes a three-layer structure of words, topics and documents. The so-called generative model means that we think that each word of an article is obtained through a process of "selecting a topic with a certain probability, and selecting a word from this topic with a certain probability". Documents to topics follow a multinomial distribution, and topics to words follow a multinomial distribution.

In an embodiment of the present invention, in the above method, generating training data according to the search click data further includes: performing word segmentation on the search query sentence and the title sentence in the sentence pair respectively; dividing the first proportion of data from the word segmentation result as The validation set is divided into a second proportion of data as a training data set; a training vocabulary is generated based on the training data set.

For example, use the jieba word segmentation tool to tokenize sentences and separate sentences by words. In a specific scenario, 20% of the data is divided as the validation set, the remaining 80% of the data is used as the training data set, and a training vocabulary is made according to the training data set. This prepares the training data.

In an embodiment of the present invention, in the above method, training the target model according to the training data, and obtaining the intermediate model includes: dividing the data in the training data set into multiple sets of training sample data; The vocabulary lists the number of the training sample data, selects the title sentence as the training input data, and selects the corresponding search query sentence as the training output data.

For example, the training data is randomly scrambled, and divided into S groups on average, set s=0 (the corresponding serial numbers of each group are 0, 1, 2...s-1); take the s-th training sample data, according to the constructed The training vocabulary lists the words of each sentence in the selected training sample data, and sends them to the target model for training. If the obtained intermediate model satisfies the preset condition after the training is completed, the training is ended; if the preset condition is not met, s=s+1 is set, and the training is repeated until the obtained intermediate model satisfies the preset condition.

In an embodiment of the present invention, in the above method, judging whether the intermediate model satisfies the preset condition includes: in the training process, calculating the loss loss _t at time t according to formula (10):

为目标词语，

为注意力权重，

为覆盖向量；in,

is the target word,

is the attention weight,

is the coverage vector;

Calculate the loss of the entire sentence according to formula (11):

The loss of the validation set is calculated according to the intermediate model, and if the loss increases, the preset conditions are met.

In the above embodiment, each preset parameter, the two-step linear transformation parameter matrix and the bias vector may be obtained according to model training.

Fig. 4 shows a schematic structural diagram of an apparatus for implementing a search according to an embodiment of the present invention. As shown in FIG. 4 , the apparatus 400 for realizing the search includes:

The obtaining unit 410 is adapted to obtain a search query statement.

The rewriting unit 420 is adapted to extract several candidate words from the search query sentence, and generate several candidate words according to the search query sentence; further generate a rewriting sentence according to the extracted candidate words and the generated candidate words.

The search unit 430 is adapted to perform a search query according to the rewritten sentence to obtain a search result.

It can be seen that the device shown in FIG. 4, after acquiring the search query statement, extracts several candidate words from the search query statement, and generates several candidate words according to the search query statement, and generates several candidate words according to the extracted candidate words and the generated candidate words. , further generate a rewritten statement, perform a search query according to the rewritten statement, and obtain a search result. The technical solution combines two methods of extraction and generation to generate candidate words, and further generates rewriting sentences, which realizes reasonable rewriting of search query sentences input by users according to semantics and scenarios, and further makes the results returned by the search closer to the actual needs of users. .

In an embodiment of the present invention, in the above device, the search query sentence is generated according to the voice input by the user.

In an embodiment of the present invention, in the above-mentioned apparatus, the rewriting unit 420 is adapted to encode the search query sentence to obtain encoded data; decode the encoded data in the extraction mode, output the first candidate vocabulary, and use the generation mode to decode the encoded data The encoded data is decoded, and the second candidate vocabulary is output.

In an embodiment of the present invention, in the above device, the rewriting unit 420 is adapted to perform word embedding processing on the search query sentence to obtain word vectors corresponding to each word contained in the search query sentence; encoding according to each word vector, obtaining Input latent vector.

In an embodiment of the present invention, in the above apparatus, the rewriting unit 420 is adapted to perform coding based on a one-layer bidirectional long short-term memory network LSTM.

In an embodiment of the present invention, in the above device, the rewriting unit 420 is adapted to calculate the attention weight at according to the input ^latent vector; calculate the extraction weight of each word in the search query sentence based on formulas (1) and (2):

Among them, P _extract (w) is the extraction weight of the target word w, p _w is the adjustment factor, f _w is the number of times the target word w appears in the search query sentence, N is the number of all queries in the corpus, and |w| is the corpus contains the number of queries of the target word w, and t is time t; the first candidate word list includes one or more words and their corresponding extraction weights.

In an embodiment of the present invention, in the above device, the rewriting unit 420 ^is adapted to calculate the attention weight at according to the input latent vector; calculate the context weight C ^t according to the attention weight ^at and the input latent vector; according to the attention weight at ^t , the context weight C ^t and the target latent vector h _t at the current moment to calculate the distribution probability P _vocab of the second candidate vocabulary.

In an embodiment of the present invention, in the above apparatus, the rewriting unit 420 is adapted to calculate the attention weight at based on ^formulas (3) and (4):

a ^t =softmax( ^et )(4);

的相似程度，cov_i ^t为t时刻的覆盖向量，v、W₁、W₂、W_c和b_atten为预置参数；

为输入隐向量，h_t为输出隐向量。Among them, the function score is used to compare the target hidden vector h _t with the input hidden vector

The similarity degree of , cov _i ^t is the coverage vector at time t, v, W ₁ , W ₂ , W _c and _batten are preset parameters;

is the input hidden vector, and h _t is the output hidden vector.

In an embodiment of the present invention, in the above apparatus, the rewriting unit 420 is adapted to calculate the context weight C ^t based on formulas (5) and (6):

where cov ^t is the coverage matrix at time t.

In an embodiment of the present invention, in the above device, the rewriting unit 420 is adapted to calculate P _vocab based on formula (7):

P _vocab =f(C ^t ,h _t )=softmax(V'(V[h _t ,C ^t ]+b)+b') (7);

Among them, V, b and V', b' are two-step linear transformation parameter matrix and bias vector.

In an embodiment of the present invention, in the above apparatus, the rewriting unit 420 is adapted to implement decoding based on a layer of unidirectional LSTM.

In an embodiment of the present invention, in the above device, the rewriting unit 420 is adapted to determine the third candidate according to the extraction weight P _extract of the first candidate vocabulary, the distribution probability P _vocab of the second candidate vocabulary, and the adjustment factor p _gen The vocabulary list is used to generate a rephrased sentence according to the third candidate vocabulary list.

In an embodiment of the present invention, in the above apparatus, the rewriting unit 420 is adapted to calculate the adjustment factor p _gen based on formula (8):

Wherein, w _h , w _s , w _x and b are preset parameters, x _t is the input search query statement, and σ is the sigmoid function;

Calculate the probability of each candidate word in the third candidate word list based on formula (9):

P(w)=p _gen P _vocab (w)+(1-p _gen )P _extract (w) (9).

It should be noted that, the specific implementations of the foregoing apparatus embodiments may be performed with reference to the specific implementations of the foregoing corresponding method embodiments, and details are not described herein again.

To sum up, the technical solution of the present invention, after obtaining the search query sentence, extracts several candidate words from the search query sentence, and generates several candidate words according to the search query sentence, according to the extracted candidate words and the generated candidate words, further generate a rewritten sentence, perform a search query according to the rewritten sentence, and obtain a search result. The technical solution combines two methods of extraction and generation to generate candidate words, and further generates rewriting sentences, which realizes reasonable rewriting of search query sentences input by users according to semantics and scenarios, and further makes the results returned by the search closer to the actual needs of users. .

It should be noted:

The algorithms and displays provided herein are not inherently related to any particular computer, virtual appliance, or other device. Various general-purpose devices can also be used with the teachings based on this. The structure required to construct such a device is apparent from the above description. Furthermore, the present invention is not directed to any particular programming language. It is to be understood that various programming languages may be used to implement the inventions described herein, and that the descriptions of specific languages above are intended to disclose the best mode for carrying out the invention.

In the description provided herein, numerous specific details are set forth. It will be understood, however, that embodiments of the invention may be practiced without these specific details. In some instances, well-known methods, structures and techniques have not been shown in detail in order not to obscure an understanding of this description.

Similarly, it is to be understood that in the above description of exemplary embodiments of the invention, various features of the invention are sometimes grouped together into a single embodiment, figure, or its description. This disclosure, however, should not be construed as reflecting an intention that the invention as claimed requires more features than are expressly recited in each claim. Rather, as the following claims reflect, inventive aspects lie in less than all features of a single foregoing disclosed embodiment. Thus, the claims following the Detailed Description are hereby expressly incorporated into this Detailed Description, with each claim standing on its own as a separate embodiment of this invention.

Those skilled in the art will understand that the modules in the device in the embodiment can be adaptively changed and arranged in one or more devices different from the embodiment. The modules or units or components in the embodiments may be combined into one module or unit or component, and further they may be divided into multiple sub-modules or sub-units or sub-assemblies. All features disclosed in this specification (including accompanying claims, abstract and drawings) and any method so disclosed may be employed in any combination, unless at least some of such features and/or procedures or elements are mutually exclusive. All processes or units of equipment are combined. Each feature disclosed in this specification (including accompanying claims, abstract and drawings) may be replaced by alternative features serving the same, equivalent or similar purpose, unless expressly stated otherwise.

Furthermore, those skilled in the art will appreciate that although some of the embodiments described herein include certain features, but not others, included in other embodiments, that combinations of features of different embodiments are intended to be within the scope of the invention within and form different embodiments. For example, in the following claims, any of the claimed embodiments may be used in any combination.

Various component embodiments of the present invention may be implemented in hardware, or in software modules running on one or more processors, or in a combination thereof. Those skilled in the art should understand that a microprocessor or a digital signal processor (DSP) may be used in practice to implement some or all of the functions of some or all of the components in the apparatus for implementing the search according to the embodiment of the present invention. The present invention can also be implemented as apparatus or apparatus programs (eg, computer programs and computer program products) for performing part or all of the methods described herein. Such a program implementing the present invention may be stored on a computer-readable medium, or may be in the form of one or more signals. Such signals may be downloaded from Internet sites, or provided on carrier signals, or in any other form.

For example, FIG. 5 shows a schematic structural diagram of an electronic device according to an embodiment of the present invention. The electronic device includes a processor 510 and a memory 520 arranged to store computer-executable instructions (computer-readable program code). The memory 520 may be electronic memory such as flash memory, EEPROM (Electrically Erasable Programmable Read Only Memory), EPROM, hard disk, or ROM. The memory 520 has storage space 530 for storing computer readable program code 531 for performing any of the method steps in the above-described methods. For example, the storage space 530 for storing computer-readable program code may include various computer-readable program codes 531 for implementing various steps in the above methods, respectively. Computer readable program code 531 can be read from or written to one or more computer program products. These computer program products include program code carriers such as hard disks, compact disks (CDs), memory cards or floppy disks. Such a computer program product is typically a computer-readable storage medium as described in FIG. 6 . FIG. 6 shows a schematic structural diagram of a computer-readable storage medium according to an embodiment of the present invention. The computer-readable storage medium 600 stores computer-readable program code 531 for performing the method steps according to the present invention, which can be read by the processor 510 of the electronic device 500 when the computer-readable program code 531 is executed by the electronic device 500 , causing the electronic device 500 to execute each step in the above-described method. Specifically, the computer-readable program code 531 stored in the computer-readable storage medium can execute the method shown in any of the above-described embodiments. The computer readable program code 531 may be compressed in a suitable form.

It should be noted that the above-described embodiments illustrate rather than limit the invention, and that alternative embodiments may be devised by those skilled in the art without departing from the scope of the appended claims. In the claims, any reference signs placed between parentheses shall not be construed as limiting the claim. The word "comprising" does not exclude the presence of elements or steps not listed in a claim. The word "a" or "an" preceding an element does not exclude the presence of a plurality of such elements. The invention can be implemented by means of hardware comprising several different elements and by means of a suitably programmed computer. In a unit claim enumerating several means, several of these means may be embodied by one and the same item of hardware. The use of the words first, second, and third, etc. do not denote any order. These words can be interpreted as names.

Claims (10)

1. A method for realizing search, comprising: Get the search query; Extracting several candidate words from the search query statement, and generating several candidate words according to the search query statement; According to the extracted candidate words and the generated candidate words, further generate a rewritten sentence; A search query is performed according to the rewritten statement, and a search result is obtained. 2. The method of claim 1, wherein the search query statement is generated based on a user's input speech. 3. The method according to claim 1 or 2, wherein the extracting several candidate words from the search query statement, and generating several candidate words according to the search query statement comprises: Encoding the search query statement to obtain encoded data; The encoded data is decoded in extraction mode to output a first candidate vocabulary, and the encoded data is decoded in generation mode to output a second candidate vocabulary. 4. The method according to any one of claims 1-3, wherein the encoding the search query statement to obtain encoded data comprises: performing word embedding processing on the search query statement to obtain word vectors corresponding to each word contained in the search query statement; According to the encoding of each word vector, the input latent vector is obtained. 5. A device for realizing search, comprising: an obtaining unit, adapted to obtain a search query statement; A rewriting unit, adapted to extract several candidate words from the search query statement, and generate several candidate words according to the search query statement; further generate a rewriting statement according to the extracted candidate words and the generated candidate words; The search unit is adapted to perform a search query according to the rewritten sentence to obtain a search result. 6. The apparatus of claim 5, wherein the search query sentence is generated according to a voice input by a user. 7. The apparatus of claim 5 or 6, wherein, The rewriting unit is adapted to encode the search query statement to obtain encoded data; decode the encoded data in an extraction mode, output a first candidate vocabulary, and decode the encoded data in a generation mode, Output the second candidate vocabulary. 8. The device of any one of claims 5-7, wherein, The rewriting unit is adapted to perform word embedding processing on the search query sentence to obtain a word vector corresponding to each word contained in the search query sentence; encode according to each word vector to obtain an input latent vector. 9. An electronic device, wherein the electronic device comprises: a processor; and a memory arranged to store computer-executable instructions which, when executed, cause the processor to perform as claimed in claims 1-4 The method of any of the above. 10. A computer-readable storage medium, wherein the computer-readable storage medium stores one or more programs that, when executed by a processor, implement any one of claims 1-4 method described in item.

Images