CN108415903A - Judge evaluation method, storage medium and the equipment of search intention identification validity - Google Patents

Abstract

The present invention provides an evaluation method for judging the effectiveness of search intent recognition, comprising the following steps: S1, obtaining the total number of word segments and the information of each word segment in the search intent recognition process to be evaluated; S2, calculating respectively according to the information The sum of the word frequencies corresponding to each of the word segments and the inverted document frequency; S3, using the sum of the word frequencies corresponding to each of the word segments obtained in step S2 and the inverted document frequency to calculate the search intent recognition process to be evaluated Effectiveness score, and then determine whether the search intent identification process is effective. The invention also relates to related computer-readable storage media and electronic devices.

Description

Evaluation method, storage medium and equipment for judging effectiveness of search intent recognition

technical field

The invention relates to the field of big data search, in particular to an evaluation method for judging the effectiveness of search intent recognition, related storage media and electronic equipment.

Background technique

On the live broadcast platform, users' real intentions can be guessed based on their search queries, and more accurate search results can be returned through their real intentions. However, it needs to be measured to what extent the result can reflect the real intention of the user. If the correlation is poor, the actual utility of the identified weak intention will be very low. Therefore, the problem we need to solve is how to measure the relevance of intent matching, so as to judge the effectiveness of intent recognition.

Unlike the scenario where the results are returned directly based on the text matching of the search terms, there may be no textual correlation between the search results returned after using the intent recognition algorithm and the search terms, so it is very important to use the text edit distance to measure the correlation. one-sided.

Therefore, it is necessary to propose a new evaluation method for judging the effectiveness of search intent recognition.

Contents of the invention

In view of this, in order to overcome at least one aspect of the above problems, an embodiment of the present invention provides an evaluation method for judging the effectiveness of search intent recognition based on TF-IDF.

According to one aspect of the present invention, an evaluation method for judging the effectiveness of search intent recognition is provided, comprising steps:

S1, obtaining the total number of word segmentations and the information of each word segmentation in the search intent recognition process to be evaluated;

S2. According to the information, respectively calculate the sum of the word frequencies corresponding to each word segment and the inverted document frequency;

S3, using the sum of word frequencies corresponding to each word segment obtained in step S2 and the inverted document frequency to calculate the effectiveness score of the search intent recognition process to be evaluated, and then determine whether the search intent recognition process is effective.

For example, the information includes the intent domain matched by each participle, where each intent domain has a preset weight; the number of times each participle is matched in the intent domain; The total number of searches within the segment, and the number of searches that include each of the word segments in the total number of searches.

For example, calculate the sum of word frequencies corresponding to each word segment according to the following formula:

Among them, H is a set of matched intent domains, which is composed of multiple different intent domains, and f is one of the intent domains;

is the number of times the participle t _i can be matched in the intent domain f;

n _f is the number of words in the intent domain f;

w _f is the weight of the intent domain f.

For example, calculate the inverted document frequency corresponding to each word segment according to the following formula

Wherein, N is the total number of searches by the user within a preset time period; N(t _i ) is the number of searches including each word t _i , and log is the natural logarithm.

For example, the effectiveness score R of the search intent recognition process to be evaluated is calculated according to the following formula:

Among them, n is the total number of word segmentation.

Further, step S3 further includes:

Comparing the validity score with a preset threshold, if the validity score is greater than the preset threshold, it is determined that the search intent recognition process is effective; if the validity score is smaller than the preset threshold, it is determined that the search intent recognition process is Invalid.

The present invention also provides a computer-readable storage medium on which executable instructions are stored, wherein when the instructions are executed by a processor, any of the above-mentioned evaluations for judging the effectiveness of search intent recognition can be realized method steps.

The present invention also provides an electronic device, comprising:

memory for storing executable instructions; and

A processor, configured to execute the executable instructions stored in the memory, so as to implement the steps of any evaluation method for judging the effectiveness of search intent recognition as described above.

Compared with the prior art, the present invention can scientifically and accurately judge whether the search intention recognition is effective, and solves the problem that the traditional correlation evaluation method cannot be applied.

Description of drawings

Other objects and advantages of the present invention will be apparent from the following description of the present invention with reference to the accompanying drawings, and may help to provide a comprehensive understanding of the present invention.

FIG. 1 is a flowchart of implementation steps of an evaluation method for judging the effectiveness of search intent recognition provided by an embodiment of the present invention;

FIG. 2 is a schematic structural diagram of a computer-readable storage medium provided by an embodiment of the present invention;

FIG. 3 is a schematic structural diagram of an electronic device provided by an embodiment of the present invention.

Detailed ways

In order to make the purpose, technical solution and advantages of the present invention clearer, the technical solution of the present invention will be clearly and completely described below in conjunction with the accompanying drawings of the embodiments of the present invention. Apparently, the described embodiment is one embodiment of the present invention, but not all of them. Based on the described embodiments of the present invention, all other embodiments obtained by persons of ordinary skill in the art without creative efforts shall fall within the protection scope of the present invention.

Unless otherwise defined, the technical terms or scientific terms used in the present invention shall have the usual meanings understood by those skilled in the art to which the present invention belongs.

In this paper, expressing search intent refers to the intended content that the user actually wants to search for, judged according to the user's search query phrase.

Expressive intent domain refers to the intent set that divides the user's search intent according to business experience. Commonly used in live search are anchor intent domain and sub-intent domain. Each intent domain consists of several index words.

According to one aspect of the present invention, an evaluation method for judging the effectiveness of search intent recognition is provided, and the specific implementation ideas are as follows:

Based on TF-FID, calculate the word frequency of all the word segmentations obtained in the process of word segmentation in the search intent recognition process and the frequency of occurrence within the preset time period, and thus obtain the evaluation score of the search intent recognition process, so that it can be judged. Whether the identification process is effective.

More specifically, the evaluation method for judging the effectiveness of search intent recognition in the present invention will be described in detail below with reference to the accompanying drawings.

Referring to Fig. 1, the evaluation method for judging the effectiveness of search intent recognition provided by the embodiments of the present invention may include the following steps:

S1, obtaining the total number of word segmentations and the information of each word segmentation in the search intent recognition process to be evaluated;

In this embodiment, the information of each participle may include the intent field matched by each participle. It should be noted that the number and types of the intentional fields matched by each participle may be different, for example, participle t ₁ It can be matched to an intent domain A, and the word segmentation _t2 can be matched to intent domain B and intent domain C. Moreover, different intent domains have preset weights, which can be set based on previous business experience.

The information of each word segment may also include the number of times each word segment can be matched in the respective matched intent domain, that is, the number of times the word segment can be matched to words in the intent domain. For example, word segmentation t ₁ can match words in intent domain A 5 times, while word segmentation t ₂ can match words in intent domain B 2 times and words in intent domain C 3 times.

The information of each participle may also include the number of searches for each participle among the total times of searches by the user within a preset period of time. In this embodiment, the preset time period may be 30 days. Of course, in other embodiments, other time lengths are also possible.

For example, within 30 days, all users have performed a total of 100,000 searches, of which 100 times include participle _t1 , and 200 times include participle _t2 .

S2, according to the information, respectively calculate the sum of the word frequencies corresponding to each word segment and the inverted document frequency;

In this embodiment, the sum of the word frequencies corresponding to each word segmentation can be calculated according to the following formula:

Among them, H is the set of intent domains that all participle can match, which is composed of multiple different intent domains, and f is one of the intent domains;

is the number of times the participle t _i can be matched in the intent domain f;

n _f is the number of words in the intent domain f;

w _f is the weight of the intent domain f.

In this embodiment, the frequency of inverted documents corresponding to each word segment can be calculated according to the following formula

Wherein, N is the total number of searches by the user within a preset time period; N(t _i ) is the number of searches including each word t _i , and log is the natural logarithm.

S3, using the sum of word frequencies corresponding to each word segment obtained in step S2 and the inverted document frequency to calculate the effectiveness score of the search intent recognition process to be evaluated, and then determine whether the search intent recognition process is effective.

In this embodiment, the effectiveness score R of the search intent recognition process to be evaluated can be calculated according to the following formula:

Among them, n is the total number of word segmentation.

In a further preferred embodiment, step S3 may further include:

Compare the validity score with the preset threshold, if the validity score is greater than the preset threshold, it can be judged that the search intent recognition process is effective; if the validity score is smaller than the preset threshold, it can be judged that the search intent recognition process is invalid .

A practical example is given below to specifically illustrate how the present invention judges the evaluation of the effectiveness of search intent recognition.

Assuming that there are three intent domains, the number of words and weights of each intent domain are:

Intent domain A: n _A =1000, w _A =1.0

Intent domain B: n _B = 500, w _B = 0.5

Intent domain C: n _C = 100, w _C = 0.8

In one recognition, it can be divided into two words t ₁ and t ₂ according to word segmentation

Among them, t ₁ matches words in intent domain A 5 times, t ₂ matches words in intent domain B 2 times, and words in intent domain C 1 time.

In 30 days the user has a total of 100,000 searches, 100 of which contain the term _t1 and 200 of which contain the term _t2 .

The intent matching relevance score for this search is then:

5/1000*1.0*log(100000/100)+(2/500*0.5+1/100*0.8)*log(100000/200)＝0.0967

Then compare 0.0967 with the preset threshold to judge whether the search intent recognition is valid.

The evaluation method for judging the effectiveness of search intent recognition proposed in this embodiment can solve the problem that traditional correlation evaluation methods cannot be applied, and can more scientifically and effectively judge the effectiveness of intent recognition.

Based on the same inventive concept, as shown in FIG. 2 , an embodiment of the present invention also provides a computer-readable storage medium 201 on which executable instructions 202 are stored. When executed by one or more processors, the executable instructions 202 , the steps of the evaluation method for judging the effectiveness of search intent recognition in the above embodiment can be implemented.

Based on the same inventive concept, as shown in FIG. 3 , an embodiment of the present invention also provides an electronic device 301, which may include:

memory 310 for storing executable instructions 311; and

The processor 320 is configured to execute the executable instructions 311 stored in the memory 310, so as to implement the steps of any evaluation method for judging the effectiveness of search intent recognition in the above embodiments.

Regarding the embodiments of the present invention, it should also be noted that, under the condition of no conflict, the embodiments of the present invention and the features in the embodiments can be combined with each other to obtain new embodiments.

Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention without limitation. Although the present invention has been described in detail with reference to the preferred embodiments, those of ordinary skill in the art should understand that the technical solutions of the present invention can be Modifications or equivalent replacements can be made without departing from the spirit and scope of the technical solutions of the present invention.

Claims (8)

1. An evaluation method for judging the effectiveness of search intent recognition, comprising the following steps: S1, obtaining the total number of word segmentations and the information of each word segmentation in the search intent recognition process to be evaluated; S2. According to the information, respectively calculate the sum of the word frequencies corresponding to each word segment and the inverted document frequency; S3, using the sum of word frequencies corresponding to each word segment obtained in step S2 and the inverted document frequency to calculate the effectiveness score of the search intent recognition process to be evaluated, and then determine whether the search intent recognition process is effective. 2. The method according to claim 1, wherein the information includes the domain of intent matched by each participle, wherein each domain of intent has a preset weight; each participle in the The number of matches in the intent domain; the total number of searches by the user within a preset time period, and the number of searches that include each participle in the total number of searches. 3. The method according to claim 2, calculate the corresponding word frequency sum of each participle according to the following formula: Among them, H is a set of matched intent domains, which is composed of multiple different intent domains, and f is one of the intent domains; is the number of times the participle t _i can be matched in the intent domain f; n _f is the number of words in the intent domain f; w _f is the weight of the intent domain f. 4. The method according to claim 3, according to the following formula, calculate the corresponding inverted document frequency of each participle Wherein, N is the total number of searches by the user within a preset time period; N(t _i ) is the number of searches including each word t _i , and log is the natural logarithm. 5. The method according to claim 4, calculate the validity score R of the search intent recognition process to be evaluated according to the following formula: Among them, n is the total number of word segmentation. 6. The method according to any one of claims 1-5, characterized in that step S3 further comprises: Comparing the validity score with a preset threshold, if the validity score is greater than the preset threshold, it is determined that the search intent recognition process is effective; if the validity score is smaller than the preset threshold, it is determined that the search intent recognition process is Invalid. 7. A computer-readable storage medium, on which executable instructions are stored, wherein when the instructions are executed by a processor, the steps of the method according to any one of claims 1-6 are implemented. 8. An electronic device comprising: memory for storing executable instructions; and A processor, configured to execute the executable instructions stored in the memory, so as to implement the steps of the method according to any one of claims 1-6.