CN119322838B - Context-based answer generation methods, devices, computer equipment, and storage media - Google Patents
Context-based answer generation methods, devices, computer equipment, and storage mediaInfo
- Publication number
- CN119322838B CN119322838B CN202411463414.8A CN202411463414A CN119322838B CN 119322838 B CN119322838 B CN 119322838B CN 202411463414 A CN202411463414 A CN 202411463414A CN 119322838 B CN119322838 B CN 119322838B
- Authority
- CN
- China
- Prior art keywords
- result
- preset
- question
- evaluation
- text information
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/30—Information retrieval; Database structures therefor; File system structures therefor of unstructured textual data
- G06F16/33—Querying
- G06F16/3331—Query processing
- G06F16/334—Query execution
-
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/30—Information retrieval; Database structures therefor; File system structures therefor of unstructured textual data
- G06F16/33—Querying
- G06F16/332—Query formulation
- G06F16/3329—Natural language query formulation
-
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/045—Combinations of networks
- G06N3/0455—Auto-encoder networks; Encoder-decoder networks
-
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/0475—Generative networks
-
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N5/00—Computing arrangements using knowledge-based models
- G06N5/04—Inference or reasoning models
- G06N5/041—Abduction
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- Mathematical Physics (AREA)
- Computational Linguistics (AREA)
- Data Mining & Analysis (AREA)
- General Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- Artificial Intelligence (AREA)
- Computing Systems (AREA)
- Software Systems (AREA)
- Evolutionary Computation (AREA)
- Biomedical Technology (AREA)
- Molecular Biology (AREA)
- General Health & Medical Sciences (AREA)
- Biophysics (AREA)
- Life Sciences & Earth Sciences (AREA)
- Health & Medical Sciences (AREA)
- Databases & Information Systems (AREA)
- Human Computer Interaction (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The invention relates to the technical field of computers and discloses a method, a device, computer equipment and a storage medium for generating a context answer, wherein the method comprises the steps of obtaining a search result according to a to-be-replied question and a search model, wherein the search model is used for obtaining first text information corresponding to the to-be-replied question in a corpus and generating a search result containing the first text information; the method comprises the steps of obtaining a generating result according to a to-be-replied question and a generating model, generating a first evaluation score of a searching result and a second evaluation score of the generating result according to a preset evaluation index, taking the searching result as a target result if the first evaluation score is larger than the second evaluation score, otherwise, taking the generating result as the target result, inputting the target result into a preset generator, and generating a context answer corresponding to the to-be-replied question. The method solves the problems that the retrieval limitation is large, misleading information is easily included in the retrieval result, and the accuracy of the generated context answer is affected.
Description
Technical Field
The present invention relates to the field of computer technologies, and in particular, to a method and apparatus for generating a context answer, a computer device, and a storage medium.
Background
The retrieval enhancement generation (RETRIEVAL-Augmented Generation, RAG) model combines a language generation model and information retrieval techniques. When the RAG model needs to generate text or answer questions, related information is firstly retrieved from a huge document set, and then the retrieved information is used for guiding the generation of context answers, so that the quality and accuracy of prediction are improved. However, if the inaccurate search result of the RAG model in the search stage will cause inaccurate context answers generated in the subsequent generation stage, the current search stage generally adopts a single search mode to perform semantic search in the document set according to the search requirement, so as to obtain the search result, which cannot effectively improve the accuracy of the search result and easily includes misleading information.
Therefore, the related art has the problems that the retrieval limitation is large, misleading information is easily included in the retrieval result, and the accuracy of the generated context answer is affected.
Disclosure of Invention
In view of the above, the present invention provides a method, apparatus, computer device and storage medium for generating a context answer, so as to solve the problem that the search limitation is large, misleading information is easily included in the search result, and the accuracy of generating the context answer is affected.
In a first aspect, the present invention provides a method for generating a context answer, including:
Obtaining a search result according to the to-be-replied problem and a search formula model, wherein the search formula model is used for obtaining first text information corresponding to the to-be-replied problem in a corpus and generating the search result containing the first text information;
Obtaining a generation result according to the to-be-replied problem and a generation type model, wherein the generation type model is used for generating second text information corresponding to the to-be-replied problem and generating a generation result containing the second text information;
generating a first evaluation score of the search result and a second evaluation score of the generated result according to a preset evaluation index, taking the search result as a target result if the first evaluation score is larger than the second evaluation score, otherwise taking the generated result as the target result, wherein the preset evaluation index is used for determining the probability that the search result and the generated result are the target result;
Inputting the target result into a preset generator to generate a context answer corresponding to the to-be-replied question, wherein the preset generator is used for constructing and generating the context answer according to the target result.
According to the context answer generation method, the search type model and the generation type model are designed in the search stage, the search result corresponding to the to-be-replied question is obtained according to the search type model, the generation result corresponding to the to-be-replied question is generated according to the generation type model, the search result and the generation result are evaluated according to the preset evaluation index, the optimal target result is conveniently obtained, and the accuracy of the search stage result is greatly improved. And generating a context answer according to the target result by using a preset generator, wherein the generated context information has higher retrieval accuracy, and the generated result is more stable and accurate. The method solves the problems that the retrieval limitation is large, misleading information is easily included in the retrieval result, and the accuracy of the generated context answer is affected.
In some optional embodiments, obtaining the search result according to the to-be-replied question and the retrievable model includes:
Inputting the problem to be replied into a search model to obtain a search result;
The retrieval model is used for carrying out feature extraction operation on a question to be replied to obtain a question feature, matching the question feature with question answering texts in a corpus to obtain intermediate text information containing a first preset number of question answering texts, determining a second preset number of target question answering texts from the first preset number of question answering texts of the intermediate text information according to a preset index, taking the target question answering texts as the first text information, and generating a retrieval result containing the first text information, wherein the second preset number is smaller than or equal to the first preset number.
In this embodiment, the to-be-replied question is input into the search model, and the search result is obtained by determining the target question-answering text corresponding to the to-be-replied question from the corpus by using the search model, and the preset generator generates the context answer according to the search result, so that the quality and accuracy of the context answer can be improved.
In some alternative embodiments, generating a first evaluation score of the search result and a second evaluation score of the generated result according to a preset evaluation index includes:
Determining related text information related to a to-be-replied problem in first text information of a search result, and determining a first quantity of the related text information;
acquiring related question-answer texts related to the questions to be replied in the corpus, and determining a second number of the related question-answer texts;
Dividing the first quantity by the second quantity to obtain the accuracy of the search result;
Dividing the first quantity by the second preset quantity to obtain the recall rate of the search result;
Obtaining a reconciliation average value corresponding to the retrieval result according to the accuracy rate, the recall rate and a preset formula, and obtaining a first evaluation score according to the reconciliation average value;
the preset formula satisfies:
wherein F is a harmonic mean value, alpha is a preset parameter, P is an accuracy rate, and R is a recall rate;
Determining the confusion degree and entropy of the second text information of the generated result, and determining the average reply length of the second text information;
Determining the total number of words, the third number of the single words and the fourth number of the double words in the second text information of the generated result;
Dividing the third quantity by the total number of words to obtain a first proportion of the single words, dividing the fourth quantity by the total number of words to obtain a second proportion of the double words, and obtaining a reply diversity index of a generated result according to the first proportion and the second proportion;
And obtaining a second evaluation score according to the confusion degree, the entropy, the replying diversity index, the average replying length and the preset weight.
In this embodiment, the search result and the generated result are evaluated according to a preset evaluation index by using an evaluation model, so as to obtain a first evaluation score of the search result and a second evaluation score of the generated result. The optimal target result is conveniently obtained through the evaluation score, and the accuracy of the result in the retrieval stage is greatly improved.
In some alternative embodiments, generating a first evaluation score of the search result and a second evaluation score of the generated result according to a preset evaluation index includes:
Inputting the search result into an evaluation model to obtain a first evaluation score, wherein the evaluation model is used for determining the probability that the search result is a target result according to a preset evaluation index, the evaluation model is obtained after training an initial evaluation model according to a first training sample, and the initial evaluation model is constructed according to the preset evaluation index;
and inputting the generated result into an evaluation model to obtain a second evaluation score, wherein the evaluation model is used for determining the probability that the generated result is a target result according to a preset evaluation index.
In this embodiment, an evaluation model is constructed by using a preset evaluation index, and the retrieval result and the generation result are evaluated by using the evaluation model according to the preset evaluation index, and the corresponding evaluation score is output. The optimal target result is conveniently obtained through the evaluation score, and the accuracy of the result in the retrieval stage is greatly improved.
In some alternative embodiments, before obtaining the generated result according to the question to be replied and the generated model, the method further includes:
constructing an initial model according to a preset algorithm;
And acquiring a second training sample, and training the initial model according to the second training sample to obtain a generated model.
In the embodiment, an initial model is built according to a preset algorithm, and the initial model is trained according to a second training sample to obtain a generated model. The generation result matched with the problem to be replied is automatically generated through the generation model, and the problem that the retrieval result possibly has information missing or hard expression can be made up through the generation result.
In some alternative embodiments, according to the question to be replied and the generative model, a generating result is obtained, including:
Inputting the problem to be replied into a generating model to obtain a generating result;
The generation type model comprises an encoder and a decoder, wherein the encoder for generating the type model is used for determining semantic features of a problem to be replied, and the decoder for generating the type model is used for generating second text information according to the semantic features and generating a generation result containing the second text information.
In some alternative embodiments, the method further comprises:
acquiring a new added evaluation index, wherein the new added evaluation index comprises the accuracy of a first preset number of first text information in the search result and/or the average accuracy of the search result, and the third preset number is smaller than or equal to the second preset number;
Updating a preset evaluation index according to the newly added evaluation index;
The average accuracy rate satisfies:
wherein MAP is average accuracy, AP is average accuracy of the first text information, Q R is total number of the first text information, and Q is sequence number.
In this embodiment, a new evaluation index is added to the preset evaluation index, so as to further refine the evaluation index of the search result and the generated result, further improve the reliability of evaluation, further determine a more accurate target result, and finally improve the accuracy of the constructed context answer.
In a second aspect, the present invention provides a context answer generation device, including:
The retrieval result obtaining module is used for obtaining a retrieval result according to the to-be-replied problem and the retrieval type model, wherein the retrieval type model is used for obtaining first text information corresponding to the to-be-replied problem in the corpus and generating a retrieval result containing the first text information;
the generation result obtaining module is used for obtaining a generation result according to the to-be-replied problem and the generation type model, wherein the generation type model is used for generating second text information corresponding to the to-be-replied problem and generating a generation result containing the second text information;
The evaluation module is used for generating a first evaluation score of the search result and a second evaluation score of the generated result according to a preset evaluation index, taking the search result as a target result if the first evaluation score is larger than the second evaluation score, otherwise taking the generated result as the target result, wherein the preset evaluation index is used for determining the probability that the search result and the generated result are the target result;
The generating module is used for inputting the target result into a preset generator and generating a context answer corresponding to the to-be-replied question, wherein the preset generator is used for constructing and generating the context answer according to the target result.
In a third aspect, the present invention provides a computer device, including a memory and a processor, where the memory and the processor are communicatively connected to each other, and the memory stores computer instructions, and the processor executes the computer instructions, thereby executing the context answer generating method of the first aspect or any implementation manner corresponding to the first aspect.
In a fourth aspect, the present invention provides a computer-readable storage medium having stored thereon computer instructions for causing a computer to execute the context answer generation method of the first aspect or any of its corresponding embodiments.
In a fifth aspect, the present invention provides a computer program product comprising computer instructions for causing a computer to perform the context answer generation method of the first aspect or any of its corresponding embodiments.
Drawings
In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the related art, the drawings that are required to be used in the description of the embodiments or the related art will be briefly described, and it is apparent that the drawings in the description below are some embodiments of the present invention, and other drawings may be obtained according to the drawings without inventive effort for those skilled in the art.
FIG. 1 is a flow chart of a context answer generation method according to an embodiment of the invention;
FIG. 2 is a flow chart of an efficient search and generation co-optimization method according to an embodiment of the present invention;
fig. 3 is a block diagram showing the construction of a context answer generation device according to an embodiment of the present invention;
fig. 4 is a schematic diagram of a hardware structure of a computer device according to an embodiment of the present invention.
Detailed Description
For the purpose of making the objects, technical solutions and advantages of the embodiments of the present invention more apparent, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present invention, and it is apparent that the described embodiments are some embodiments of the present invention, but not all embodiments of the present invention. All other embodiments, which can be made by those skilled in the art based on the embodiments of the invention without making any inventive effort, are intended to be within the scope of the invention.
The existing RAG model is insufficient in collaborative optimization in a retrieval stage and a generation stage, so that the overall performance of the RAG model is limited. In particular, in the retrieval stage, the inaccuracy of the retrieval result will cause the inaccuracy of the context information generated in the subsequent generation stage, but the existing retrieval is generally in a single retrieval mode, so that the retrieval limitation is large, misleading information is easy to generate, for example, the retrieval requirement of a user is obtained, a plurality of retrieval results are obtained according to the retrieval requirement, wherein the retrieval results comprise a plurality of text information and a plurality of history dialogues, the target semantic distance between each retrieval result and the retrieval requirement is obtained according to each retrieval result and the retrieval requirement, each target semantic distance is sequenced, an array of the retrieval requirement is obtained, and each element in the array is sequentially subjected to byte processing, so that the context information of the large language model is obtained. Although the above technical solution can reduce crashes of the large language model, it cannot effectively improve accuracy of the search result.
Based on the above, the embodiment of the invention provides a context answer generation method, in which a search model and a generation model are designed in a search stage, and a search model evaluation index and a generation model evaluation index are respectively designed for evaluating results obtained by the search model evaluation index and the generation model evaluation index, so that an optimal result is conveniently obtained, the accuracy of the search result is greatly improved, and then the construction and the generation of the context answer are realized through a generator according to the optimal result. The new evaluation index can be added in the search type model evaluation index in an expanding way, further accuracy of the search type evaluation index is achieved, reliability of an evaluation result is further improved, further accurate search results are determined, and finally accuracy of the constructed context answer is improved. The method solves the problems that the existing RAG model is in a single retrieval mode in the retrieval stage, so that the retrieval limitation is large and misleading information is easy to generate. The method achieves the technical effects that the generated context information has higher retrieval accuracy, and the generated result is more stable and accurate.
According to an embodiment of the present invention, a context answer generation embodiment is provided, it should be noted that the steps shown in the flowchart of the drawings may be performed in a computer device having data processing capability, such as a computer, a server, etc., and that although a logical order is shown in the flowchart, in some cases the steps shown or described may be performed in an order different from that shown or described herein.
In this embodiment, a method for generating a context answer is provided, fig. 1 is a flowchart of a method for generating a context answer according to an embodiment of the present invention, and as shown in fig. 1, the flowchart includes the following steps:
Step S101, obtaining a search result according to the to-be-replied question and a search formula model, wherein the search formula model is used for obtaining first text information corresponding to the to-be-replied question in a corpus and generating the search result containing the first text information.
Specifically, in this embodiment, a dialogue corpus composed of a large number of answer pairs is first constructed, and the corpus may adopt an open domain or a closed domain.
Inputting the to-be-replied question into a search model, extracting the characteristics of the input to-be-replied question by the search model, matching a small number of question-answer pairs in a corpus according to the extracted characteristics to serve as a candidate set, selecting the best reply, namely first text information, from the candidate set by using a proper index, and generating a search result containing the first text information. Questions to be replied to such as emotion analysis questions, text classification questions, language acceptability questions (whether sentences are reasonable, whether grammar is smooth) and the like.
Step S102, a generation result is obtained according to the to-be-replied question and a generation formula model, wherein the generation formula model is used for generating second text information corresponding to the to-be-replied question and generating a generation result containing the second text information.
Specifically, a bert algorithm is adopted to construct a model and the model is subjected to large-scale corpus training to obtain a generated model. In the large-scale expected training process, the generated model learns the mode and the characteristics of human dialogue, so that the generated model can automatically generate replies matched with the generated models, and the problems of information loss or hard expression possibly existing in the searched replies are solved.
And inputting the to-be-replied problem into a generating model, generating second text information corresponding to the to-be-replied problem by the generating model, and generating a generating result containing the second text information.
Step S103, generating a first evaluation score of the search result and a second evaluation score of the generated result according to a preset evaluation index, wherein the search result is used as a target result if the first evaluation score is larger than the second evaluation score, otherwise, the generated result is used as the target result, and the preset evaluation index is used for determining the probability that the search result and the generated result are the target result.
Specifically, the preset evaluation indexes include indexes for evaluating the search result, such as accuracy (P), recall (R), a harmonic mean F of Recall and accuracy, and indexes for evaluating the generated result. The indexes for evaluating the generated result are, for example, confusion, entropy, reply diversity index, average reply length, and the like. The search model evaluation index focuses on the evaluation of recall rate and accuracy, the generation model evaluation index focuses on the generation of reply length, and the model generating long sentences is considered to have higher relative quality, based on which the search result of the search model or the generation result of the generation model can be selected according to the required reply content characteristics in the specific evaluation.
And according to a preset evaluation index, evaluating the search result of the search type model and the generation result of the generation type model, determining the similarity or quality of the search result and the generation result with the best reply, further determining the probability that the search result and the generation result are target results, and generating a first evaluation score of the search result and a second evaluation score of the generation result to reflect the evaluation result. And determining a target result according to the first evaluation score and the second evaluation score, and if the first evaluation score is larger than the second evaluation score, indicating that the retrieval result of the retrievable model has better effect, taking the retrieval result as the target result. Otherwise, the generation result of the generation formula model is better, and the generation result is taken as a target result.
Step S104, inputting the target result into a preset generator to generate a context answer corresponding to the to-be-replied question, wherein the preset generator is used for constructing and generating the context answer according to the target result.
Specifically, the preset generator is a system or model that is capable of automatically generating an output based on a particular input (e.g., a target result). Inputting the target result into a preset generator, arranging all context information related to the to-be-replied question by the preset generator, and constructing and generating a context answer corresponding to the to-be-replied question by the generator.
According to the context answer generation method, the search type model and the generation type model are designed in the search stage, the search result corresponding to the to-be-replied question is obtained according to the search type model, the generation result corresponding to the to-be-replied question is generated according to the generation type model, the search result and the generation result are evaluated according to the preset evaluation index, the optimal target result is conveniently obtained, and the accuracy of the search stage result is greatly improved. And generating a context answer according to the target result by using a preset generator, wherein the generated context information has higher retrieval accuracy, and the generated result is more stable and accurate. The method solves the problems that the retrieval limitation is large, misleading information is easily included in the retrieval result, and the accuracy of the generated context answer is affected.
In some optional embodiments, obtaining the search result according to the to-be-replied question and the retrievable model includes:
Inputting the problem to be replied into a search model to obtain a search result;
The retrieval model is used for carrying out feature extraction operation on a question to be replied to obtain a question feature, matching the question feature with question answering texts in a corpus to obtain intermediate text information containing a first preset number of question answering texts, determining a second preset number of target question answering texts from the first preset number of question answering texts of the intermediate text information according to a preset index, taking the target question answering texts as the first text information, and generating a retrieval result containing the first text information, wherein the second preset number is smaller than or equal to the first preset number.
Specifically, the to-be-recovered problem is input into a search model, and the search model performs feature extraction on the input to-be-recovered problem to extract the problem features. The prediction library is composed of a large number of question-answer texts such as dialogue information and the like. The search model matches the question features with question and answer texts in the corpus, determines text information which is successfully matched, namely intermediate text information, wherein the number of the intermediate text information is a first preset number, and the first preset number represents a small number, such as 5, 8, 10 and the like, and specific numerical values are set according to actual demands. The search model may also employ deep learning technique NLP (Natural Language Processing ) to perform question-answering dialogue training in the corpus.
The preset indexes include semantic similarity, correlation, fluency, information richness and the like. And determining a second preset number of target question-answer texts with optimal preset indexes from the first preset number of question-answer texts of the intermediate text information according to the preset indexes, and taking the target question-answer texts as the first text information. The second preset number is a number less than or equal to the first preset number.
In this embodiment, the to-be-replied question is input into the search model, and the search result is obtained by determining the target question-answering text corresponding to the to-be-replied question from the corpus by using the search model, and the preset generator generates the context answer according to the search result, so that the quality and accuracy of the context answer can be improved.
In some alternative embodiments, generating a first evaluation score of the search result and a second evaluation score of the generated result according to a preset evaluation index includes:
Determining related text information related to a to-be-replied problem in first text information of a search result, and determining a first quantity of the related text information;
acquiring related question-answer texts related to the questions to be replied in the corpus, and determining a second number of the related question-answer texts;
Dividing the first quantity by the second quantity to obtain the accuracy of the search result;
Dividing the first quantity by the second preset quantity to obtain the recall rate of the search result;
Obtaining a reconciliation average value corresponding to the retrieval result according to the accuracy rate, the recall rate and a preset formula, and obtaining a first evaluation score according to the reconciliation average value;
the preset formula satisfies:
wherein F is a harmonic mean value, alpha is a preset parameter, P is an accuracy rate, and R is a recall rate;
Determining the confusion degree and entropy of the second text information of the generated result, and determining the average reply length of the second text information;
Determining the total number of words, the third number of the single words and the fourth number of the double words in the second text information of the generated result;
Dividing the third quantity by the total number of words to obtain a first proportion of the single words, dividing the fourth quantity by the total number of words to obtain a second proportion of the double words, and obtaining a reply diversity index of a generated result according to the first proportion and the second proportion;
And obtaining a second evaluation score according to the confusion degree, the entropy, the replying diversity index, the average replying length and the preset weight.
Specifically, related text information related to a problem to be replied in the first text information of the search result is determined, and a first quantity of the related text information is determined. When the related text information is determined, a threshold value can be set, the first text information with the score higher than the threshold value is judged to be related text information related to the problem to be replied, and the related return number searched by the system can be obtained by counting the first quantity of the related text information with the score higher than the threshold value. Similarly, the question-answer texts with scores higher than the threshold value in the corpus are judged to be related question-answer texts related to the questions to be replied, and the second quantity of the related question-answer texts is determined, so that the total number of all related replies of the system can be obtained.
Accuracy (P) is used to see how much of the replies given by the system are truly relevant, p=the total number of relevant replies retrieved by the system/all relevant replies of the system, i.e. p=the first number/the second number, and therefore, the accuracy P of the retrieved result is obtained by dividing the first number by the second number.
According to the above embodiment, the second preset number of the first text messages is included in the search result, so that the total number of all the replies searched by the system is equal to the second preset number. Recall (R) is used to check the ability of the system to find all relevant replies, r=the number of relevant replies retrieved by the system/the total number of replies retrieved by the system, i.e. r=the first number/the second preset number, so that the Recall R of the retrieved result is obtained by dividing the first number by the second preset number.
And inputting the accuracy rate P and the recall rate R into the preset formula, and calculating to obtain a harmonic mean value F corresponding to the retrieval result. The F value is a harmonic average value of the recall rate and the accuracy rate and is used for comprehensively evaluating the performance of the retrieval system, and the higher the F value is, the better the retrieval system is in terms of both the accuracy rate and the recall rate. In addition, the recall rate is the capability of the investigation system to find the full reply, and the accuracy rate investigation system is the capability of the investigation system to find the reply, and the recall rate and the accuracy rate complement each other to more comprehensively reflect the system performance from two different sides. When a=1, it is stated that the system achieves a better balance between accuracy and recall, i.e., the F1 value is the harmonic mean of recall and accuracy, as shown in equation (1).
The first evaluation score is obtained according to the harmonic mean, for example, the harmonic mean is directly used as the first evaluation score, the harmonic mean is adjusted according to actual demands and then used as the first evaluation score, and the ratio of the harmonic mean to the harmonic mean of the artificially generated standard result is used as the first evaluation score.
The probability of occurrence of a sentence is often measured by confusion in a language model, and is also often used in dialogue generation evaluation to evaluate the language quality of a generated reply sentence. The basic idea is that the higher the quality of the generated reply language, the smaller the sensitivity, the closer to the normal speaking of human, and the better the model. A disadvantage of the confusion index is that the relevance of replies to the above in a conversation cannot be assessed. Entropy can be used to measure the amount of information of the generated reply. The average length is used to measure the dialog generation effect and the model that generates long sentences is considered to be of higher relative quality.
And determining the confusion degree and entropy of the second text information of the generated result, and determining the average reply length of the second text information. The step of calculating the degree of confusion may comprise calculating a conditional probability of, for each word (x_t) in the second text information, its conditional probability (p (x_t|x_ { < t }) at a given prefix (x_ { < t }). And taking the logarithm of the conditional probability of each word to obtain the logarithmic probability. The average of the log probabilities of all words is calculated as the average log probability. The final degree of confusion is exp (average log probability). The step of calculating entropy may include calculating, for each word of the second text information, a conditional probability (p (x_n|x_ { < n }) thereof at a given prefix (x_ { < n }). For each word (x_n), the entropy of its conditional probability distribution is calculated.
A unigram (Unigrams) refers to a collection of individual words or symbols. In a language model, univariate words are typically used to count how frequently individual words are in text. The bigram (Bigrams) refers to a combination of two words or symbols in succession. In a language model Bigrams is typically used to capture word-to-word sequential relationships and to calculate the probabilities of word pairs. And determining the total number of words, the third number of the single words and the fourth number of the double words in the second text information of the generated result.
The first proportion (Distinct-1) of the single words is obtained by dividing the third quantity by the total number of words, the second proportion (Distinct-2) of the binary words is obtained by dividing the fourth quantity by the total number of words, the reply diversity index of the generated result is obtained according to the first proportion and the second proportion, the reply diversity index Distinct-1&2 is used for solving the problem of universal replies in a dialogue system, and the reply diversity is measured by calculating the proportions of the single words unigram (the richness of words in the generated replies) and the binary words bigrams (the influence of word sequences and phrases on the reply diversity).
And obtaining a second evaluation score according to the confusion degree, the entropy, the replying diversity index, the average replying length and the preset weight, wherein the preset weights of the confusion degree, the entropy, the replying diversity index and the average replying length are respectively 0.2, 0.3 and 0.3, and the preset weights can be set according to actual requirements, but the sum of the preset weights of all the items is ensured to be 1. In the process of determining the second evaluation score, the confusion degree, entropy, reply diversity index and average reply length of the standard result generated by the user can be obtained first, the calculated confusion degree, entropy, reply diversity index and average reply length are divided by the confusion degree, entropy, reply diversity index and average reply length of the standard result respectively, the four results are multiplied by corresponding preset weights, and the final result is used as the second evaluation score.
In this embodiment, the search result and the generated result are evaluated according to a preset evaluation index by using an evaluation model, so as to obtain a first evaluation score of the search result and a second evaluation score of the generated result. The optimal target result is conveniently obtained through the evaluation score, and the accuracy of the result in the retrieval stage is greatly improved.
In some alternative embodiments, generating a first evaluation score of the search result and a second evaluation score of the generated result according to a preset evaluation index includes:
Inputting the search result into an evaluation model to obtain a first evaluation score, wherein the evaluation model is used for determining the probability that the search result is a target result according to a preset evaluation index, the evaluation model is obtained after training an initial evaluation model according to a first training sample, and the initial evaluation model is constructed according to the preset evaluation index;
and inputting the generated result into an evaluation model to obtain a second evaluation score, wherein the evaluation model is used for determining the probability that the generated result is a target result according to a preset evaluation index.
Specifically, the embodiment constructs an initial evaluation model according to the preset evaluation index by using a neural network, for example, a Recurrent Neural Network (RNN), which may be used for text quality evaluation. And training the initial evaluation model according to a first training sample to obtain an evaluation model, wherein the first training sample comprises a manually generated standard result and a corresponding evaluation score, and an error result and a corresponding evaluation score. The step of creating an assessment model based on the preset assessment indicators may include collecting a first training sample, preprocessing the first training sample, including word segmentation, stop word removal, conversion to lower case, and the like. Feature extraction converts text into numerical form, each word being represented by a vector. A common approach is to use word embedding (word embeddings), and also to use the RNN's own encoder to extract features. Model construction, using one or more RNN layers to process an input word vector sequence, RNNs can capture order information and context dependencies in text. The type of RNN may be selected from standard RNNs, LSTM (long short term memory) or GRU (gated loop unit), etc. 4. The model is trained, and the RNN model is trained by using the first marked training sample. The goal is to adjust the model parameters to minimize the difference between the predicted and actual quality. Suitable loss functions are selected, such as Mean Square Error (MSE), cross entropy loss, etc.
And inputting the search result and the generated result into an evaluation model, evaluating the search result and the generated result by the evaluation model according to a preset evaluation index, and outputting a first evaluation score of the search result and a second evaluation score of the generated result. The similarity or quality of the result to the best result is reflected by the evaluation score.
In this embodiment, an evaluation model is constructed by using a preset evaluation index, and the retrieval result and the generation result are evaluated by using the evaluation model according to the preset evaluation index, and the corresponding evaluation score is output. The optimal target result is conveniently obtained through the evaluation score, and the accuracy of the result in the retrieval stage is greatly improved.
In some alternative embodiments, before obtaining the generated result according to the question to be replied and the generated model, the method further includes:
constructing an initial model according to a preset algorithm;
And acquiring a second training sample, and training the initial model according to the second training sample to obtain a generated model.
Specifically, the preset algorithm is, for example, a BERT algorithm, which is a deep learning technology based on a transducer architecture. And constructing an initial model by adopting a BERT algorithm. A second training sample is obtained, for example, a large-scale corpus, such as a collection of large-scale unlabeled text data including encyclopedia, news, web pages, questions and answers, conversations, and the like. Carrying out large-scale corpus training on the initial model according to the second training sample, so that the model learns the mode and the characteristics of human dialogue, the model can automatically generate a generation result matched with the problem to be replied, and the problems of information deletion or hard expression possibly existing in the search type reply are solved;
Additionally, the step of creating a generative model may include data preparation, collecting a second training sample, and preprocessing it, including word segmentation, removal of stop words, addition of special markers (e.g., [ CLS ], [ SEP ]), etc. The BERT architecture is built using a deep learning framework (e.g., tensorFlow, pyTorch) as an initial model, including components of a multi-layer transducer encoder, self-attention mechanism, position coding, etc. The initial model is pre-trained by performing a Mask Language Model (MLM) and Next Sentence Prediction (NSP) task on the second training sample, and model parameters are adjusted by back propagation so that it learns the generic language representation. And (3) fine tuning the initial model, adding a task-specific output layer on the basis of the pre-trained BERT model aiming at a specific NLP task (text generation), and then fine tuning on a corresponding small-scale annotation data set to optimize the adaptability of the model to the specific task.
In the embodiment, an initial model is built according to a preset algorithm, and the initial model is trained according to a second training sample to obtain a generated model. The generation result matched with the problem to be replied is automatically generated through the generation model, and the problem that the retrieval result possibly has information missing or hard expression can be made up through the generation result.
In some alternative embodiments, according to the question to be replied and the generative model, a generating result is obtained, including:
Inputting the problem to be replied into a generating model to obtain a generating result;
The generation type model comprises an encoder and a decoder, wherein the encoder for generating the type model is used for determining semantic features of a problem to be replied, and the decoder for generating the type model is used for generating second text information according to the semantic features and generating a generation result containing the second text information.
Specifically, the generative model of the present embodiment includes an encoder and a decoder.
And inputting the problem to be replied into the generated model, and learning semantic features of the problem to be replied by an encoder of the generated model. And then generating second text information according to the semantic features by using a decoder of the generated model and generating a generation result containing the second text information.
In some alternative embodiments, the method further comprises:
acquiring a new added evaluation index, wherein the new added evaluation index comprises the accuracy of a first preset number of first text information in the search result and/or the average accuracy of the search result, and the third preset number is smaller than or equal to the second preset number;
Updating a preset evaluation index according to the newly added evaluation index;
The average accuracy rate satisfies:
wherein MAP is average accuracy, AP is average accuracy of the first text information, Q R is total number of the first text information, and Q is sequence number.
Specifically, with the expansion of the test set scale and the deep understanding of the test result, the embodiment can add a new evaluation index capable of more accurately reflecting the system performance to the preset evaluation index, and the process includes obtaining the new evaluation index and updating the preset evaluation index according to the new evaluation index.
The newly added evaluation index can comprise the accuracy of the first text information of the first preset quantity in the retrieval result, the average accuracy of the retrieval result, the accuracy of the first 10 replies in the single result and the like.
The third preset number is a value W that is less than or equal to the second preset number. The accuracy of the first third preset number of first text messages in the search result indicates how many first text messages are related to the to-be-replied problem in the first W first text messages, namely the accuracy of the search result in the first W positions.
According to the formula of the average accuracy, the calculation of the average accuracy requires the calculation of the AP, and the method for calculating the AP is to directly perform numerical integration on the P-R curve (accuracy-recall curve).
In this embodiment, a new evaluation index is added to the preset evaluation index, so as to further refine the evaluation index of the search result and the generated result, further improve the reliability of evaluation, further determine a more accurate target result, and finally improve the accuracy of the constructed context answer.
In some alternative embodiments, an efficient search and generation collaborative optimization method is provided, which can solve the same technical problems and produce the same technical effects as those of the steps S101 to S104, and the method includes, as shown in fig. 2, step S1, constructing a search model. And S2, constructing a generating model. And S3, constructing a model evaluation index. And S4, evaluating the search formula and the generated model algorithm. And S5, constructing and generating a context answer.
Specifically, the search type model and the generation type model are used in parallel, the answer design algorithm of the search type model and the generation type model is evaluated, if the search type answer is good in effect, the search type answer is taken as a target result, and otherwise, the generation type answer is returned as a target result. And according to the finally determined target result, constructing and generating the context answer through a generator. The method has higher retrieval accuracy, and the generated result is more stable and accurate.
In the embodiment, the search model and the generation model are designed in the search stage, and the search model evaluation index and the generation model evaluation index are respectively designed for evaluating search results of the search model evaluation index and the generation model evaluation index, so that an optimal result is conveniently obtained, and then the context answer is constructed and generated through the generator, so that the accuracy of the search result is greatly improved. The method solves the problems that the existing RAG model is in a single retrieval mode in the retrieval stage, so that the retrieval limitation is large and misleading information is easy to generate.
In some optional embodiments, the specific process of the retrievable model obtaining the retrieval result corresponding to the to-be-replied question may further include steps A1 to A3.
And A1, inputting the to-be-replied problem into a first retrieval module, and retrieving in a pre-stored database to obtain a first set number of to-be-selected samples.
Specifically, after the problem to be replied is acquired, a shared index is built in a pre-stored database by using a first pre-built retriever based on the problem to be replied, and a first set number of samples to be selected are retrieved from the pre-stored database based on the shared index. The first set number may be 100 or 150. In this stage, the first retriever needs to retrieve k samples to be selected from the large-scale external data set. Classical BM25, BERT or SimCSE retrievers were employed, large scale indexes were constructed using the Faiss open source framework and retrieved using MIPS. It should be noted that the first retriever is shared across all tasks. Higher efficiency can be achieved due to the smaller size of the first retriever used in this stage.
And step A2, inputting the samples to be selected and the questions to be replied to a second retrieval module, sequencing the samples to be selected in response to the task prompt, and obtaining a second set number of retrieval samples.
Specifically, after obtaining a first set number of samples to be selected, rearranging k samples searched in the previous stage by using a second searcher, and taking d samples out of the k samples to serve as final search samples. This stage uses task-specific hints and pre-built language models (LLMs) such as OPT13b for task-specific retrieval. And after the first set number of samples to be selected are obtained, acquiring a task prompt. For k samples to be selected and questions (queries) among questions to be replied, a prompt method is first used for transformation. The various tasks can be converted into language model tasks through prompt. Different forms of NKI tasks can be unified and task-specific characterizations obtained using the hinting approach. For example: what is the question to reply to is the emotion of this sentence positive or negative? the language model will first convert it to an input-output task that requires a completion statement, the output hint-labeling answer is negative. After the sample similarity of each prompt labeling candidate sample is calculated, the prompt labeling candidate samples are rearranged according to the sample similarity by adopting a preset sequence, and a second preset number of prompt labeling candidate samples before selection are used as retrieval samples. The preset sequence may be a sequence from large to small.
And step A3, inputting the retrieval sample into a pre-stored answer generation module to obtain a retrieval result.
Specifically, after obtaining a retrieval sample, inputting the retrieval sample into a pre-stored answer generation module to read the retrieval sample, obtaining a reading result and generating a retrieval result according to the reading result. In some embodiments, the answer generation module includes a reader/generator, and in particular FiD (Fusion in Decoder) may be used as a generator.
In the embodiment, a first retriever general for tasks is used for constructing a shared index and taking out samples to be selected in a first stage, the samples to be selected are rearranged by using a pre-training language model guided by prompts in a second stage, retrieval samples are selected, and finally the retrieval samples are used as input of an answer generation module and the answer generation module generates retrieval results. Such a two-stage approach can save considerable computational overhead while achieving better performance for various NKI tasks.
The embodiment also provides a context answer generating device, which is used for implementing the above embodiment and the preferred implementation, and is not described in detail. As used below, the term "module" may be a combination of software and/or hardware that implements a predetermined function. While the means described in the following embodiments are preferably implemented in software, implementation in hardware, or a combination of software and hardware, is also possible and contemplated.
The present embodiment provides a context answer generation device, as shown in fig. 3, including:
the retrieval result obtaining module 301 is configured to obtain a retrieval result according to a to-be-replied question and a retrievable model, where the retrievable model is configured to obtain first text information corresponding to the to-be-replied question in a corpus and generate a retrieval result including the first text information;
The generating result obtaining module 302 is configured to obtain a generating result according to the to-be-replied question and a generating formula model, where the generating formula model is configured to generate second text information corresponding to the to-be-replied question and generate a generating result including the second text information;
the evaluation module 303 is configured to generate a first evaluation score of the search result and a second evaluation score of the generated result according to a preset evaluation index, and if the first evaluation score is greater than the second evaluation score, take the search result as a target result, otherwise, take the generated result as the target result, where the preset evaluation index is used to determine the probability that the search result and the generated result are the target result;
The generating module 304 is configured to input the target result into a preset generator, and generate a context answer corresponding to the to-be-replied question, where the preset generator is configured to construct and generate the context answer according to the target result.
In some alternative embodiments, the retrieval result obtaining module 301 obtains a retrieval result according to the to-be-replied question and the retrievable model, including:
Inputting the problem to be replied into a search model to obtain a search result;
The retrieval model is used for carrying out feature extraction operation on a question to be replied to obtain a question feature, matching the question feature with question answering texts in a corpus to obtain intermediate text information containing a first preset number of question answering texts, determining a second preset number of target question answering texts from the first preset number of question answering texts of the intermediate text information according to a preset index, taking the target question answering texts as the first text information, and generating a retrieval result containing the first text information, wherein the second preset number is smaller than or equal to the first preset number.
In some alternative embodiments, the evaluation module 303 generates a first evaluation score of the search result and a second evaluation score of the generated result according to a preset evaluation index, including:
Determining related text information related to a to-be-replied problem in first text information of a search result, and determining a first quantity of the related text information;
acquiring related question-answer texts related to the questions to be replied in the corpus, and determining a second number of the related question-answer texts;
Dividing the first quantity by the second quantity to obtain the accuracy of the search result;
Dividing the first quantity by the second preset quantity to obtain the recall rate of the search result;
Obtaining a reconciliation average value corresponding to the retrieval result according to the accuracy rate, the recall rate and a preset formula, and obtaining a first evaluation score according to the reconciliation average value;
the preset formula satisfies:
wherein F is a harmonic mean value, alpha is a preset parameter, P is an accuracy rate, and R is a recall rate;
Determining the confusion degree and entropy of the second text information of the generated result, and determining the average reply length of the second text information;
Determining the total number of words, the third number of the single words and the fourth number of the double words in the second text information of the generated result;
Dividing the third quantity by the total number of words to obtain a first proportion of the single words, dividing the fourth quantity by the total number of words to obtain a second proportion of the double words, and obtaining a reply diversity index of a generated result according to the first proportion and the second proportion;
And obtaining a second evaluation score according to the confusion degree, the entropy, the replying diversity index, the average replying length and the preset weight.
In some alternative embodiments, the evaluation module 303 generates a first evaluation score of the search result and a second evaluation score of the generated result according to a preset evaluation index, including:
Inputting the search result into an evaluation model to obtain a first evaluation score, wherein the evaluation model is used for determining the probability that the search result is a target result according to a preset evaluation index, the evaluation model is obtained after training an initial evaluation model according to a first training sample, and the initial evaluation model is constructed according to the preset evaluation index;
and inputting the generated result into an evaluation model to obtain a second evaluation score, wherein the evaluation model is used for determining the probability that the generated result is a target result according to a preset evaluation index.
In some alternative embodiments, before obtaining the generated result according to the question to be replied and the generated model, the apparatus is configured to:
constructing an initial model according to a preset algorithm;
And acquiring a second training sample, and training the initial model according to the second training sample to obtain a generated model.
In some alternative embodiments, the generating result obtaining module 302 obtains a generating result according to the to-be-replied question and the generating model, including:
Inputting the problem to be replied into a generating model to obtain a generating result;
The generation type model comprises an encoder and a decoder, wherein the encoder for generating the type model is used for determining semantic features of a problem to be replied, and the decoder for generating the type model is used for generating second text information according to the semantic features and generating a generation result containing the second text information.
In some alternative embodiments, the apparatus is further for:
acquiring a new added evaluation index, wherein the new added evaluation index comprises the accuracy of a first preset number of first text information in the search result and/or the average accuracy of the search result, and the third preset number is smaller than or equal to the second preset number;
Updating a preset evaluation index according to the newly added evaluation index;
The average accuracy rate satisfies:
wherein MAP is average accuracy, AP is average accuracy of the first text information, Q R is total number of the first text information, and Q is sequence number.
Further functional descriptions of the above respective modules and units are the same as those of the above corresponding embodiments, and are not repeated here.
The context answer generation device in this embodiment is presented in the form of a functional unit, where the unit refers to an ASIC (Application SPECIFIC INTEGRATED Circuit) Circuit, a processor and a memory that execute one or more software or firmware programs, and/or other devices that can provide the above-described functions.
The embodiment of the invention also provides a computer device which is provided with the context answer generating device shown in the figure 3.
Referring to fig. 4, fig. 4 is a schematic structural diagram of a computer device according to an alternative embodiment of the present invention, and as shown in fig. 4, the computer device includes one or more processors 10, a memory 20, and interfaces for connecting components, including a high-speed interface and a low-speed interface. The various components are communicatively coupled to each other using different buses and may be mounted on a common motherboard or in other manners as desired. The processor may process instructions executing within the computer device, including instructions stored in or on memory to display graphical information of the GUI on an external input/output device, such as a display device coupled to the interface. In some alternative embodiments, multiple processors and/or multiple buses may be used, if desired, along with multiple memories and multiple memories. Also, multiple computer devices may be connected, each providing a portion of the necessary operations (e.g., as a server array, a set of blade servers, or a multiprocessor system). One processor 10 is illustrated in fig. 4.
The processor 10 may be a central processor, a network processor, or a combination thereof. The processor 10 may further comprise, among other things, an integrated circuit, a programmable logic device, or a combination thereof. The programmable logic device may be a complex programmable logic device, a field programmable gate array, a general-purpose array logic, or any combination thereof.
Wherein the memory 20 stores instructions executable by the at least one processor 10 to cause the at least one processor 10 to perform a method for implementing the embodiments described above.
The memory 20 may include a storage program area that may store an operating system, application programs required for at least one function, and a storage data area that may store data created according to the use of the computer device, etc. In addition, the memory 20 may include high-speed random access memory, and may also include non-transitory memory, such as at least one magnetic disk storage device, flash memory device, or other non-transitory solid-state storage device. In some alternative embodiments, memory 20 may optionally include memory located remotely from processor 10, which may be connected to the computer device via a network. Examples of such networks include, but are not limited to, the internet, intranets, local area networks, mobile communication networks, and combinations thereof.
The memory 20 may comprise volatile memory, such as random access memory, or nonvolatile memory, such as flash memory, hard disk or solid state disk, or the memory 20 may comprise a combination of the above types of memory.
The computer device also includes a communication interface 30 for the computer device to communicate with other devices or communication networks.
The embodiments of the present invention also provide a computer readable storage medium, and the method according to the embodiments of the present invention described above may be implemented in hardware, firmware, or as a computer code which may be recorded on a storage medium, or as original stored in a remote storage medium or a non-transitory machine readable storage medium downloaded through a network and to be stored in a local storage medium, so that the method described herein may be stored on such software process on a storage medium using a general purpose computer, a special purpose processor, or programmable or special purpose hardware. The storage medium may be a magnetic disk, an optical disk, a read-only memory, a random-access memory, a flash memory, a hard disk, a solid state disk, or the like, and further, the storage medium may further include a combination of the above types of memories. It will be appreciated that a computer, processor, microprocessor controller or programmable hardware includes a storage element that can store or receive software or computer code that, when accessed and executed by the computer, processor or hardware, implements the methods illustrated by the above embodiments.
Portions of the present invention may be implemented as a computer program product, such as computer program instructions, which when executed by a computer, may invoke or provide methods and/or aspects in accordance with the present invention by way of operation of the computer. Those skilled in the art will appreciate that the existence of computer program instructions in a computer-readable medium includes, but is not limited to, source files, executable files, installation package files, and the like, and accordingly, the manner in which computer program instructions are executed by a computer includes, but is not limited to, the computer directly executing the instructions, or the computer compiling the instructions and then executing the corresponding compiled programs, or the computer reading and executing the instructions, or the computer reading and installing the instructions and then executing the corresponding installed programs. Herein, a computer-readable medium may be any available computer-readable storage medium or communication medium that can be accessed by a computer.
Although the embodiments of the present application have been described with reference to the accompanying drawings, various modifications and variations may be made by those skilled in the art without departing from the spirit and scope of the application, and such modifications and variations fall within the scope of the application as defined by the claims.
Claims (8)
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202411463414.8A CN119322838B (en) | 2024-10-18 | 2024-10-18 | Context-based answer generation methods, devices, computer equipment, and storage media |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202411463414.8A CN119322838B (en) | 2024-10-18 | 2024-10-18 | Context-based answer generation methods, devices, computer equipment, and storage media |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN119322838A CN119322838A (en) | 2025-01-17 |
| CN119322838B true CN119322838B (en) | 2025-11-18 |
Family
ID=94233287
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202411463414.8A Active CN119322838B (en) | 2024-10-18 | 2024-10-18 | Context-based answer generation methods, devices, computer equipment, and storage media |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN119322838B (en) |
Families Citing this family (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN116501835A (en) * | 2023-04-11 | 2023-07-28 | 清华大学 | Retrieval enhancement method and device for non-knowledge-intensive tasks |
| CN120234408B (en) * | 2025-05-29 | 2025-09-16 | 深圳华为云计算技术有限公司 | Data flywheel fine-tuning method and device |
Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110750616A (en) * | 2019-10-16 | 2020-02-04 | 网易（杭州）网络有限公司 | Retrieval type chatting method and device and computer equipment |
| CN118503389A (en) * | 2024-06-19 | 2024-08-16 | 中国工商银行股份有限公司 | Question and answer result processing method, device, equipment, medium and program product |
Family Cites Families (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110362651A (en) * | 2019-06-11 | 2019-10-22 | 华南师范大学 | Dialogue method, system, device and the storage medium that retrieval and generation combine |
| CN114416927B (en) * | 2022-01-24 | 2024-04-02 | 招商银行股份有限公司 | Intelligent question-answering method, device, equipment and storage medium |
| CN116561538A (en) * | 2023-04-04 | 2023-08-08 | 厦门美柚股份有限公司 | Question-answer scoring method, question-answer scoring device, electronic equipment and storage medium |
| CN118689976A (en) * | 2024-06-06 | 2024-09-24 | 河南省种业发展中心 | Dynamic multi-level retrieval enhanced knowledge base question answering method and device, medium, and equipment |
-
2024
- 2024-10-18 CN CN202411463414.8A patent/CN119322838B/en active Active
Patent Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110750616A (en) * | 2019-10-16 | 2020-02-04 | 网易（杭州）网络有限公司 | Retrieval type chatting method and device and computer equipment |
| CN118503389A (en) * | 2024-06-19 | 2024-08-16 | 中国工商银行股份有限公司 | Question and answer result processing method, device, equipment, medium and program product |
Also Published As
| Publication number | Publication date |
|---|---|
| CN119322838A (en) | 2025-01-17 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN116719520B (en) | Code generation method and device | |
| CN108363790B (en) | Method, device, equipment and storage medium for evaluating comments | |
| US9830315B1 (en) | Sequence-based structured prediction for semantic parsing | |
| CN119322838B (en) | Context-based answer generation methods, devices, computer equipment, and storage media | |
| JP2022003520A (en) | Machine learning system for digital assistant | |
| CN109840287A (en) | A kind of cross-module state information retrieval method neural network based and device | |
| US20210117625A1 (en) | Semantic parsing of natural language query | |
| CN112712804A (en) | Speech recognition method, system, medium, computer device, terminal and application | |
| CN117435716B (en) | Data processing method and system for power grid human-computer interaction terminal | |
| CN118093834A (en) | AIGC large model-based language processing question-answering system and method | |
| CN108595629B (en) | Data processing method and application for answer selection system | |
| AU2016383052A1 (en) | Systems and methods for suggesting emoji | |
| CN110019471A (en) | Generate text from structured data | |
| CN111191002A (en) | Neural code searching method and device based on hierarchical embedding | |
| CN115344811B (en) | Processing method and system of mathematical problem solver based on deep learning | |
| CN113157727A (en) | Method, apparatus and storage medium for providing recall result | |
| CN112883713B (en) | Evaluation object extraction method and device based on convolutional neural network | |
| Alshammari et al. | TAQS: an Arabic question similarity system using transfer learning of BERT with BiLSTM | |
| CN118152428A (en) | A method and device for predicting and enhancing query instructions of power customer service system | |
| CN119166785A (en) | Automated evaluation method and system for search enhancement generation system | |
| CN120470090A (en) | Text block retrieval method, computer program product and device | |
| CN118349818A (en) | A performance evaluation method for large language models of scientific literature | |
| CN118427215A (en) | Method for generating structured query statement, method for processing question and answer and corresponding device | |
| Williams | Zero Shot Intent Classification Using Long-Short Term Memory Networks. | |
| CN119830001A (en) | Training method and device for text generation model |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |