CN109299245B - Method and device for recalling knowledge points - Google Patents
Method and device for recalling knowledge points Download PDFInfo
- Publication number
- CN109299245B CN109299245B CN201811427653.2A CN201811427653A CN109299245B CN 109299245 B CN109299245 B CN 109299245B CN 201811427653 A CN201811427653 A CN 201811427653A CN 109299245 B CN109299245 B CN 109299245B
- Authority
- CN
- China
- Prior art keywords
- service
- matching
- question
- user
- knowledge
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000000034 method Methods 0.000 title claims abstract description 51
- 238000010586 diagram Methods 0.000 claims description 24
- 230000011218 segmentation Effects 0.000 claims description 17
- 238000013145 classification model Methods 0.000 claims description 12
- 238000012549 training Methods 0.000 claims description 11
- 230000006399 behavior Effects 0.000 claims description 10
- 238000004590 computer program Methods 0.000 claims description 3
- 238000012163 sequencing technique Methods 0.000 claims description 3
- 230000000694 effects Effects 0.000 abstract description 8
- 238000004422 calculation algorithm Methods 0.000 description 6
- 230000008569 process Effects 0.000 description 6
- 238000012423 maintenance Methods 0.000 description 3
- 238000001914 filtration Methods 0.000 description 2
- 230000006870 function Effects 0.000 description 2
- 230000036541 health Effects 0.000 description 2
- 230000006872 improvement Effects 0.000 description 2
- 230000007774 longterm Effects 0.000 description 2
- 230000008520 organization Effects 0.000 description 2
- 238000013528 artificial neural network Methods 0.000 description 1
- 230000008901 benefit Effects 0.000 description 1
- 230000008859 change Effects 0.000 description 1
- 238000006243 chemical reaction Methods 0.000 description 1
- 238000007635 classification algorithm Methods 0.000 description 1
- 238000010276 construction Methods 0.000 description 1
- 230000009849 deactivation Effects 0.000 description 1
- 201000010099 disease Diseases 0.000 description 1
- 208000037265 diseases, disorders, signs and symptoms Diseases 0.000 description 1
- 230000003993 interaction Effects 0.000 description 1
- 238000013507 mapping Methods 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
- 238000012360 testing method Methods 0.000 description 1
Images
Landscapes
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The embodiment of the specification provides a method and a device for recalling knowledge points, wherein the method comprises the following steps: firstly, determining a matching service corresponding to a user question, wherein the matching service is a service with a specific service type, then selecting a root node from a plurality of service guide maps as a service guide map of the matching service, determining the service guide map as a matching service guide map corresponding to the user question, then determining a matching knowledge point matched with the user question by using a pre-trained rule matching model in the matching service guide map, and finally determining a knowledge point recalled aiming at the user question based on a result of determining the matching knowledge point, so that when the knowledge point is recalled aiming at the user question, the question and answer effect of a robot is effectively improved.
Description
Technical Field
One or more embodiments of the present specification relate to the field of computers, and more particularly, to a method and apparatus for knowledge point recall.
Background
In the question-answering system of the intelligent customer service, in the interaction process of a client (namely a user) and a robot, the question of the user is spoken and simplified, and when a knowledge point is recalled aiming at the question of the user, the matching capability and effect of the customer service robot are improved, so that the whole question-answering system is very critical, and the service experience of the question-answering system is directly influenced. However, as business logic becomes more complex, the method of pushing knowledge points most relevant to the question description of the user by simply relying on the matching model gradually encounters a bottleneck, on one hand, maintenance work of each model becomes more complicated along with the increase of the models, and on the other hand, the intervention capability of operators on the models is weak, so that professional business knowledge capability of the operators cannot be exerted.
The question-answering system is based on a knowledge base, and answers related to user question sentences in the knowledge base are matched through a model. However, the knowledge base lacks necessary organization and abstraction, and when the quantity of the knowledge base is larger and larger, the cost for the operator to maintain the knowledge base is gradually increased until errors occur frequently in the maintenance process or even the maintenance is impossible, which seriously affects the improvement of the question and answer effect of the robot.
Therefore, an improved scheme is desired, and when knowledge points are recalled for user question sentences, the question-answering effect of the robot is effectively improved.
Disclosure of Invention
One or more embodiments of the present specification describe a method and an apparatus for recalling knowledge points, so that when a knowledge point is recalled for a question of a user, a question and answer effect of a robot is effectively improved.
In a first aspect, a method for recalling knowledge points is provided, where the method is performed based on a plurality of service guide maps established in advance, each service guide map includes a plurality of nodes organized into a tree-like hierarchical structure according to service dimensions, the plurality of nodes includes a root node, an intermediate node, and a leaf node, the root node represents a service with a specific service type, the intermediate node represents service knowledge keywords of different levels related to the service, and the leaf node mounts a knowledge point in a knowledge base associated with the leaf node, and the method includes:
determining a matching service corresponding to the question of the user, wherein the matching service is a service with the specific service type;
selecting a root node from the plurality of service guide graphs as a service guide graph of the matched service, and determining the service guide graph as a matched service guide graph corresponding to the question of the user;
in the matching service guide map, determining a matching knowledge point matched with the question of the user by using a pre-trained rule matching model;
and determining knowledge points recalled for the user question based on a result of determining the matched knowledge points.
In a possible implementation manner, the determining a matching service corresponding to a user question includes:
and when the word segmentation result of the user question sentence comprises the word segmentation used for indicating the service, determining the service indicated by the word segmentation as the matching service corresponding to the user question sentence.
In a possible implementation manner, the determining a matching service corresponding to a user question includes:
acquiring historical behavior data of the user corresponding to the question of the user;
and determining the service indicated by the historical behavior data as a matching service corresponding to the user question.
In a possible implementation manner, the determining a matching service corresponding to a user question includes:
acquiring scene embedded point information corresponding to the question of the user;
and determining the service corresponding to the scene indicated by the scene embedded point information as the matching service corresponding to the question of the user.
In a possible embodiment, the determining knowledge points recalled for the user question based on the result of determining the matching knowledge points includes:
and when the determined number of the matched knowledge points is less than or equal to a preset threshold value, taking the matched knowledge points as knowledge points recalled for the question of the user.
In a possible embodiment, the determining knowledge points recalled for the user question based on the result of determining the matching knowledge points includes:
when the number of the determined matching knowledge points is larger than a preset threshold value, sequencing the matching knowledge points according to the matching degree with the question of the user from high to low;
and selecting the matched knowledge points with the number of preset threshold values in the front sequence as the knowledge points recalled aiming at the question of the user.
In one possible embodiment, the method further comprises:
determining a preset level matching node matched with the matching service guide diagram by the user question by utilizing a pre-trained classification model;
the determining knowledge points recalled for the user question based on the determining the matching knowledge points comprises:
and when the matched knowledge points are not determined, taking the user question, the matched nodes and the matched services as input, searching the knowledge points in a knowledge base by using a search sorting system, and taking the searched knowledge points with the number of the front preset threshold values as the knowledge points recalled aiming at the user question according to the sorting.
Further, the training samples of the pre-trained classification model include:
taking any one item in a knowledge point title and a knowledge point corresponding question set corresponding to the knowledge point as an input sample;
and taking the node of the preset level to which the knowledge point belongs in the service guide map as an output sample.
In a second aspect, an apparatus for recalling knowledge points is provided, where the apparatus is performed based on a plurality of service guide maps established in advance, each service guide map includes a plurality of nodes organized into a tree-like hierarchical structure according to service dimensions, the plurality of nodes includes a root node, an intermediate node, and a leaf node, the root node represents a service with a specific service type, the intermediate node represents a service knowledge keyword at a different level related to the service, and the leaf node mounts a knowledge point in a knowledge base associated with the leaf node, and the apparatus includes:
a service matching unit, configured to determine a matching service corresponding to a question of a user, where the matching service is a service with the specific service type;
a guide map matching unit, configured to select a service guide map, which is determined by the service matching unit and matches a service by a root node, from the multiple service guide maps, and determine the service guide map as a matching service guide map corresponding to the question of the user;
a knowledge point matching unit, configured to determine, in the matching service guide map determined by the guide map matching unit, matching knowledge points that match the question of the user by using a pre-trained rule matching model;
and the knowledge point recalling unit is used for determining the knowledge points recalled aiming at the question of the user based on the result of determining the matched knowledge points by the knowledge point matching unit.
In a third aspect, there is provided a computer readable storage medium having stored thereon a computer program which, when executed in a computer, causes the computer to perform the method of the first aspect.
In a fourth aspect, there is provided a computing device comprising a memory having stored therein executable code and a processor that, when executing the executable code, implements the method of the first aspect.
The method and the device provided by the embodiment of the specification determine a matching service corresponding to a user question, wherein the matching service is a service with a specific service type, then select a root node from a plurality of service guide maps as a service guide map of the matching service, determine the service guide map as a matching service guide map corresponding to the user question, determine a matching knowledge point matched with the user question in the matching service guide map by using a pre-trained rule matching model, and finally determine a knowledge point recalled for the user question based on a result of determining the matching knowledge point. As can be seen from the above, in the embodiments of the present specification, the knowledge point recall is performed based on the pre-established knowledge guide diagram, so that the operator can influence the result of the knowledge point recall by adjusting the knowledge guide diagram, and thus the question and answer effect of the robot is effectively improved when the knowledge point recall is performed for the question of the user.
Drawings
In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings needed to be used in the description of the embodiments are briefly introduced below, and it is obvious that the drawings in the following description are only some embodiments of the present invention, and it is obvious for those skilled in the art to obtain other drawings based on these drawings without creative efforts.
FIG. 1 is a schematic diagram illustrating an implementation scenario of an embodiment disclosed herein;
FIG. 2 illustrates a flow diagram of a method of knowledge point recall according to one embodiment;
FIG. 3 shows a flow diagram of a method of knowledge point recall according to another embodiment;
FIG. 4 is a flow chart of training and predicting three classes provided by embodiments of the present disclosure;
FIG. 5 is a block diagram of a FastText algorithm according to an embodiment of the present disclosure;
fig. 6 is a flowchart of a method for service prediction according to an embodiment of the present disclosure;
FIG. 7 is a flowchart of a method for search fine-ranking based on map guide information according to an embodiment of the present disclosure;
FIG. 8 shows a schematic block diagram of an apparatus for knowledge point recall according to one embodiment.
Detailed Description
The scheme provided by the specification is described below with reference to the accompanying drawings.
Fig. 1 is a schematic view of an implementation scenario of an embodiment disclosed in this specification. The implementation scenario involves a recall of knowledge points based on a traffic guide map. Referring to fig. 1, the service guide diagram is a tree structure that is combed by operators and organizes knowledge points of a knowledge base in a hierarchical manner. In this embodiment of the present specification, a plurality of service guide maps may be pre-established, each service guide map 100 includes a plurality of nodes organized into a tree-like hierarchical structure according to service dimensions, where the plurality of nodes includes a root node 11, an intermediate node 12, and a leaf node 13, the root node represents a service with a specific service type, the intermediate node represents a service knowledge keyword at a different level related to the service, and the leaf node mounts a knowledge point 14 in a knowledge base associated with the leaf node. In the embodiment of the description, based on the service guide diagram, the matching result of the question and the guide diagram of the user is determined through the model, and the knowledge point recall is performed according to the matching result, so that an operator can conveniently and quickly modify and adjust the guide diagram of the knowledge base and can timely feed back the knowledge point recall to a matching system for knowledge point recall.
The business guide map is an organization form of tree-shaped knowledge combed by operators. Taking the service guide diagram shown in fig. 1 as an example, the root node of the tree represents a service (e.g., good medical insurance long-term medical care) with a specific service type (e.g., insurance service type), and the leaf nodes of the tree are knowledge points in the knowledge base through layer-by-layer branching, and the structure and the node names of the guide diagram can be modified and adjusted, so that great convenience is brought to an operator for editing and adjusting the knowledge base.
It can be understood that knowledge points of multiple services of the same service type generally have some identical keywords, and therefore, when constructing the service guide map for the multiple services, the service guide map corresponding to each service may have the same preset level nodes, for example, the nodes of the third level (abbreviated as "third level category"). The root node of the service guide graph represents a specific service, and the service guide graph with the root node as the service can be found by determining the service corresponding to the question of the user and used as the service guide graph matched with the question of the user.
Fig. 2 shows a flowchart of a method for recalling knowledge points according to an embodiment, the method is performed based on a plurality of pre-established service guide maps, each service guide map comprises a plurality of nodes organized into a tree-like hierarchical structure according to service dimensions, the plurality of nodes comprises a root node, an intermediate node and a leaf node, the root node represents services with a specific service type, the intermediate node represents service knowledge keywords of different levels related to the services, and the leaf node mounts a knowledge point in a knowledge base associated with the leaf node. As shown in fig. 2, the method for recalling knowledge points in this embodiment includes the following steps: step 21, determining a matching service corresponding to the question of the user, wherein the matching service is a service with the specific service type; step 22, selecting a root node from the plurality of service guide maps as a service guide map of the matching service, and determining the service guide map as a matching service guide map corresponding to the question of the user; step 23, determining a matching knowledge point matched with the question of the user in the matching service guide map by using a pre-trained rule matching model; and step 24, determining knowledge points recalled aiming at the question of the user based on the result of determining the matched knowledge points. Specific execution modes of the above steps are described below.
First, in step 21, a matching service corresponding to a question of a user is determined, where the matching service is a service having the specific service type.
In an example, when the word segmentation result of the user question includes a word segmentation for indicating a service, the service indicated by the word segmentation is determined as a matching service corresponding to the user question.
In another example, historical behavior data (e.g., historical purchase factors) of a user corresponding to the user question is obtained, and then a service indicated by the historical behavior data is determined as a matching service corresponding to the user question.
In another example, the scene embedding point information corresponding to the user question is obtained first, and then the service corresponding to the scene indicated by the scene embedding point information is determined as the matching service corresponding to the user question.
In the embodiments of the present specification, the method for determining the matching service may be used alternatively or in combination.
Next, in step 22, a root node is selected from the plurality of service guide maps as the service guide map of the matching service, and the service guide map is determined as the matching service guide map corresponding to the question of the user.
It can be understood that the question of the user corresponds to a specific service type, a plurality of service guide maps which are pre-established exist in the service type, and after the matching service corresponding to the question of the user is determined, the matching service guide map corresponding to the question of the user can be screened from the plurality of service guide maps for the matching service through the root node.
Then, in step 23, in the matching service guide map, a matching knowledge point matching the question of the user is determined by using a pre-trained rule matching model.
In one example, a user question may be input into a rule engine of the guide graph, a node of the service guide graph is matched, and a corresponding knowledge point is searched for according to the matched node. There may be a plurality of matched knowledge points or none.
Finally, in step 24, knowledge points recalled for the user question are determined based on the results of determining matching knowledge points.
In one example, when the number of the determined matching knowledge points is less than or equal to a preset threshold, the matching knowledge points are used as knowledge points recalled for the question of the user.
In one example, when the number of the determined matching knowledge points is greater than a preset threshold, sorting the matching knowledge points according to the matching degree with the question of the user from high to low; and selecting the matched knowledge points with the number of preset threshold values in the front sequence as the knowledge points recalled aiming at the question of the user.
In one example, a pre-trained classification model may be used to determine a preset hierarchy of matching nodes where the user question matches the matching traffic guide graph, where determining a matching node may be performed when no matching knowledge point is determined; and when the matched knowledge points are not determined, taking the user question, the matched nodes and the matched services as input, searching the knowledge points in a knowledge base by using a search sorting system, and taking the searched knowledge points with the number of the front preset threshold values as the knowledge points recalled aiming at the user question according to the sorting.
The classification model may be a FastText model, or may be other classification models such as an SVM and MaxEnt.
Further, the training samples of the pre-trained classification model may include: taking any one item in a knowledge point title and a knowledge point corresponding question set corresponding to the knowledge point as an input sample; and taking the node of the preset level to which the knowledge point belongs in the service guide map as an output sample.
The method provided by the embodiment of the specification comprises the steps of firstly determining a matching service corresponding to a user question, wherein the matching service is a service with a specific service type, then selecting a root node from a plurality of service guide maps as a service guide map of the matching service, determining the service guide map as a matching service guide map corresponding to the user question, then determining a matching knowledge point matched with the user question by utilizing a pre-trained rule matching model in the matching service guide map, and finally determining a knowledge point recalled aiming at the user question based on a result of determining the matching knowledge point. As can be seen from the above, in the embodiments of the present specification, the knowledge point recall is performed based on the pre-established knowledge guide diagram, so that the operator can influence the result of the knowledge point recall by adjusting the knowledge guide diagram, and thus the question and answer effect of the robot is effectively improved when the knowledge point recall is performed for the question of the user.
Fig. 3 shows a flow diagram of a method of knowledge point recall according to another embodiment. This example gives a more specific implementation, and with reference to fig. 3, the method includes:
first, in step 31, a user's question entering the customer service system is subjected to word segmentation and word deactivation, and a simple entity recognition process is performed to recognize entities such as place names, disease names, and the like.
Then, in step 32, a third-level category prediction is performed by using the FastText model, where the third-level category refers to node information in the service guide graph, for example, in the service guide graph system of insurance, the third-level category refers to "insurance application", "insurance withdrawal", "claim settlement", etc., and can reflect the core requirements of the user, and these nodes are followed by more detailed service division.
Fig. 4 is a flowchart of training and predicting three-level categories according to an embodiment of the present disclosure. The method comprises two stages of model training and online prediction, wherein the model can be a FastText model, and the FastText is a text classification algorithm.
Fig. 5 is a schematic structural framework diagram of a FastText algorithm provided in an embodiment of the present specification, and referring to fig. 5, where x1, x2, …, xN-1, xN are ngram features in text, and are mapped to outputs of classification categories through a hidden (hidden) layer neural network and an output (output) layer. And using the three-level categories corresponding to the titles of the knowledge points in the existing service guide diagram as training data, wherein each knowledge point also corresponds to a corresponding extended question set, and the titles and the questions correspond to the three-level categories of the knowledge points in the guide diagram. And collecting all titles, questions and corresponding knowledge point three-level categories as training data, and training by using a FastText model to obtain a model (model) for predicting the three-level categories. When the method is applied on line, the user question is subjected to word segmentation and word stop, and prediction is carried out by using a trained model, so that the three-level category of the service corresponding to the current user question can be obtained.
Next, in step 33, the service where the user question is located is predicted by using various information in combination.
Fig. 6 is a flowchart of a method for service prediction according to an embodiment of the present disclosure, where a service corresponding to a user question is determined by combining the user question, a historical purchase factor, and a scene embedding point. Referring to fig. 6, for the input user question, if the user explicitly mentions service words, such as "how to refund good medical insurance long-term medical care", "what the balance is good", and the like, the service described at the user side is directly output. If not explicitly described, but the three-level service classification of the question of the user corresponds to the after-sale problem, such as "how to settle the claims" and "how to find the policy", and the corresponding three-level service classification is "settle the claims" and "policy", which belong to the after-sale three-level service classification, the historical purchase factor of the user is extracted to inquire that the user purchases the things on the platform, such as that the user purchases the insurance "worry free from major illness", and then the "settle the claims" and "policy" mentioned currently by the user refer to the service "worry free from major illness". Finally, if the service cannot be judged by the methods, the scene embedded point information where the question of the user is located is extracted, the service is determined according to the scene embedded point information, for example, the user enters a question generated by my customer service from an insurance page of good medical insurance hospitalization, the corresponding service is the good medical insurance hospitalization, the user enters the generated question from a income page of balance treasured, and the corresponding service is the balance treasured.
Then, in step 34, the user question enters the rule engine for map matching, and the knowledge points potentially corresponding to the user question are searched in the tree-like map.
The number of results of the rules engine is divided into three categories: a. the number of results is 3; b. no result is obtained; c. the number of results was > 3. Different ways can be taken to determine the knowledge points recalled depending on the category to which the number of results of the rules engine belongs.
As an example, if the number of found knowledge points is less than or equal to three, directly outputting answers to the user terminal; if the answers of the knowledge points are more than three, executing step 35 to perform fine sorting on the knowledge points by using a sorting engine, and outputting the three answers which are most relevant; if the mapping rules do not result, execution proceeds to step 36 where the search ranking engine is used to obtain the most relevant answer using the business prediction and tertiary category prediction information.
Fig. 7 is a flowchart of a method for search fine-ranking based on the map guide information according to an embodiment of the present specification, and referring to fig. 7, if the number of results output by the map guide rule engine is greater than 3, the results are directly used as recall results, the results are directly ranked by using the search fine-ranking system, and the top-ranked 3(top 3) result is output. If the rule engine has no result, business prediction and three-level category prediction information are needed, a searched full-text search engine (lucene) recall module is used for retrieving the knowledge point recall result from the knowledge base, and the business prediction and the three-level category prediction are used as filtering conditions and applied to filtering the knowledge points in the knowledge base. In the fine sorting process, various feature scores of the question and the title of the knowledge point of the user, such as WMD score, Second Sort score, intention tree score and the like, are extracted, all the scores are sorted through a fine LamdamAT model, and a top 3 result with the highest sorting score is output to a user end as a final knowledge point result.
In the embodiment of the specification, the overall algorithm of knowledge point recall is used for putting test points on a health insurance service line paying for precious customer service, service line operators can better participate in the construction process of a service guide map in the process of putting the algorithm, and when a matching bad case (badcase) is found, the service capability of the service line operators can be quickly repaired by modifying the service guide map nodes and other methods, so that the service capability of the service line operators is well released into the customer service matching algorithm. Since the engine based on the map guide recall matching is on-line, the overall self-help conversion labor rate presents a descending trend, the matching rate of the engine on the client problem covered by the mark reaches nearly 80%, meanwhile, in the continuous expansion and popularization of various services of the health risk, the map guide recall engine can well deal with the rapid change of the service logic, and the hysteresis of the algorithm model is effectively improved.
According to another embodiment, an apparatus for recalling knowledge points is further provided, where the apparatus is performed based on a plurality of service guide maps established in advance, each service guide map includes a plurality of nodes organized into a tree-like hierarchical structure according to service dimensions, the plurality of nodes includes a root node, an intermediate node, and a leaf node, the root node represents a service with a specific service type, the intermediate node represents service knowledge keywords of different levels related to the service, and the leaf node mounts a knowledge point in a knowledge base associated with the leaf node. FIG. 8 shows a schematic block diagram of an apparatus for knowledge point recall according to one embodiment. As shown in fig. 8, the apparatus 800 includes:
a service matching unit 81, configured to determine a matching service corresponding to a question of a user, where the matching service is a service having the specific service type;
a guide map matching unit 82, configured to select a service guide map, of which a root node is determined by the service matching unit 81, from the multiple service guide maps, and determine the service guide map as a matching service guide map corresponding to the question of the user;
a knowledge point matching unit 83, configured to determine, in the matching service guide map determined by the guide map matching unit 82, a matching knowledge point matching the question of the user by using a pre-trained rule matching model;
a knowledge point recall unit 84 for determining a knowledge point recalled for the user question based on a result of the knowledge point matching unit 83 determining a matched knowledge point.
Optionally, as an embodiment, the service matching unit 81 is specifically configured to determine, when a word segmentation result of the user question includes a word segmentation for indicating a service, the service indicated by the word segmentation as a matching service corresponding to the user question.
Optionally, as an embodiment, the service matching unit 81 is specifically configured to:
acquiring historical behavior data of the user corresponding to the question of the user;
and determining the service indicated by the historical behavior data as a matching service corresponding to the user question.
Optionally, as an embodiment, the service matching unit 81 is specifically configured to:
acquiring scene embedded point information corresponding to the question of the user;
and determining the service corresponding to the scene indicated by the scene embedded point information as the matching service corresponding to the question of the user.
Optionally, as an embodiment, the knowledge point recalling unit 84 is specifically configured to, when the number of the matched knowledge points determined by the knowledge point matching unit 83 is less than or equal to a preset threshold, take the matched knowledge points as knowledge points recalled for the question of the user.
Optionally, as an embodiment, the knowledge recall unit 84 is specifically configured to:
when the number of the matched knowledge points determined by the knowledge point matching unit 83 is greater than a preset threshold value, sorting the matched knowledge points according to the matching degree with the question of the user from high to low;
and selecting the matched knowledge points with the number of preset threshold values in the front sequence as the knowledge points recalled aiming at the question of the user.
Optionally, as an embodiment, the apparatus further includes:
the node matching unit is used for determining a preset hierarchical matching node matched with the matching service guide map by the user question by utilizing a pre-trained classification model;
the knowledge point recalling unit 84 is specifically configured to, when the knowledge point matching unit 83 does not determine the matching knowledge point, take the user question, the matching node determined by the node matching unit, and the matching service determined by the service matching unit 81 as inputs, search knowledge points in a knowledge base by using a search ranking system, and take the number of the searched knowledge points with the number of the preset threshold values as the knowledge points recalled for the user question according to the ranking.
Further, the training samples of the pre-trained classification model include:
taking any one item in a knowledge point title and a knowledge point corresponding question set corresponding to the knowledge point as an input sample;
and taking the node of the preset level to which the knowledge point belongs in the service guide map as an output sample.
With the apparatus provided in this specification, first, a service matching unit 81 determines a matching service corresponding to a question of a user, where the matching service is a service with a specific service type, then a guide map matching unit 82 selects a root node from a plurality of service guide maps as a service guide map of the matching service, determines the service guide map as a matching service guide map corresponding to the question of the user, then a knowledge point matching unit 83 determines a matching knowledge point matching the question of the user in the matching service guide map by using a pre-trained rule matching model, and finally a knowledge point recall unit 84 determines a knowledge point recalled for the question of the user based on a result of determining the matching knowledge point by the knowledge point matching unit 83. As can be seen from the above, in the embodiments of the present specification, the knowledge point recall is performed based on the pre-established knowledge guide diagram, so that the operator can influence the result of the knowledge point recall by adjusting the knowledge guide diagram, and thus the question and answer effect of the robot is effectively improved when the knowledge point recall is performed for the question of the user.
According to an embodiment of another aspect, there is also provided a computer-readable storage medium having stored thereon a computer program which, when executed in a computer, causes the computer to perform the method described in connection with any of fig. 2 to 7.
According to an embodiment of yet another aspect, there is also provided a computing device comprising a memory and a processor, the memory having stored therein executable code, the processor, when executing the executable code, implementing the method described in conjunction with any of fig. 2 to 7.
Those skilled in the art will recognize that, in one or more of the examples described above, the functions described in this invention may be implemented in hardware, software, firmware, or any combination thereof. When implemented in software, the functions may be stored on or transmitted over as one or more instructions or code on a computer-readable medium.
The above-mentioned embodiments, objects, technical solutions and advantages of the present invention are further described in detail, it should be understood that the above-mentioned embodiments are only exemplary embodiments of the present invention, and are not intended to limit the scope of the present invention, and any modifications, equivalent substitutions, improvements and the like made on the basis of the technical solutions of the present invention should be included in the scope of the present invention.
Claims (16)
1. A method of recalling knowledge points, the method being performed based on a plurality of pre-established service guide maps, each service guide map comprising a plurality of nodes organized into a tree-like hierarchical structure according to service dimensions, the plurality of nodes comprising a root node, an intermediate node and a leaf node, the root node representing a service having a specific service type, the intermediate node representing a service knowledge keyword of a different hierarchy related to the service, the leaf node mounting a knowledge point in a knowledge base associated with the leaf node, the method comprising:
determining a matching service corresponding to the question of the user, wherein the matching service is a service with the specific service type;
selecting a root node from the plurality of service guide graphs as a service guide graph of the matched service, and determining the service guide graph as a matched service guide graph corresponding to the question of the user;
in the matching service guide map, determining a matching knowledge point matched with the question of the user by using a pre-trained rule matching model;
determining knowledge points recalled for the user question based on a result of determining matching knowledge points;
wherein the method further comprises:
determining a preset level matching node matched with the matching service guide diagram by the user question by utilizing a pre-trained classification model;
the determining knowledge points recalled for the user question based on the determining the matching knowledge points comprises:
and when the matched knowledge points are not determined, taking the user question, the matched nodes and the matched services as input, searching the knowledge points in a knowledge base by using a search sorting system, and taking the searched knowledge points with the number of the front preset threshold values as the knowledge points recalled aiming at the user question according to the sorting.
2. The method of claim 1, wherein the determining the matching service corresponding to the user question comprises:
and when the word segmentation result of the user question sentence comprises the word segmentation used for indicating the service, determining the service indicated by the word segmentation as the matching service corresponding to the user question sentence.
3. The method of claim 1, wherein the determining the matching service corresponding to the user question comprises:
acquiring historical behavior data of the user corresponding to the question of the user;
and determining the service indicated by the historical behavior data as a matching service corresponding to the user question.
4. The method of claim 1, wherein the determining the matching service corresponding to the user question comprises:
acquiring scene embedded point information corresponding to the question of the user;
and determining the service corresponding to the scene indicated by the scene embedded point information as the matching service corresponding to the question of the user.
5. The method of claim 1, wherein the determining knowledge points for the user question recall based on the results of determining matching knowledge points comprises:
and when the determined number of the matched knowledge points is less than or equal to a preset threshold value, taking the matched knowledge points as knowledge points recalled for the question of the user.
6. The method of claim 1, wherein the determining knowledge points for the user question recall based on the results of determining matching knowledge points comprises:
when the number of the determined matching knowledge points is larger than a preset threshold value, sequencing the matching knowledge points according to the matching degree with the question of the user from high to low;
and selecting the matched knowledge points with the number of preset threshold values in the front sequence as the knowledge points recalled aiming at the question of the user.
7. The method of claim 1, wherein the training samples of the pre-trained classification model comprise:
taking any one item in a knowledge point title and a knowledge point corresponding question set corresponding to the knowledge point as an input sample;
and taking the node of the preset level to which the knowledge point belongs in the service guide map as an output sample.
8. An apparatus for recalling knowledge points, the apparatus being performed based on a plurality of pre-established service guide maps, each service guide map comprising a plurality of nodes organized into a tree-like hierarchical structure according to service dimensions, the plurality of nodes comprising a root node, an intermediate node and a leaf node, the root node representing a service with a specific service type, the intermediate node representing a service knowledge keyword of a different hierarchy related to the service, the leaf node mounting a knowledge point in a knowledge base associated with the leaf node, the apparatus comprising:
a service matching unit, configured to determine a matching service corresponding to a question of a user, where the matching service is a service with the specific service type;
a guide map matching unit, configured to select a service guide map, which is determined by the service matching unit and matches a service by a root node, from the multiple service guide maps, and determine the service guide map as a matching service guide map corresponding to the question of the user;
a knowledge point matching unit, configured to determine, in the matching service guide map determined by the guide map matching unit, matching knowledge points that match the question of the user by using a pre-trained rule matching model;
a knowledge point recall unit for determining a knowledge point recalled for the question of the user based on a result of the knowledge point matching unit determining the matched knowledge point;
wherein the apparatus further comprises:
the node matching unit is used for determining a preset hierarchical matching node matched with the matching service guide map by the user question by utilizing a pre-trained classification model;
the knowledge point recalling unit is specifically configured to, when the knowledge point matching unit does not determine the matching knowledge point, take the user question sentence, the matching node determined by the node matching unit, and the matching service determined by the service matching unit as inputs, search for the knowledge point in a knowledge base by using a search ranking system, and take the number of the searched knowledge points with the number of the preset threshold values as the knowledge point recalled for the user question sentence according to the ranking.
9. The apparatus according to claim 8, wherein the service matching unit is specifically configured to, when a word segmentation result of the user question includes a word segmentation for indicating a service, determine the service indicated by the word segmentation as a matching service corresponding to the user question.
10. The apparatus according to claim 8, wherein the service matching unit is specifically configured to:
acquiring historical behavior data of the user corresponding to the question of the user;
and determining the service indicated by the historical behavior data as a matching service corresponding to the user question.
11. The apparatus according to claim 8, wherein the service matching unit is specifically configured to:
acquiring scene embedded point information corresponding to the question of the user;
and determining the service corresponding to the scene indicated by the scene embedded point information as the matching service corresponding to the question of the user.
12. The apparatus according to claim 8, wherein the knowledge point recalling unit is specifically configured to regard the matching knowledge points as knowledge points recalled for the question of the user when the number of the matching knowledge points determined by the knowledge point matching unit is less than or equal to a preset threshold.
13. The apparatus of claim 8, wherein the knowledge recall unit is specifically configured to:
when the number of the matched knowledge points determined by the knowledge point matching unit is larger than a preset threshold value, sequencing the matched knowledge points from high to low according to the matching degree of the matched knowledge points with the question of the user;
and selecting the matched knowledge points with the number of preset threshold values in the front sequence as the knowledge points recalled aiming at the question of the user.
14. The apparatus of claim 8, wherein the training samples of the pre-trained classification model comprise:
taking any one item in a knowledge point title and a knowledge point corresponding question set corresponding to the knowledge point as an input sample;
and taking the node of the preset level to which the knowledge point belongs in the service guide map as an output sample.
15. A computer-readable storage medium, on which a computer program is stored which, when executed in a computer, causes the computer to carry out the method of any one of claims 1-7.
16. A computing device comprising a memory having stored therein executable code and a processor that, when executing the executable code, implements the method of any of claims 1-7.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201811427653.2A CN109299245B (en) | 2018-11-27 | 2018-11-27 | Method and device for recalling knowledge points |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201811427653.2A CN109299245B (en) | 2018-11-27 | 2018-11-27 | Method and device for recalling knowledge points |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN109299245A CN109299245A (en) | 2019-02-01 |
| CN109299245B true CN109299245B (en) | 2021-12-07 |
Family
ID=65143804
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201811427653.2A Active CN109299245B (en) | 2018-11-27 | 2018-11-27 | Method and device for recalling knowledge points |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN109299245B (en) |
Families Citing this family (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110209790B (en) * | 2019-06-06 | 2023-08-25 | 创新先进技术有限公司 | Question-answer matching method and device |
| CN110377803B (en) * | 2019-06-14 | 2023-04-07 | 创新先进技术有限公司 | Information processing method and device |
| CN110400219B (en) * | 2019-06-14 | 2024-05-17 | 创新先进技术有限公司 | Service processing method and system, and transaction monitoring method and system |
| CN110362668A (en) * | 2019-07-16 | 2019-10-22 | 阿里巴巴集团控股有限公司 | For the classification method and device of user's question sentence |
| CN112256850A (en) * | 2019-12-31 | 2021-01-22 | 北京来也网络科技有限公司 | Data processing method, equipment and storage medium combining RPA and AI |
| CN111461753B (en) * | 2020-04-17 | 2022-05-17 | 支付宝(杭州)信息技术有限公司 | Method and device for recalling knowledge points in intelligent customer service scene |
| CN113343089B (en) * | 2021-06-11 | 2024-09-06 | 北京完美赤金科技有限公司 | User recall method, device and equipment |
| CN117633208B (en) * | 2024-01-26 | 2024-08-06 | 北京网藤科技有限公司 | Knowledge recall fusion method and system based on reciprocal ordering |
| CN118535715B (en) * | 2024-07-26 | 2024-10-29 | 四川声通玄武信息科技有限公司 | Automatic reply method, equipment and storage medium based on tree structure knowledge base |
Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN105184709A (en) * | 2015-08-20 | 2015-12-23 | 浙江通关教育科技有限公司 | Subject evaluation system and method based on knowledge point system |
| CN105810045A (en) * | 2016-03-09 | 2016-07-27 | 深圳大学 | Cognitive model-based computational thinking training method and device |
Family Cites Families (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US10002185B2 (en) * | 2015-03-25 | 2018-06-19 | International Business Machines Corporation | Context-aware cognitive processing |
-
2018
- 2018-11-27 CN CN201811427653.2A patent/CN109299245B/en active Active
Patent Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN105184709A (en) * | 2015-08-20 | 2015-12-23 | 浙江通关教育科技有限公司 | Subject evaluation system and method based on knowledge point system |
| CN105810045A (en) * | 2016-03-09 | 2016-07-27 | 深圳大学 | Cognitive model-based computational thinking training method and device |
Also Published As
| Publication number | Publication date |
|---|---|
| CN109299245A (en) | 2019-02-01 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN109299245B (en) | Method and device for recalling knowledge points | |
| CN109033229B (en) | Question and answer processing method and device | |
| CN110069607B (en) | Methods, devices, electronic devices, and computer-readable storage media for customer service | |
| CN108073568B (en) | Keyword extraction method and device | |
| CN110032623B (en) | Method and device for matching question of user with title of knowledge point | |
| CN109947902B (en) | Data query method and device and readable medium | |
| CN111125334A (en) | Search question-answering system based on pre-training | |
| JP2020512651A (en) | Search method, device, and non-transitory computer-readable storage medium | |
| CN119719312B (en) | Intelligent government affair question-answering method, device, equipment and storage medium | |
| CN108920543A (en) | query and interaction method and device, computer device and storage medium | |
| CN109376219A (en) | Matching process, device, electronic equipment and the storage medium of text attributes field | |
| KR20200139008A (en) | User intention-analysis based contract recommendation and autocomplete service using deep learning | |
| EP3815026B1 (en) | Systems and methods for identifying and linking events in structured proceedings | |
| CN117668199A (en) | Intelligent customer service question-answer prediction and recommendation dialogue generation method and device | |
| CN118761735A (en) | An electronic contract management method and system based on deep learning model | |
| CN117725169A (en) | Legal question answering method and device based on large model and knowledge guidance | |
| CN118260396A (en) | Bank system question and answer method, device, equipment, medium and program product | |
| CN113688633B (en) | Method and device for determining outline | |
| CN114328820B (en) | Information search method and related equipment | |
| CN119938824A (en) | Interaction method and related equipment | |
| CN110851560A (en) | Information retrieval method, device and equipment | |
| CN118761844A (en) | Information recommendation method, system and device | |
| CN118277401A (en) | A natural language query method and device | |
| CN118394894A (en) | End-to-end agent financial model method suitable for multi-mode financial analysis task | |
| CN111984859A (en) | Recommended method, device, electronic device and storage medium for business card |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| TA01 | Transfer of patent application right | ||
| TA01 | Transfer of patent application right |
Effective date of registration: 20201012 Address after: Cayman Enterprise Centre, 27 Hospital Road, George Town, Grand Cayman Islands Applicant after: Innovative advanced technology Co.,Ltd. Address before: Cayman Enterprise Centre, 27 Hospital Road, George Town, Grand Cayman Islands Applicant before: Advanced innovation technology Co.,Ltd. Effective date of registration: 20201012 Address after: Cayman Enterprise Centre, 27 Hospital Road, George Town, Grand Cayman Islands Applicant after: Advanced innovation technology Co.,Ltd. Address before: A four-storey 847 mailbox in Grand Cayman Capital Building, British Cayman Islands Applicant before: Alibaba Group Holding Ltd. |
|
| GR01 | Patent grant | ||
| GR01 | Patent grant |