CN116991969B - Configurable grammatical relationship retrieval method, system, electronic device and storage medium - Google Patents

Abstract

The invention discloses a search method of a configurable grammar relation, which comprises the steps of obtaining a target to be searched, wherein the target to be searched comprises a search text, and dividing the search text by taking sentences as units; performing regular matching on the search term list and the database, and outputting full-quantity terms conforming to the regular matching rule; identifying whether a grammar relation configuration table exists in the target to be searched, if so, extracting the grammar relation configuration table, and if not, constructing the grammar relation configuration table; and carrying out grammar relation matching on the grammar relation configuration table and the full quantity entry list, outputting entries conforming to grammar relation matching rules, and obtaining a final matching list. The method for searching the configurable grammar relationship in the embodiment of the invention searches through configuring the grammar relationship of the search word and uses a graph algorithm to implement, thereby reducing the calculation complexity; the method has higher simplicity and flexibility, does not need to enumerate the grammar relation of fuzzy matching parts, and is suitable for sentences with more complex grammar relation.

Description

Retrieval method, system, electronic device and storage medium for configurable grammatical relations

Technical Field

The present invention relates to the field of natural language processing, and in particular to a retrieval method, system, electronic device and storage medium for configurable grammatical relations.

Background Art

Grammatical relations such as part of speech and syntactic structure can describe the complex relationship between individual words and words within a sentence. With the emergence of large-scale sentence grammatical relationship annotation datasets, the performance of grammatical relationship analysis (such as part of speech tagging and syntactic analysis) has been greatly improved, and it has been widely used in many different fields, such as information retrieval. On the one hand, as one of the basic components of natural language processing technology, grammatical relationship analysis has been relatively mature and there are many open source software that can be used out of the box. On the other hand, compared with model training, the time and economic costs of grammatical relationship analysis are much smaller. In the retrieval system, the traditional string matching method has the defect of not being able to match semantics. Grammatical relationship analysis can understand the multiple expressions of the same semantics by describing the internal relationship of the sentence.

At present, there have been many studies trying to solve these problems by using grammatical relations. For example, the Chinese invention patent with application publication number CN106716408A discloses a semantic text search method, which performs syntactic analysis and part-of-speech tagging on the search text, and constructs search trees of different semantic categories according to different syntactic structures. In this way, the search term will only match the relevant semantic categories, thereby eliminating noise; the Chinese invention patent with application publication number CN113407739A discloses a method, device and storage medium for determining concepts in information titles, wherein the search text is subjected to dependency syntactic analysis, and the conceptual trunk of the title (such as a noun phrase: "how to make bubble glue") is extracted, thereby simplifying and focusing semantics, removing irrelevant information, and optimizing the search effect; the Chinese invention patent with application publication number CN105786963A discloses a corpus search method and system, providing a new type of search expression that can complete the mixed search of regular and syntactic structures. In particular, the search expression can flexibly describe fuzzy search: only specify grammatical relations, and no specific words need to be given. By performing syntactic analysis and part-of-speech tagging on the search text, we can eventually match the results that meet the search expression.

The above solutions all achieve the purpose of improving the search effect by embedding grammatical relations and making lightweight changes. However, the processing of grammatical relations is performed on the text to be searched. For the search terms themselves, only fuzzy search is supported, and the configuration of the grammatical relations of the search terms themselves is not supported. The problem is that more flexible search requirements cannot be met, and on the other hand, complex sentence searches cannot be handled.

Summary of the invention

In response to the above-mentioned defects, an embodiment of the present invention discloses a retrieval method, system, electronic device and storage medium with configurable grammatical relations, which performs retrieval by configuring the grammatical relations of the search terms themselves and implements them using graph algorithms to reduce computational complexity. It has higher simplicity and flexibility, does not require enumeration of the grammatical relations of the fuzzy matching part, and is suitable for sentences with more complex grammatical relations.

A first aspect of an embodiment of the present invention discloses a retrieval method for a configurable grammatical relationship, comprising:

Obtaining a target to be searched, the target to be searched includes a search text, and dividing the search text into sentences; the divided individual sentences form search terms, and all the divided sentences form a search term list;

Based on the search term list, regular matching is performed with the database, and all terms that meet the regular matching rules are output to form a full term list;

Identify whether there is a grammatical relationship configuration table in the target to be searched, if yes, extract the grammatical relationship configuration table, if no, construct the grammatical relationship configuration table;

Based on the grammatical relationship configuration table and the full term list, grammatical relationship matching is performed, and terms that meet the grammatical relationship matching rules are output to obtain a final matching list.

As an optional implementation, in the first aspect of the embodiment of the present invention, the search terms include several participles; the grammatical relationship configuration table includes the grammatical relationship between the participles, and the grammatical relationship includes the dependency relationship between the participles and the part of speech of the participles themselves.

As an optional implementation, in the first aspect of the embodiment of the present invention, the grammatical relationship storage format is: termSRC|posSRC, termDST|posDST, dep;

Wherein, termSRC is the dominant word, posSRC is the part of speech of the dominant word, termDST is the subordinate word, posDST is the part of speech of the subordinate word, and dep is the dependency relationship between the dominant word and the subordinate word;

The dominant word is a specific word, a fuzzy word or an unknown word; when the dominant word is a fuzzy word, termSRC=*; when the dominant word is an unknown word, termSRC=[n], (n=0, 1, 2, ..., n);

The subordinate word is a specific word, a fuzzy word or an unknown word; when the subordinate word is a fuzzy word, termDST=*; when the dominant word is an unknown word, termDST=[n], (n=0, 1, 2, ..., n);

The part of speech is a specific part of speech or a fuzzy part of speech; when the part of speech is a fuzzy part of speech, posSRC=*;

The dependency relationship is a specific dependency relationship or a fuzzy dependency relationship. When the dependency relationship is a fuzzy dependency relationship, dep=*.

As an optional implementation, in the first aspect of the embodiment of the present invention, in the step of identifying whether there is a grammatical relationship configuration table in the target to be searched, extracting the grammatical relationship configuration table if there is, and constructing the grammatical relationship configuration table if there is not, the method for constructing the grammatical relationship configuration table includes:

Randomly sampling the full list of terms to obtain a sample list of terms;

Based on the sample word list and the grammatical relationship analysis method, obtaining a sample grammatical relationship list;

Constructing grammatical relationship options based on the sample grammatical relationship list, and statistically forming a grammatical relationship option list;

The grammatical relationship requirement option information is read, and a grammatical relationship configuration table is generated based on the grammatical relationship requirement option information and the grammatical relationship option list.

As an optional implementation, in the first aspect of the embodiment of the present invention, the step of performing grammatical relationship matching based on the grammatical relationship configuration table and the full term list, outputting terms that meet the grammatical relationship matching rules, and obtaining the final matching list includes:

Based on the full word list and the grammatical relationship analysis method, a full grammatical relationship list is obtained;

Extracting entries in the full grammatical relationship list that have the same dependency information as the grammatical relationship configuration table to obtain a matching entry list;

Extract the entries in the matching entry list whose part-of-speech information is consistent with that in the grammatical relationship configuration table to obtain a final matching list.

As an optional implementation manner, in a first aspect of the embodiment of the present invention, the grammatical relationship analysis method includes:

Segmenting each term in the sample term list or the full term list to obtain segmentation information of each term to form a first nested list;

Based on the first nested list and the dependency syntax training model, dependency syntax analysis is performed on each entry to obtain dependency relationship information between each word of each entry, thereby forming a second nested list;

Performing part-of-speech tagging on the word segments of each entry in the first nested list to obtain the part-of-speech information of each word segment of each entry to form a third nested list;

A sample grammatical relationship list or a full grammatical relationship list is constructed based on the first nested list, the second nested list and the third nested list.

As an optional implementation, in the first aspect of the embodiment of the present invention, the step of constructing a sample grammatical relationship list or a full grammatical relationship list based on the first nested list, the second nested list and the third nested list includes:

Creating a bidirectional weighted graph based on the first nested list, the second nested list, and the third nested list,

The starting point of the bidirectional weighted graph is termSRC|posSRC, where termSRC is the dominant word in the dependency information, and posSRC is the part of speech of the dominant word;

The end point of the bidirectional weighted graph is termDST|posDST, where termDST is the dependent word in the dependency information, and posDST is the part of speech of the dependent word;

The edge of the bidirectional weighted graph is dep, where is the dependency relationship between the dominant word and the subordinate word in the dependency relationship information;

The information of each node of the bidirectional weighted graph is converted into a graph data structure, and a sample grammatical relationship list or a full grammatical relationship list is constructed based on the graph data structure.

A second aspect of an embodiment of the present invention discloses a retrieval system capable of configuring grammatical relations, comprising:

An input module, the input module is used to input or read the target to be retrieved;

A text segmentation module, the text segmentation module is used to identify the search text in the target to be searched, and segment the search text to form a search term list;

A first matching module, the first matching module is used to perform regular matching on the search word list and the database to form a full word list;

A grammar configuration identification module, the grammar configuration identification module is used to identify whether there is a grammar relationship configuration table in the target to be searched;

A grammar configuration building module, wherein the grammar configuration module is used to build a grammar configuration table;

A second matching module, the second matching module is used to perform grammatical relationship matching with the full word list according to the grammatical relationship configuration table to form a final matching list;

The display module is used to output the final matching list and display it in order.

A third aspect of an embodiment of the present invention discloses an electronic device, comprising: a memory storing executable program code; a processor coupled to the memory; the processor calls the executable program code stored in the memory to execute the retrieval method of configurable grammatical relations disclosed in the first aspect of the embodiment of the present invention.

A fourth aspect of an embodiment of the present invention discloses a computer-readable storage medium storing a computer program, wherein the computer program enables a computer to execute the retrieval method of a configurable grammatical relationship disclosed in the first aspect of an embodiment of the present invention.

Compared with the prior art, the embodiments of the present invention have the following beneficial effects:

The retrieval method with configurable grammatical relations in the embodiment of the present invention matches and searches by configuring the grammatical relations of the search terms themselves, has higher simplicity and flexibility, does not need to enumerate the grammatical relations of the fuzzy matching part, and is suitable for sentences with more complex grammatical relations. By introducing the processing method of the graph algorithm, the computational efficiency of the entire retrieval matching is greatly improved. This retrieval method is very practical in many applications, and the database therein can be implemented as a large database, corpus, target file library, etc. according to the actual application scenario. For example, in the statistics of keywords of public opinion information, using the retrieval method of this embodiment, keywords can be retrieved through a more maintainable configuration template, thereby improving the effect of public opinion monitoring. For example, in linguistics and literary research, it is very important to use examples in existing corpora or literature to support one's own views. Through the retrieval method of this embodiment, examples of specific grammatical relations in the corpus can be retrieved more easily and accurately, accelerating the progress of scientific research.

BRIEF DESCRIPTION OF THE DRAWINGS

In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the drawings required for use in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

FIG1 is a schematic flow chart of a configurable grammatical relationship retrieval method disclosed in an embodiment of the present invention;

FIG2 is a dependency syntax tree of the sentence “This is mine” according to an embodiment of the present invention;

FIG3 is a dependency syntax tree of sentence 1) according to an embodiment of the present invention;

FIG4 is a dependency syntax tree of sentence 2) according to an embodiment of the present invention;

FIG5 is a dependency syntax tree of sentence 3) according to an embodiment of the present invention;

FIG6 is a dependency syntax tree of sentence 4) according to an embodiment of the present invention;

FIG7 is a schematic diagram of a specific flow chart of step S3 of an embodiment of the present invention;

FIG8 is a schematic diagram of a specific flow chart of step S32 of an embodiment of the present invention;

FIG9 is a schematic diagram of a specific flow chart of step S324 of an embodiment of the present invention;

FIG10 is a schematic diagram of a specific flow chart of step S4 of an embodiment of the present invention;

11 is a schematic diagram of the structure of a retrieval system with configurable grammatical relations provided in an embodiment of the present invention;

FIG. 12 is a schematic diagram of the structure of an electronic device provided by an embodiment of the present invention.

DETAILED DESCRIPTION

The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

It should be noted that the terms "first", "second", "third", "fourth", etc. in the specification and claims of the present invention are used to distinguish different objects rather than to describe a specific order. The terms "including" and "having" in the embodiments of the present invention and any variations thereof are intended to cover non-exclusive inclusions. For example, a process, method, system, product or device including a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

In the current retrieval methods, the processing of grammatical relations is performed on the text to be retrieved. For the search terms themselves, only the fuzzy search part is supported, and the grammatical relations of the search terms themselves cannot be configured for retrieval. Therefore, it is unable to meet the retrieval needs for more flexible or complex sentences. Based on this, the embodiments of the present invention disclose a retrieval method, system, electronic device and storage medium with configurable grammatical relations, which performs retrieval by configuring the grammatical relations of the search terms themselves and uses a graph algorithm for implementation, thereby reducing computational complexity; it has higher simplicity and flexibility, does not require enumeration of the grammatical relations of the fuzzy matching part, and is suitable for sentences with more complex grammatical relations.

Embodiment 1

Please refer to Figures 1-10. Figure 1 is a flowchart of a method for retrieving configurable grammatical relations disclosed in an embodiment of the present invention. The method is applicable to smart devices such as mobile phones and tablet computers with processing functions, as well as computing devices such as computers and servers. As shown in Figure 1, the method for retrieving configurable grammatical relations includes the following steps:

Step S1: obtaining a target to be searched, the target to be searched includes a search text, and dividing the search text into sentences; the divided individual sentences form search terms, and all the divided sentences form a search term list.

Step S2: Perform regular expression matching on the search term list and the database, output all entries that meet the regular expression matching rules, and form a full entry list R.

Step S3: Identify whether there is a grammatical relationship configuration table in the target to be retrieved, if so, extract the grammatical relationship configuration table, if not, construct a grammatical relationship configuration table.

Step S4: Perform grammatical relationship matching based on the grammatical relationship configuration table and the full term list R, output terms that meet the grammatical relationship matching rules, and obtain a final matching list.

This example uses the search term "strike *evil forces" ("*" represents any item) as an example, assuming that the purpose of this search is to obtain the following matching results:

1) Crack down on cyber-related criminal gangs in S City. (“S City cyber-related” here refers to any part of the content)

2) Increase the intensity of the crackdown on evil forces. (Here, "increase" and "intensity" are sentence components other than the search terms in the original sentence)

3) Severely crack down on any form of evil forces in accordance with the law. (Here, "any form" refers to any part of the item, and "severely in accordance with the law" is a sentence component other than the search term in the original sentence)

First, the search term is converted into a regular expression "Strike.*? Evil forces". The full term list R is obtained by regular matching with the terms in the database.

In the prior art, if it is necessary to accurately match the above 1) 2) 3) sentences, three different corresponding search expressions are required to describe the grammatical relationship; in this embodiment, by introducing the search conditions of the grammatical relationship configuration table, the above three sentences are classified into one category, and the matching results can be obtained.

Dependency syntactic theory believes that there is a master-slave relationship between words, namely, the "head" and "dependent", which can be called a "dependency relation". Please refer to Figure 2. Taking the sentence "This is mine" as an example, its dependency syntactic relation tree is that the head "I" points to the dependent "this", and the dependency relation between the two is "nsubj (noun subject)", and the part of speech of "I" and "this" is "PN (pronoun)". For specific labels and their interpretations, please refer to Tables 2 and 3.

By performing syntactic analysis on sentences 1), 2), and 3), the corresponding dependency syntactic trees are shown in Figures 3-5. From the above dependency syntactic trees, it can be seen that through the solution of this embodiment, only one dependency syntactic relationship condition is needed for the search term "strike against *evil forces", that is, the dependency relationship between the dominant term "strike against" and the subordinate term "force" after regular matching is "dobj (direct object)". In this way, the three sentences with different grammatical relationships as a whole can be classified into one category without the need to design three separate search conditions, which saves time cost and increases the robustness of the search.

By using the method of this embodiment, when the following sentence appears: 4) Crack down on corruption and punish evil forces. (Here, "crack down on corruption and punish" is any part of the content of the item), the dependency syntax tree of sentence 4) is shown in FIG6 , which can be successfully matched by regular expression, but because the grammatical relationship does not conform to the preset grammatical relationship configuration table, it will not be finally matched.

This embodiment proposes a new retrieval method: by configuring the grammatical relationship of the search term itself for matching and searching, it has higher simplicity and flexibility, does not need to enumerate the grammatical relationship of the fuzzy matching part, and is suitable for sentences with more complex grammatical relationships. This matching method is very practical in many applications, and the database therein can be implemented as a large database, corpus, target file library, etc. according to the actual application scenario. For example, in the statistics of keywords of public opinion information, using the retrieval method of this embodiment, keywords can be retrieved through a more maintainable configuration template, thereby improving the effect of public opinion monitoring. For example, in linguistics and literary research, it is very important to use examples in existing corpora or literature to support your own views. Through the retrieval method of this embodiment, examples of specific grammatical relationships in the corpus can be retrieved more easily and accurately, accelerating the progress of scientific research.

Specifically, in step S1, in order to ensure that the constructed dependency syntax tree is more reasonable and avoid affecting the complexity and rationality of the dependency relationship. Therefore, this embodiment performs sentence segmentation on the search text in step S1. The segmentation rule is to use punctuation marks ".", "?", and "!" as separators. For example: "Li Bai (701-762), with the name Taibai, the name Qinglian Jushi, and the name 'Zhexianren'. He is a great romantic poet in the Tang Dynasty. He was praised as the 'Poet Immortal' by later generations, and was called 'Li Du' together with Du Fu." It will be divided into three sentences to form three search terms, namely "Li Bai (701-762), with the name Taibai, the name Qinglian Jushi, and the name 'Zhexianren'", "He is a great romantic poet in the Tang Dynasty" and "He was praised as the 'Poet Immortal' by later generations, and was called 'Li Du' together with Du Fu." The collection of these three search terms forms a search term list.

In step S2, the search terms in the search term list are first matched with the terms in the database by regular expression matching. If the matching result is empty, it means that there is no expected result in the database, and there is no need to enter the next step and the no matching result can be directly output. If there is a matching result output in this step, the terms that meet the regular matching rules are output and named as full terms, and all the matched full terms are collected to form a full term list R.

In step S3, it is necessary to identify whether there is a grammatical relationship configuration table in the target to be searched. This embodiment provides a variety of methods for obtaining grammatical relationship configuration tables for different users. For example, for users who have clear matching requirements and clearly know the dependency relationship of the search terms and can directly provide the configuration table, the grammatical relationship configuration table is directly extracted to proceed to the next step. For users who fail to clearly translate their search requirements into dependency syntax search, it is necessary to build a grammatical configuration table for them.

In this embodiment, the search terms include several segmented words; the grammatical relationship configuration table includes the grammatical relationship between the segmented words in each search term, and the grammatical relationship includes the dependency relationship information between the segmented words and the part-of-speech information of the segmented words themselves. In some embodiments, the grammatical relationship configuration table is directly stored in the "txt" text format, and each line represents the dependency relationship between two segmented words and the part-of-speech of the segmented words themselves, with carriage returns as the interval between each line. The format of each line is: termSRC|posSRC, termDST|posDST, dep.

Among them, "termSRC" represents the dominant word, "posSRC" represents the part of speech of the dominant word, and the two are separated by "|"; while "termDST" represents the subordinate word, "posDST" represents the part of speech of the subordinate word, and the two are separated by "|"; the last item is the dependency relationship "dep" between the dominant word and the subordinate word. As before, the part of speech and dependency relationship are shown in Table 2-Table 3. In addition, the three categories of dominant word description, subordinate word description and dependency relationship are all separated by ",".

Therefore, in the embodiment of searching the term "strike against *evil forces", if you want to match sentences 1), 2), and 3), according to the dependency information and part-of-speech information, the dependency relationship between the dominant word "strike" (part-of-speech is "VV") and the subordinate word "force" (part-of-speech is "NN") can be set to "dobj (direct object)". In the configuration table, the format is: strike|VV, force|NN, dobj.

Furthermore, if you do not want to specify specific parts of speech and dependencies, you can use "*" to represent any part of speech. In the configuration table, the format is: attack|*, force|NN, dobj.

The above configuration indicates that the dependency relationship between the dominant word "strike" (any part of speech) and the subordinate word "force" (part of speech is "NN") is set to "dobj (direct object)". Similarly, the part of speech of "force" can also be set to "*". The part of speech of the dominant word and the subordinate word can be "*" at the same time. This means that the part of speech is not considered, as long as the dependency relationship, dominant word, and subordinate word match.

Dependency relations can also be arbitrary relations. From the perspective of graph, it is equivalent to changing from a bidirectional weighted graph to a bidirectional unweighted graph, that is, retaining the pointing relationship between the dominant word and the subordinate word, but ignoring the dependency relationship. As long as there is a relationship between "strike" and "force", and "strike" is the dominant word, it is sufficient. In the configuration table, the format is: strike|*, force|*,*.

Furthermore, if you need to match a more complex relationship, for example, if you need to match a sentence like sentence 4), you can specify the dependency relationship between "strike" and the unknown verb in the fuzzy matching part (VV, for example, sentence (4) is "punish") as "conj (connection)". Note that the part of speech of the unknown verb can also be "*", which means that as long as there is a "conj" relationship, it will be fine. In the configuration table, the format is: strike|*, [0]|VV, conj.

A new expression is introduced here. The expression "[0]" exists in the fuzzy search, but does not exist in the unknown word in the search term. If you want to continue to describe the second unknown word, use "[1]". That is, a natural number is contained in the brackets. Note that it needs to be incremented in sequence and cannot be skipped. Usually, such a configuration will use multiple expressions to describe the dependency relationship between the unknown word and multiple words in the sentence. For example, more conditions can be added to ensure matching of sentences such as 4). It is expressed as follows in the configuration table:

hit|*,[0]|VV,conj

[0]|VV, power|NN, dobj

hit|*,[1]|*,cc

The second line indicates that the dependency relationship between the unknown verb and "power" is "dobj". The third line indicates that the relationship between the second unknown word and "strike" is "cc". Please note that when there are multiple expressions in the configuration table, the parts of speech need to be unified, otherwise an error will be reported. For example: the first line cannot be "[0]|VV", and the second line cannot be "[0]|NN" or "[0]|*". The relationship between multiple expressions in the configuration table is "AND" (no need to add explicitly). If you want to perform an "OR" operation, you can configure multiple files. The search function of one search term plus multiple configuration tables is supported.

The method for constructing the grammatical relationship configuration table in step S3 includes:

Step S31: Randomly sample the full term list R to obtain a sample term list Ra.

In this embodiment, the full word list R is randomly sampled at a ratio of 1% to obtain a sample word list Ra. The ratio here can be adjusted according to the size of the database. The principle is to rationally control the amount of sampled result data to avoid putting pressure on the next screening mode.

Step S32: Based on the sample entry list Ra and the grammatical relationship analysis method, a sample grammatical relationship list Ga is obtained. The specific steps include:

Step S321: Segment each term in the sample term list Ra to obtain segmentation information of each term, and form a first nested list Ta.

Before dependency parsing, each search term needs to be segmented first. This embodiment uses the third-party segmentation library "jieba" of python. This is a library specifically used for Chinese segmentation. If there are terms or combination words in the search term such as "Technology Innovation Center" that need to be regarded as a word, the "jieba.add_word" method can be used to add the word to the default dictionary, and it will be regarded as a word during segmentation. After adding, using the "jieba.lcut" method, each search term will get a segmentation list, and the set of the list after segmentation of each search term in the sample term list Ra is a nested list, named the first nested list Ta. For example, in python, the search term "this is mine" is expressed as a string (string), and jieba.lcut ("this is mine") will output the list after segmentation (list): ["this", "is", "I", "of"].

Step S322: performing dependency syntactic analysis on each word based on the first nested list Ta and the dependency syntactic training model to obtain dependency relationship information between each word segment of each word to form a second nested list Da.

This step uses the third-party Python library "hanlp" to perform dependency parsing on each sentence. For ease of demonstration, the following sentence "This is mine" is used as an example. hanlp is a library that supports many Chinese natural language processing tasks.

Use hanlp to load the pre-trained model of dependency syntactic analysis: dep = hanlp.load(hanlp.pretrained.dep.CTB9_UDC_ELECTRA_SMALL). The "CTB9_UDC_ELECTRA_SMALL" used here is an open source pre-trained model of dependency syntactic analysis based on Chinese. hanlp supports the import of pre-trained models in various other languages. The Chinese search in this embodiment is only for the convenience of explanation. The implementation of other languages is within the scope of protection of this patent. After loading, enter the list after word segmentation: dep(["this","is","I","of"]), and you can get the dependency syntactic relationship of this search term: [(3,"nsubj"),(3,"cop"),(0,"root"),(3,"case")]. The result is a list consisting of several tuples, each of which consists of the position number of the word segmentation in the term and the dependency syntactic relationship. It needs to be interpreted in conjunction with the first nested list Ta. Referring to Table 1, the first tuple (3,"nsubj") in the list means that the first participle "this" is a subordinate word, and the third participle ("I") is a dominant word and has a dependency syntactic relationship with it, which is "nsubj". The following tuples are similar. In particular, the third tuple (0,"root") means that the third participle as a subordinate word has no dependency syntactic relationship with other words. After performing dependency syntactic analysis on each element in the first nested list Ta, the second nested list Da is obtained.

Step S323: perform part-of-speech tagging on the word segments of each entry in the first nested list Ta to obtain the part-of-speech information of each word segment of each entry, and form a third nested list Pa.

This step performs part-of-speech tagging on the first nested list Ta to obtain the part-of-speech information of each word segment of each entry. Use the part-of-speech tagging interface of hanlp: pos = hanlp.load (hanlp.pretrained.pos.CTB9_POS_ELECTRA_SMALL). For example: pos (["this", "is", "I", "of"]), get the part-of-speech list: ["PN", "VC", "PN", "DEG"]. Therefore, after the first nested list Ta is processed by part-of-speech tagging, the nested third nested list Pa is obtained.

Step S324: construct a sample grammatical relationship list Ga based on the first nested list Ta, the second nested list Da and the third nested list Pa. The specific steps include:

Step S3241: Create a bidirectional weighted graph based on the first nested list Ta, the second nested list Da and the third nested list Pa.

The starting point of the bidirectional weighted graph is termSRC|posSRC, where termSRC is the dominant word in the dependency information, and posSRC is the part of speech of the dominant word;

The end point of the bidirectional weighted graph is termDST|posDST, where termDST is the dependent word in the dependency information, and posDST is the part of speech of the dependent word;

The edge of the bidirectional weighted graph is dep, where is the dependency relationship between the dominant word and the subordinate word in the dependency relationship information;

Step S3242: Convert the node information of the bidirectional weighted graph into a graph data structure, and construct a sample grammatical relationship list Ga based on the graph data structure.

In terms of implementation, the dependency syntax tree can be viewed as a problem of a bidirectional weighted graph. A graph (or network) is usually composed of nodes and edges. In a bidirectional weighted graph, each edge is directional and has a weight (attribute). Therefore, a triple can be used to represent the relationship from one point to another point - (src, dst, dep). "src" represents the starting point, corresponding to the dominant word in the dependency syntax theory, and "dst" represents the end point, corresponding to the subordinate word. The attribute of the "dep" edge corresponds to the dependency relationship. Therefore, the graph representation method of the sentence "This is mine" is shown in Table 1. After using a graph to represent the dependency syntax tree, the required information can be obtained through graph algorithms or simple node attributes. For example, for a given node, all its connected nodes can be obtained, which greatly reduces the computational complexity and has higher simplicity and flexibility.

Use the third-party python library "networkx" to construct a graph. This is a library commonly used to construct graphs (networks) in python. It has a large number of built-in graph algorithms and can be called directly. Taking "This is mine" as an example, first create a bidirectional graph: g = networkx.DiGraph(). Then, you can use the script to transcribe the relationship of the syntax tree into the relationship described in Table 1, as a nested list in python, and each word is represented by the sequence number of the list after word segmentation: dep = [["3","1","nsubj"],["3","2","cop"],["3","4","case"]]. Add the relationship of this nested list dep to the bidirectional graph just created to obtain a complete bidirectional weighted graph: g.add_weighted_edges_from(dep,weight＝"dep"). Then use the part-of-speech list to further add part-of-speech information: networkx.set_node_attributes(g,{"1":"PN","2":"VC","3":"PN","4":"DEG"},name＝'pos'). Here g represents the created bidirectional graph, which continuously updates information from the empty graph. In the second nested list Da, after the graph construction operation, a sample grammatical relation list Ga is obtained, and the elements in the list are composed of graphs.

Table 1

src dst dep I this nsubj I yes cop I of case

Step S33: construct grammatical relationship options based on the sample grammatical relationship list Ga, and statistically form a grammatical relationship option list.

Step S34: read the grammatical relationship requirement option information, and generate a grammatical relationship configuration table based on the grammatical relationship requirement option information and the grammatical relationship option list.

After the sample grammatical relationship list Ga is constructed, the user can use the python script to count the grammatical relationship items and list the ten items with the highest number. The user can also directly visualize the dependency syntactic relationship, confirm their own search needs, and configure the search term grammatical relationship table. At this point, the search term grammatical relationship table has been constructed. This step is mainly for users who have not built a search term configuration table. Through small-scale sampling, the distribution of dependency syntactic relationships can be understood, and the search conditions can be further confirmed. The configuration of the grammatical table is completed and the search is performed in the next step. The learning curve of this patent is greatly reduced.

In step S4, grammatical relationship matching is performed based on the grammatical relationship configuration table and the full term list R, and terms that meet the grammatical relationship matching rules are output. The specific steps of obtaining the final matching list include:

Step S41: Based on the full word list R and the grammatical relationship analysis method, a full grammatical relationship list G is obtained.

This step is consistent with the method of step S32, and the execution object is changed from the sample word list Ra to the full word list R. After performing these operations, the full grammatical relationship list G is obtained, and the elements in it are single graphs, each of which contains the dependency syntactic relationship and part of speech of a sentence, which will not be repeated here.

Step S42: extract the entries in the full grammatical relationship list G that are consistent with the dependency information in the grammatical relationship configuration table to obtain a matching entry list.

According to the requirements of the dependency information in the grammar configuration table, each graph in the full grammar relationship list G is searched. For example, the search term is "strike.*? evil forces", and the configuration table is "strike|VV, forces|NN,dobj". Assume that all the terms in the regular matching of the search term have been constructed as graphs. Then in the graphs converted from sentences 1), 2), and 3), you can use the networkx built-in method to query whether "strike" and "force" have a dependency relationship "dobj": gi.get_edge_data(ni,mi). Among them, gi represents the graphs of the three sentences, and ni and mi represent the ordinal numbers of the dominant and subordinate words, that is, the ordinal numbers of "strike" and "force" in the word list of the three search terms. If the result is empty, the method returns empty and outputs no matching result information. Otherwise, it returns the dictionary: {"dep":depi}. depi represents the dependency relationship between "strike" and "force" in the three sentences. In practice, we can know that the depi of the three sentences is "dobj". This shows that the dependency relationships in the three sentences are matched successfully.

Step S43: extract the entries in the matching entry list whose part-of-speech information is consistent with that in the grammatical relationship configuration table, and obtain a final matching list F.

Further, check the part of speech. Execute the networkx method to access the node attributes, that is, the part of speech: gi.nodes[ni] and gi.nodes[mi]. If the result output corresponds to {"pos":"VV"}{"pos":"NN"}, it means that the part of speech also meets the requirements of the configuration table. The dependency relationship and part of speech meet the description of the grammatical relationship configuration table. All three sentences will be output and displayed as the final matching list F.

In the case of unknown words in the grammatical relationship configuration table, the script will first confirm whether there are unknown words in the actual grammatical relationship. In this case, the edge attributes of the two nodes cannot be directly obtained, but the edges of the known words need to be traversed. For example: the grammatical relationship configuration table is "strike|*,[0]|VV,conj", with the known words as the dominant words, traverse the "out_edges" of the dominant word node: for src,dst,dep in g.out_edges(xi,data＝True). Among them, g is the graph instance constructed in the sentence where "strike" is located, and xi represents the sequence number of the word list of "strike" in the sentence. If there is a result output, it means that "strike" has "out_edges". The output of src is xi, and the output of dst is the sequence number of the word list of the unknown word in the sentence. dep is the dependency syntactic relationship between the two. Therefore, when there is an output dst part of speech that meets the requirements of "VV" and dep is "conj", it means that the match is successful. Otherwise, it fails. And dst is the unknown word [0]. If the grammatical relationship configuration table is "[0]|VV,力量|NN,dobj", the operation is the same as before, but the known word is used as a subordinate word, and the "in_edges" of the word is traversed: for src,dst,dep in g.in_edges(xi,data＝True).

In some preferred implementations, GPUs will be further used to accelerate the search process. The Python-based GPU data analysis framework "RAPIDS" provides a replacement product "cuGraph" for NetworkX on the GPU platform. The calculation can be accelerated without modifying too much code. In addition, parallel computing technology can also be used when searching. These optimizations can effectively ensure the search speed.

In general, this embodiment proposes a retrieval method with configurable grammatical relations. Using this method, matching can be completed more conveniently and fault-tolerantly than traditional methods. In terms of implementation, this embodiment introduces the concept of graph representation, which greatly improves the computational efficiency of the entire matching. This method can be widely used in public opinion monitoring and linguistic corpus research, and will greatly improve the accuracy compared to previous retrieval tools.

Embodiment 2

Please refer to Figure 11, which is a schematic diagram of the structure of a retrieval system with configurable grammatical relations disclosed in an embodiment of the present invention. As shown in Figure 11, the retrieval system with configurable grammatical relations may include:

An input module, the input module is used to input or read the target to be retrieved;

A text segmentation module, the text segmentation module is used to identify the search text in the target to be searched, and segment the search text to form a search term list;

A first matching module, the first matching module is used to perform regular matching on the search word list and the database to form a full word list;

A grammar configuration identification module, the grammar configuration identification module is used to identify whether there is a grammar relationship configuration table in the target to be searched;

A grammar configuration building module, wherein the grammar configuration module is used to build a grammar configuration table;

A second matching module, the second matching module is used to perform grammatical relationship matching with the full word list according to the grammatical relationship configuration table to form a final matching list;

The display module is used to output the final matching list and display it in order.

The configurable grammatical relationship retrieval system of the embodiment of the present invention completes the matching more conveniently and fault-tolerantly than the traditional method. The concept of graph representation is introduced to greatly improve the computational efficiency of the entire matching. This method can be widely used in public opinion monitoring and linguistic corpus research, and will greatly improve the accuracy compared with previous retrieval tools.

Embodiment 3

Please refer to FIG. 12, which is a schematic diagram of the structure of an electronic device disclosed in an embodiment of the present invention. The electronic device may be a computer, a server, etc. Of course, in certain circumstances, it may also be a smart device such as a mobile phone, a tablet computer, and a monitoring terminal, as well as an image acquisition device with processing functions. As shown in FIG. 12, the electronic device may include:

A memory 510 storing executable program codes;

a processor 520 coupled to the memory 510;

The processor 520 calls the executable program code stored in the memory 510 to execute part or all of the steps in the configurable grammatical relationship search method in the first embodiment.

The embodiment of the present invention discloses a computer-readable storage medium storing a computer program, wherein the computer program enables a computer to execute part or all of the steps in the retrieval method of configurable grammatical relations in the first embodiment.

The embodiment of the present invention further discloses a computer program product, wherein when the computer program product is run on a computer, the computer is enabled to execute part or all of the steps in the retrieval method for configurable grammatical relations in the first embodiment.

An embodiment of the present invention further discloses an application publishing platform, wherein the application publishing platform is used to publish a computer program product, wherein when the computer program product runs on a computer, the computer executes part or all of the steps in the retrieval method of configurable grammatical relations in embodiment one.

In various embodiments of the present invention, it should be understood that the size of the serial numbers of the processes does not necessarily mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present invention.

The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, i.e., they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

In addition, each functional unit in each embodiment of the present invention may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The integrated unit may be implemented in the form of hardware or in the form of software functional units.

If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-accessible memory. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product, which is stored in a memory and includes several requests for a computer device (which can be a personal computer, a server or a network device, etc., specifically a processor in a computer device) to perform some or all of the steps of the method described in each embodiment of the present invention.

In the embodiments provided by the present invention, it should be understood that "B corresponding to A" means that B is associated with A, and B can be determined according to A. However, it should also be understood that determining B according to A does not mean determining B only according to A, and B can also be determined according to A and/or other information.

A person of ordinary skill in the art can understand that some or all of the steps in the various methods of the embodiments can be completed by instructing related hardware through a program, and the program can be stored in a computer-readable storage medium, and the storage medium includes a read-only memory (ROM), a random access memory (RAM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), a one-time programmable read-only memory (OTPROM), an electronically erasable rewritable read-only memory (EEPROM), a compact disc (CD-ROM) or other optical disc storage, magnetic disk storage, magnetic tape storage, or any other computer-readable medium that can be used to carry or store data.

The above is a detailed introduction to the retrieval method, system, electronic device and storage medium for configurable grammatical relationships disclosed in the embodiments of the present invention. Specific examples are used herein to illustrate the principles and implementation methods of the present invention. The description of the above embodiments is only used to help understand the method of the present invention and its core idea. At the same time, for those skilled in the art, according to the ideas of the present invention, there will be changes in the specific implementation methods and application scopes. In summary, the content of this specification should not be understood as a limitation on the present invention.

Table 2: CTB part-of-speech tagging tags

Table 3: UD dependency labels

Claims (8)

1. A method for searching a configurable grammar relationship is characterized by comprising the following steps of,

obtaining a target to be searched, wherein the target to be searched comprises a search text, and dividing the search text by taking sentences as units; the single divided sentences form search terms, and all divided sentences form a search term list; the search term comprises a plurality of segmentation words;

based on the search term list and the database, carrying out regular matching, outputting full-quantity terms conforming to a regular matching rule, and forming a full-quantity term list;

Identifying whether a grammar relation configuration table exists in the target to be searched, if so, extracting the grammar relation configuration table, and if not, constructing the grammar relation configuration table; the grammar relation configuration table comprises grammar relations among the segmented words, wherein the grammar relations comprise dependency relations among the segmented words and parts of speech of the segmented words;

the grammar relation storage format is as follows: termsc|possrc, termdst|posdst, dep;

wherein termsc is a dominant word, posSRC is a part of speech of the dominant word, termsst is a subordinate word, posDST is a part of speech of the subordinate word, and dep is a dependency relationship between the dominant word and the subordinate word;

the dominant words are specific words, fuzzy words or unknown words; when the dominant word is a fuzzy vocabulary, termscc=; when the dominant word is an unknown word, termscc= [ n ], (n=0, 1, 2, … …, n);

the subordinate words are specific words, fuzzy words or unknown words; when the subordinate word is a fuzzy vocabulary, termdst=; termdst= [ n ], (n=0, 1, 2, … …, n) when the dominant word is an unknown word;

the parts of speech are specific parts of speech or fuzzy parts of speech; when the part of speech is a ambiguous part of speech, possrc=;

The dependency relationship is a specific dependency relationship or a fuzzy dependency relationship, and when the dependency relationship is a fuzzy dependency relationship, dep=;

and carrying out grammar relation matching on the grammar relation configuration table and the full quantity vocabulary entry list, outputting vocabulary entries conforming to grammar relation matching rules, and obtaining a final matching list.

2. The method for retrieving a configurable grammar relationship of claim 1,

the method for constructing the grammar relation configuration table in the step of identifying whether the grammar relation configuration table exists in the target to be searched, if yes, extracting the grammar relation configuration table, and if no, constructing the grammar relation configuration table comprises the steps of,

randomly sampling the full vocabulary entry list to obtain a sample vocabulary entry list;

obtaining a sample grammar relation list based on the sample vocabulary entry list and a grammar relation analysis method;

constructing grammar relation options based on the sample grammar relation list, and forming a grammar relation option list through statistics;

reading grammar relation demand option information, and generating a grammar relation configuration table based on the grammar relation demand option information and a grammar relation option list.

3. The method for retrieving a configurable grammar relationship of claim 1,

The step of obtaining a final matching list comprises the steps of performing grammar relation matching based on the grammar relation configuration list and the full quantity vocabulary entry list, outputting vocabulary entries conforming to grammar relation matching rules,

acquiring a full grammar relationship list based on the full vocabulary entry list and a grammar relationship analysis method;

extracting entries in the full grammar relationship list, which are consistent with dependency relationship information in the grammar relationship configuration list, to obtain a matched entry list;

and extracting the vocabulary entries in the matched vocabulary entry list, which are consistent with the word nature information in the grammar relation configuration list, to obtain a final matched list.

4. The method for retrieving a configurable grammar relationship according to claim 2, wherein the grammar relationship analysis method includes,

performing word segmentation on each term in the sample term list or the full-quantity term list to obtain word segmentation information of each term, and forming a first nested list;

performing dependency syntax analysis on each term based on the first nested list and the dependency syntax training model to obtain dependency relationship information among each word segment of each term, and forming a second nested list;

part-of-speech tagging is carried out on the word parts of the vocabulary entries of the first nesting list to obtain part-of-speech information of the word parts of the vocabulary entries, and a third nesting list is formed;

And constructing a sample grammar relation list or a full grammar relation list based on the first nested list, the second nested list and the third nested list.

5. The method of claim 4, wherein the step of constructing a sample grammar relationship list or a full grammar relationship list based on the first, second, and third nested lists comprises,

creating a bi-directional weighted graph based on the first nested list, the second nested list and the third nested list,

the starting point of the bidirectional weighted graph is termSRC|posSRC, wherein termSRC is a dominant word in the dependency relationship information, and posSRC is the part of speech of the dominant word;

the end point of the bidirectional weighted graph is termDST|posDST, wherein termDST is a subordinate word in the dependency relationship information, and posDST is the part of speech of the subordinate word;

the edge of the bidirectional weighted graph is dep, wherein the edge is the dependency relationship between the dominant word and the subordinate word in the dependency relationship information;

and converting the node information of the bidirectional weighted graph into a graph data structure, and constructing a sample grammar relation list or a full grammar relation list based on the graph data structure.

6. A search system capable of configuring grammar relation is characterized by comprising,

the input module is used for inputting or reading the target to be searched;

the text segmentation module is used for identifying a search text in the target to be searched, and segmenting the search text to form a search term list; the search term list comprises a plurality of search terms, and the search terms comprise a plurality of segmentation words;

the first matching module is used for carrying out regular matching on the search term list and a database to form a full term list;

the grammar configuration identification module is used for identifying whether a grammar relation configuration table exists in the target to be searched; the grammar relation configuration table comprises grammar relations among the segmented words, wherein the grammar relations comprise dependency relations among the segmented words and parts of speech of the segmented words; the grammar relation storage format is as follows: termsc|possrc, termdst|posdst, dep; wherein termsc is a dominant word, posSRC is a part of speech of the dominant word, termsst is a subordinate word, posDST is a part of speech of the subordinate word, and dep is a dependency relationship between the dominant word and the subordinate word; the dominant words are specific words, fuzzy words or unknown words; when the dominant word is a fuzzy vocabulary, termscc=; when the dominant word is an unknown word, termscc= [ n ], (n=0, 1, 2, … …, n); the subordinate words are specific words, fuzzy words or unknown words; when the subordinate word is a fuzzy vocabulary, termdst=; termdst= [ n ], (n=0, 1, 2, … …, n) when the dominant word is an unknown word; the parts of speech are specific parts of speech or fuzzy parts of speech; when the part of speech is a ambiguous part of speech, possrc=; the dependency relationship is a specific dependency relationship or a fuzzy dependency relationship, and when the dependency relationship is a fuzzy dependency relationship, dep=;

The grammar configuration construction module is used for constructing a grammar configuration table;

the second matching module is used for carrying out grammar relation matching with the full term list according to the grammar relation configuration table to form a final matching list;

and the display module is used for outputting the final matching list and carrying out sequencing display.

7. An electronic device, comprising: a memory storing executable program code; a processor coupled to the memory; the processor invokes the executable program code stored in the memory for performing the method of retrieving the configurable grammar relationship of any one of claims 1 to 5.

8. A computer-readable storage medium storing a computer program, wherein the computer program causes a computer to execute the retrieval method of the configurable grammar relationship of any one of claims 1 to 5.