RU2392660C2 - Method of searching for information in text array - Google Patents

Abstract

FIELD: physics, computer engineering.

SUBSTANCE: invention relates to processing natural language text and can be used to automate search of required documents in their large collection. Upon request, its content is processed on sentences. Sentences of the text array and the search request are compared pairwise and relevancy of each document of the text array to the request is calculated from the results based on sentences included in the document. The text array is indexed on separate sentences. Precise meaning of words in sentences is identified first and semantic links between them are established. The precise word meanings are then replace them by breaking down to elementary meanings which are stored for each meaning in thesaurus, after which a matrix is made for each sentence, which contains the link between all pairs of objects included in the sentence. An inverted index is then made, where for each object included in the text array, the documents, sentences and the number of times it is met is indicated.

EFFECT: invention enables comparison of phrases according to sense.

2 cl

Description

The invention relates to the field of information technology, namely to the processing of natural language texts, and can be used to automate the search for necessary documents in a large collection of them.

Currently, for searching large collections of documents, they use either keywords, taking into account the form of words only separately and the distance between words, or use the results of parsing and primary semantic analysis and take into account the semantic role of the word, but do not take into account the polysemy of words and the possibility of expressing it same meaning in other words.

There is a method of synthesis of a self-learning system for extracting knowledge from text documents for search engines by providing a self-learning mechanism in the form of a stochastically indexed artificial intelligence system. Morphological and syntactic analysis is carried out, as well as stochastic indexing of text documents according to a given scheme for the formation of semantic analysis knowledge bases. The user’s request is converted in a stochastically indexed form to many new queries that are equivalent to the original query, and stochastically indexed fragments of text documents are selected in all phrases of the converted query, from which they form a stochastically indexed systematic structure, form a short response of the system based on this structure and check the relevance of the short the response of the system to the request by comparing them (RU 2273879, IPC 7 G06F 17/30, publ. 10.04.2006).

A disadvantage of the known solution is the use of natural language words as indivisible elements, while words are ambiguous and the same phrase can be expressed in different words, which makes it impossible to take into account equivalence in meaning between different expressions.

The technical result consists in providing the possibility of a detailed semantic analysis of a natural language text, including the processing of meanings included in the text of words due to the possibility of describing the meanings of words as constituent elements on which it is possible to perform actions and conversions, which allows comparison of phrases within the meaning.

The indicated technical result is achieved by the fact that the method of searching for information in an array of texts consists in preliminarily indexing the array of texts according to the sentences included in it. Upon receipt of a request, its contents are processed by offers, a pair of text array sentences and a search query is matched, the results of which calculate the relevance of each text array document to the query based on the sentences included in the documents, then, in order of decreasing relevance, the text array documents are presented to the user as a search result. Indexing an array of text occurs on individual sentences. In sentences, they first recognize the exact meanings of words and establish semantic relationships between them, then the exact meanings of the words are replaced by decomposing them into elementary meanings, which are stored for each value in the thesaurus, after which, for each sentence, a matrix is constructed containing links between all pairs of objects included in sentence, then make up an inverted index, where for each object included in the text array, it is indicated in which documents and in which sentences how many times it occurs. Processing a search query upon its receipt involves determining the exact meanings of all the words included in the query’s sentences, then the exact word meanings are replaced by decomposing them into elementary values, which are stored for each value in the thesaurus, after which a matrix containing links between all by pairs of objects included in the proposal, then they search on the thesaurus of objects having a correlation with the objects included in the request, after which all objects included in the request and correl ruyuschie object is in the inverted index and on the basis of its data from an array of texts selected proposals, which then perform the pairwise comparison with the request for proposals. When pairwise matching sentences of an array of texts and query sentences, a matrix is preliminarily constructed that reflects the correspondence between the objects included in the text and the objects included in the request and correlated with the request objects. Then, for the objects connected in the obtained matrix, the values of the matrix cells corresponding to these objects are constructed for the sentences of the text array and for the sentences of the query, on the basis of which the relevance of the query sentences to the sentences of the text array is calculated. Then, by calculating the arithmetic mean, the relevance of each sentence of the array of text to the entire query is determined. For each document an array of text, based on the relevancy of its sentences, the relevance of the entire document to the query is calculated using statistical indicators.

The following are definitions of terms used in the description.

A document is a unit of an array of texts, one or more of which the user needs to find.

Denotat (intensional) - the exact unambiguous meaning of a word in context.

An object is an object or phenomenon of the surrounding world that has a correspondence in the language as an independent concrete meaning, i.e. variety of denotate.

A predicate is a denotate, characterized by the ability to be supplemented by other denotates to obtain a value, which, unlike an object, does not have a specific value in itself.

Arguments (actants) are denotations that supplement predicates to obtain specific values.

Clause is a simple sentence as part of a complex.

Referential relationships are relationships that indicate that different expressions mean the same thing or phenomenon.

Semantic connections are connections between denotations, with the help of which they form a single meaning.

Syntactic relations - relations between words in a sentence, defined by the grammar of the language.

The semantic class is a group of denotations that have the same ability to establish semantic relationships.

Indexing - preprocessing an array of texts, which then allows for faster searches.

Relevance - a value in the range from 0 to 1, characterizing the degree of correspondence of a fragment of text to a request.

An inverted index is a data structure that lists some elements of a collection of documents, and for each of them lists all the places where they occur.

Structure - a composite data type, which includes a set of elements of a certain type with specific names.

Capitalized in italics are the names of data types, structures, their elements and their possible values.

The way of representing the meanings of words is based on the following. For analysis, not just separate words, but specific intensions (denotations) of lexemes should be presented taking into account anaphoric expressions, and each lexeme should be defined at the same time, as a specific extensional, or as an attributive or autonomous use. In the case of a diffuse value, or the use of a token in several values at once, it is necessary to conduct an analysis separately for each intensional.

Component values are divided into objects and predicates. Objects and predicates are divided into elementary (basic) and composite. There are a limited number of basic predicates; objects and compound predicates are unlimited. Predicates have arguments (actants), objects do not. Compound predicates can include objects. Objects are divided into semantic classes, moreover, one object can belong to several semantic classes having hierarchies, moreover, there are several different hierarchies, starting from different semantic classes. Objects have hierarchies similar to semantic classes. Other objects and predicates can be arguments of a predicate, moreover, for a particular argument a type must be specified - an object or a predicate, and if an object, then its semantic class is specified, while objects of all children in the hierarchy of semantic classes are allowed. Predicates can be nested as arguments to other predicates an unlimited number of times. The arguments of a compound predicate are divided into primary and secondary. The number of main arguments is equal to the number of unspecified arguments of all nested predicates. Additional arguments are objects in the predicate that can be replaced by children in the hierarchy. A compound predicate with all the specified main arguments can be considered as an object, or as a predicate using only additional arguments. The universal and empty objects can also serve as object arguments, and references to the arguments of the considered predicate can also serve as object arguments of a nested predicate. Some basic predicates have numerical arguments. Component values can be combined not only by predicates, but also by conjunctions and disjunctions, such a union is written in conjunctive normal form, moreover, conjunctive terms are divided into differential and characteristic ones. When performing a conjunction or writing a single predicate, it must be indicated whether it is a differential or characteristic conjunctive member. Disjunction is possible either between two differential or between two characteristic terms. Any object or predicate can have a negation, moreover, the negation can be either logical (complete) or linguistic. Linguistic negation negates only differential components (for compound predicates, disjunctive terms in conjunctive normal form). When transforming objects with predicates indicating all arguments, and when navigating through the hierarchy of objects, mutual conversion of differential and characteristic components is possible. Objects have attributes joined by predicates. Signs are also divided into differential and characteristic. A set of features determines the belonging of an object to a semantic class.

At the same time, objects mean everything that an object or phenomenon means, regardless of the fact that the semantic role can correspond to the subject. Thus, they include nouns denoting an object or phenomenon, pronouns that replace nouns, in some cases numerals, if they indicate simply the number, not the number of objects. The main difference between the object's token is the presence of a specific meaning in itself. Specific is a meaning that can be imagined or depicted in any form (not necessarily in visual, there may be emotional, emotional, etc. meanings) without resorting to metaphor or metonymy. The same denotations that require the addition of other denotations to obtain a specific value relate to predicates. These include nouns that do not have a specific meaning and serve to refine the meaning of other nouns (for example, the word part), verbs, adjectives and participles, adverbs, ordinal numbers and numerals that indicate the number of objects. Special parts of speech, like participles, are not included in this classification, for their interpretation a grammatical restructuring of the phrase using other parts of speech is required, which is always possible for such parts of speech.

A characteristic feature of a basic predicate is an absolutely abstract meaning, applicable to any topic, i.e., for example, while the formation of a relative and qualitative adjective from a noun is different basic predicates, a specific type of relative adjective (time, material, etc. ) can no longer be specified by the basic predicate itself, such characteristics can only be arguments (objects). In this case, the hierarchies of objects reflect the fact that an object downstream in the hierarchy is a type of object that is higher in the hierarchy, while it inherits the attributes of the latter, but with the possible transition of differential attributes into characteristic ones, and may also have its own characteristics.

Basic predicates with numeric arguments are used when describing non-discrete objects, like numbers or colors themselves.

If denotations are recognized, as well as semantic and referential connections, then it is possible to build links in clauses. To do this, object tokens become arguments of predicate tokens, subject to semantic classes, i.e. so that the objects have signs corresponding to predicates. Relationships are considered constructed if all arguments of all predicates are specified. Applying predicates to objects in a clause creates situational objects, moreover, they can use fuzzy disjunction, conjunction and negations, transformations are performed according to the rules of fuzzy logic.

As an example of the implementation of the proposed method, it is necessary to consider a search system that allows you to find equivalent in meaning expressions. This requires the use of a semantic analyzer that gives the exact meaning (denotation) of all words in a sentence and the structure of semantic relations between them. Moreover, within clauses, semantic relations should be described using basic predicates adopted in the system. The search task itself is divided into two parts: preliminary processing (indexing) of the text and the search process itself when requests are received. Indexing is required once for a given array of texts, after which an unlimited number of search queries can be processed. Indexing consists of the following:

1. Semantic analysis of the entire text. The result is lexeme denotation dictionary definitions and semantic relationships between them.

2. Substitution instead of denotations of their decomposition into elementary values according to the thesaurus. If the lexeme is not found in the dictionary by the semantic analyzer, or no semantic class is suitable for the lexeme in the context, then the lexeme is considered an indivisible object with possible additions of basic predicates to ensure a syntactic role, and then it is equivalent to all tokens identical in initial form with an unknown semantic class . The result is a single basic predicate with nested objects and other basic predicates for each clause.

3. Selecting a list of objects for each sentence and constructing a matrix, where both the set of rows and the set of columns are the objects present in the proposal. The result is the indicated matrix for each sentence, and the cells contain arrays of structures of the following form

Object Link {

Predicate sublevels [array] predicates of the first object;

Predicate sublevels [array] predicates of the second object;

Predicate sublevels [array] general predicates;

Type of association of objects;

Negation <negation of the first object>, <negation of the second object>;

argument numbers of the current predicate;

predicate name;

}

where Predicate sublevels is also a structure of the following form: Predicate sublevels {

predicate name;

argument number

negation on the predicate;

}

The predicates of the first object, the Predicates of the second object, and General predicates make up the arrays of the structures of the Predicate sublevels. The first two are nested predicates, the arguments of which are the first and second objects, respectively, for some (current) predicate, the nested sub-arguments of which are both objects. General predicates - a list of all nested predicates with respect to the main clause predicate, with respect to which the (current) predicate combining two objects is the next in terms of nesting level. Thus, in the first three elements of the structure information is stored on the names and numbers of the arguments of the predicates that determine the path by the nested predicates to both objects. The type of union of objects takes one of four values:

- joint arguments of the current predicate;

- conjunction;

- disjunction;

- the object itself.

The Joint arguments type of the current predicate means that both objects are combined according to the levels of attachment on some basic (current) predicate, of which they are different arguments. This means that either the objects themselves are the arguments of one basic predicate, or one or both of these arguments is some basic predicate, in one of the further levels of embedding of which the corresponding object appears. Information about predicates at nesting levels is contained in the arrays of predicate sublevels described above. The value The object itself means that we are talking about considering a diagonal cell, the role of which is simply to establish the presence of the object, and not its connections with other objects, for which the predicates of the first object, the predicates of the second object do not make sense, but General predicates retain their meaning. The values Disjunction and Conjunction mean, respectively, the union of objects by disjunction and conjunction.

The negation of the first object and the Negation of the second object contain information on whether each of the objects has negation on it. As already mentioned, negation is of two types, from which it follows that Negation can have three meanings: Negation is absent, Linguistic negation, Logical negation. The Argument Numbers element of the current predicate contains two integers and is used only if the Combination Type of the objects is set to Joint arguments of the current predicate. These two numbers indicate which arguments of the current predicate contain both objects, respectively. The predicate name indicates the type of the current predicate.

The meaning of the array of structures is that each structure reflects only one connection between objects, while in a sentence the same pairs of objects can be connected several times. Each element of the array corresponds to one connection, respectively, if the objects are connected once, then one element is used.

Thus, the matrix describes the relationships between all pairs of objects included in the proposal.

4. The construction of the inverted index on the objects contained in the text, where for each object it is indicated, firstly, whether it appears and how many times in the document, and secondly, whether it appears and how many times in which sentence. As a result, the matrices described in Section 3 for each sentence and the inverted index are saved.

After indexing, a search is possible. It consists of the following:

1. Semantic analysis of the request. The result is a denotation dictionary of query tokens established in the dictionary and semantic relations between them. Similar to item 1 for indexing.

2. Substitution instead of denotations of the request for their decomposition into elementary values according to the thesaurus. The result is a single basic predicate with nested objects and other basic predicates for each query clause. Similar to item 2 for indexing.

3. Construction of a matrix for each sentence of the request described in clause 3 for indexing.

4. Search by the thesaurus of objects that have connections with query objects, i.e. capable of mutual transformation using predicates and other objects. For this, a special list is used that contains not the conversion paths between objects, but the correlation value between the objects numerically expressed in the range from 0 to 1. Associated objects are those having a correlation level above a predetermined threshold value determined by the selected mode of correlation of speed and search accuracy. Those. the higher the speed, the higher the threshold value and the less related objects are selected. With the greatest accuracy and the lowest speed, you can use all objects for which the correlation value is greater than zero. The result is a list of objects in the proposal itself and related objects.

5. All objects of the proposal and related objects are searched in the inverted index and, thus, the necessary documents and proposals are selected. In the high accuracy and low speed mode, all documents are selected where at least one object from the request or a related object occurs at least once. In higher speed modes, only documents and sentences are selected where a sufficient number of objects are found enough times, which is determined by any predetermined formula (for example, the arithmetic average or in any other way averaged value of the frequency of objects in a document or sentence related to the frequency of those the same objects in the entire array of texts, or the specified value, weightedly summed up with the proportion of request objects included in the document or proposal, or use separately threshold values for these two criteria). The result is the selected documents and proposals in them.

6. For all objects of all selected text sentences, their correspondence with the request objects is established. This correspondence can be of two types: either objects coincide, or by using basic predicates and other objects, one object can be transformed into another. To receive the objects to be converted, the objects associated with the request objects and the objects encountered in the proposal must match. The result is a correspondence matrix between objects. The cells contain the following structures:

Matching Objects {

The presence of communication;

Coincidence;

Predicate sublevels [array] transforming predicates;

}

The first element of the Boolean type indicates the existence of a connection between objects, if its value is true, then the second element of the Boolean type indicates the fact of coincidence, and if its value is not true, then the third element, described in Section 3, contains information about conversion between objects. In this case, if the objects are connected by a hierarchy, the relationship is built in the form of one element of the array Transforming predicates, which reflects that they are connected by a hierarchy. The meaning of the Match element is that, with a true value, it allows you to establish the correspondence of the column and row numbers of the query sentence matrices and text sentences, since the row and column numbers for the same objects in both matrices may well differ.

7. For the entire Cartesian product of the selected sentences of the array of texts and query sentences, the display of the matrices described in Section 3 is considered in pairs. For this, the mappings between objects described in Section 6 are used. For a given pair of sentences for each structure in each cell in the object matrix from the query sentence and for each pair of objects associated with them, the corresponding pair of objects is searched in the text array sentence. Between pairs of the same objects, the relevance of the mapping of the structures described in clause 3 is calculated. In this case, all the elements of the Object Link structure are used. If the array of the corresponding cell in both the query sentence and the text sentence has more than one element, then the relevances for the entire Cartesian product of those and other structures are calculated. First, the predicate name value is compared. If it matches, then the relevance of this element is considered equal to 1, i.e. complete. If it does not match, then the correlation table between the basic predicates is used, and a certain value from 0 to 1 is taken from this table, which reflects the correlation of the corresponding predicates. Then, the argument numbers of the current predicate are compared. Their relevance is taken equal to unity in case of coincidence and less in case of mismatch, just as in the table reflecting the correlations between the arguments of the predicates. Then the Negation of the first object and the Negation of the second object are compared. Likewise, full relevance corresponds to coincidence - a partial correlation table between negatives. Similarly, the type of combining objects either full relevance when matching, or correlation.

When analyzing the mapping of all three arrays of the Predicate sublevels, the code distance between them is determined first. Those. for each of them the smallest number of replacements, modifications, exceptions and additions of predicates is required, which is necessary for converting one into another. By modification is meant a change in the number of the argument or negation that follows. To calculate the code distance in high accuracy and low speed mode, algorithms should be used that provide complete reliability, and in modes starting from a certain speed, it is possible to use algorithms that give a reliable code distance with only a certain probability (the so-called randomized algorithms). Full correspondence means the relevance of an element is 1, predicate substitutions reduce relevance in accordance with the correlation between the corresponding predicates, modifications in accordance with the correlations between the arguments, exclusion and addition - by some value specified for a particular predicate. Moreover, a special situation is represented by modifications of denial. If negation occurs many times, then relevance can increase, i.e., firstly, negatives are taken into account directly on objects, and secondly, on each of the embedded predicates. If in a matrix cell for a sentence of a text and for a sentence of a query of negation there are the same even or equally odd number of times, then the relevance will be higher than if in one case it is even and in another - odd. But this takes into account how many and what predicates denials share, i.e. the farther across the code distance matched by the code distance the negatives corresponding to the text and request are, the lower the relevance, even if the number of negatives is the same. This reduction in relevance is determined by the table of values specified for each predicate. If there are more negatives in the text or in the query, then if there are an even number of negated negatives, then the number of separating predicates in accordance with the values set for them is taken into account when calculating the relevances, and if the odds are odd, the assigned low relevance is assigned, which is established in the presence of unpaired unmatched negation. In this case, if not the object included in the query is used, but the object associated with it, then for the predicates of the corresponding object, the Transforming predicates are added to the existing array of Predicates of the first object or Predicates of the first object, and the relevance is calculated taking into account their presence in the chain, but in a chain formed by merging arrays, it is possible to exclude the following consecutive predicates that perform opposite transformations (for example, substantivization of a verb and then verbalization of a noun). The final relevance for the corresponding elements can be defined as the product of all quantities less than 1, multiplied by the relevance of the correspondence of negatives, or by any other formula. The obtained relevance for all structural elements according to some predetermined formula (for example, partially weighted summation and partially multiplication) are recalculated into a single relevance of the mapping of the corresponding cells of the two matrices. Moreover, in the indicated formula, for example, the Predicates of the first object with General predicates and the Predicates of the second object with General predicates can be multiplied according to the relevance obtained for them, and then these two quantities can be summed with weights proportional to the inverse frequency of each of the objects in the text array. Similarly, for example, the relevance of negatives for two objects can be weighted summarized in the same way, and then be a common factor for the relevance of the whole pair of structures.

8. For a given pair of sentences, all the relevance factors calculated for individual cell structures are reduced to a common one. The basis for this is the objects included in the request. Those. for each relationship of each pair of query objects, their relevance to the corresponding text sentence is calculated. If the relationship of a pair of query objects occurs in a text sentence, then the relevance value for them is taken. If related objects are found for query objects in the text, then the probability calculated for them is also used, taking into account corrections for the predicates necessary for the transformation, as described in clause 7. If both options are possible, moreover, several times, then some final one is established, calculated according to some given formula (for example, maximum relevance or unit minus the product of differences of unity and relevance) relevance for a pair of query objects, which can be found in the sentence using the method described in clause 7 text. If a pair of query objects is not present in the text in any form, then the relevance for this cell is considered equal to zero. If both in the query and in the sentence of the text the structure occurs several times, i.e. Since the array of structures for a cell contains more than one element, those structures in the text sentence are used for comparison, which ensure the highest relevance for each structure from the query sentence, i.e. the comparison is based on the principle of greatest relevance. If the cell array in the query is longer, then for the remaining structures, which, by the principle of the highest relevance, did not match any structures in the text sentence, the relevance is considered equal to zero. After that, the general relevance of the query proposal matrix is considered, for which the relevances obtained in the described manner for the cell structures of the query proposal matrix are weighted summarized. The weight of each such cell is determined by the value of the General predicates element, i.e. position of a pair of objects with respect to the main clause predicate. The weight is determined by the types and numbers of arguments of nested predicates, based on predefined values, moreover, the more weight, the less elements in the General predicates array, i.e. the closer the cell is to the main clause predicate. In this case, of course, if the structure in the cell occurs several times, then the weight for these structures will be the same. There may be additional factors affecting the weight of the cell, for example, the weight of the diagonal cells, indicating only the presence of objects, can be adjusted by a certain amount. The result is the relevance values for each pair of query sentences and selected sentences in the text array.

9. For each selected sentence of the array of texts, the arithmetic average of all sentences of the query is calculated, which is considered the relevance of each selected sentence of the array of texts to the query.

10. For each of the selected documents, the relevance of each of its proposals to the request is taken. If the proposal of the document is not included in those selected in clause 5, then its relevance to the request is considered equal to zero. Based on these values for each document, the following indicators are calculated:

- the arithmetic mean of the relevance of the proposals;

- the highest relevance value among all proposals;

- the value of the relevance range into which the greatest number of sentences falls;

- the value of the relevance threshold above which there is a given number of sentences.

All these indicators are weighted summarized, from which the relevance of the document to the query is calculated. The weight of each of the indicators is set by the user settings. The user also sets the size of the range for the third indicator and the number of offers in relation to the number of document offers for the fourth indicator. In this case, the size of the range is either fixed or automatically adjusted depending on the variance of the relevance of the proposals, and in any case, the average value of the range is taken as the value of this parameter. As a result, the relevance of each document to the query is calculated, and in order of decreasing relevance, they are issued in the search results. In addition, some of the most relevant sentences for each document may appear in the search results.

Relative to that described in paragraph 1 for searching and indexing semantic analysis, one of the possible methods for recognizing denotations will be given below. The method described below will proceed from the parsing that has already been performed, the results of which are presented in the form of tokens and syntactic links, and grammes are also defined for each token.

According to the proposed denotation recognition method, a set of sets of semantic classes is defined for each token. Semantic classes are defined in such a way that the combination of a lexeme and a semantic class uniquely defines a denotation. At the same time, several semantic classes can be assigned to one denotation, if the latter have common denotations.

In the hierarchies of semantic classes, the one higher in the hierarchy includes all denotations of all those below it in the hierarchy of semantic classes. There can be an unlimited number of hierarchies, the same semantic class can belong to several hierarchies, each hierarchy has one upper semantic class, including denotations of all semantic classes included in the hierarchy. If several semantic classes of one set for one lexeme are in one hierarchy as a variety of another, then in further consideration of the set of semantic classes of a certain denotation, only the lower semantic class in the hierarchy is used, several semantic classes are indicated only if they are in parallel hierarchies.

, где c1 - семантический класс, входящий в определенное множество набора для первой лексемы, с2 - семантический класс, входящий в определенное множество набора для второй лексемы, syn - тип синтаксической связи, pr - служебное слово в сочетании с граммемами управляемой лексемы, sem - тип семантической связи, с - семантический класс полученной объединенной группы. Согласно предлагаемому алгоритму, как и для синтаксического анализа, возможны нисходящий и восходящий анализ, поиск в ширину и в глубину. При нисходящим анализе для некоторого <с,sem> подбирается лексема с семантическим классом c1, имеющая синтаксическую связь syn и управление pr с некоторым непосредственным составляющим, в результате чего отделяется семантический класс с2, который раскладывается дальше аналогично семантическому классу с. При восходящем анализе выбираются лексемы с семантическими классами c1 и с2, с синтаксической связью syn и управлением pr, в результате чего образуется семантический класс с, который аналогично c1 и с2 объединяется со следующими лексемами. Алгоритмами, применяемыми при синтаксическом анализе, чередуя нисходящий и восходящий анализ, поиск в ширину и в глубину, необходимо достичь объединения семантическими связями sem всех лексем предложения, изначально связанных синтаксическими связями.According to the proposed algorithm, semantic analysis uses the method of operating with context-free grammars and is based on a set of rules of the form

, where c1 is the semantic class that is included in the set of the set for the first token, c2 is the semantic class that is included in the set of the set for the second token, syn is the type of syntax, pr is the service word in combination with the grammar of the managed token, sem is the type semantic connection, with - the semantic class of the resulting combined group. According to the proposed algorithm, as for parsing, downward and upward analysis, wide and deep search are possible. In a top-down analysis, for some <c, sem>, a lexeme is selected with the semantic class c1, which has syntactic connection syn and control pr with some immediate components, as a result of which the semantic class c2 is separated, which decomposes further similarly to the semantic class c. In the upward analysis, tokens are selected with the semantic classes c1 and c2, with syn syntax and control pr, resulting in the formation of a semantic class c, which, like c1 and c2, is combined with the following tokens. Algorithms used in parsing, alternating downward and upward analysis, search in width and in depth, it is necessary to achieve the union of semantic links sem all tokens of the sentence, originally connected by syntactic links.

и

. Если правила не позволяют построить семантические связи в предложении, то делаются попытки приписать и известным словам, крайним случаем может быть, что всем словам приписан неизвестный семантический класс u.Moreover, if unknown words are found, then an unknown semantic class u is assigned to them, while it is possible

and

. If the rules do not allow building semantic relations in a sentence, then attempts are made to ascribe well-known words, as an extreme case, it may be that an unknown semantic class u is assigned to all words.

Compared with known solutions, the proposed method allows you to find sentences that are equivalent in meaning to the sentences of the request, even if the same is expressed in other words. This gives advantages in comparison with the used known search algorithms, which find only sentences with the same words as in the query, or with very narrowly described synonyms of them.

Claims (2)

1. A method of searching for information in an array of texts, which consists in preliminarily indexing an array of texts by the sentences included in it with recognition of the exact meaning of the words included in the query sentences by replacing them by decomposing them into elementary values that are stored for each value in the thesaurus then, for each sentence of the query, a matrix is constructed containing links between all pairs of objects included in the sentence, then a search is performed on the thesaurus of objects correlating with objects in going into the query, after which all the objects included in the query and correlating objects are found in the inverted index and, based on its data, the sentences are selected from the array of texts, for which they then perform pairwise comparisons with the sentences of the query, for which a matrix is constructed that reflects the correspondence between the objects included in the text and the objects included in the request, and the correlating objects of the request, then for the objects connected in the resulting matrix, the values of the matrices corresponding to these objects are matched IC constructed for an array of texts and proposals to the request for proposals, which is calculated on the basis of the relevance of the request for proposals of the proposed text of the array using statistical indicators, followed in decreasing order of relevance of texts array documents presented to the user as a search result. 2. The method according to claim 1, characterized in that the array of text is indexed according to individual sentences, while the sentences first recognize the exact meaning of the words and establish semantic relationships between them, then the exact meaning of the words replaces them by decomposing them into elementary values that are stored for of each value in the thesaurus, after which a matrix is constructed for each sentence containing relationships between all pairs of objects included in the sentence, then an inverted index is made, where for each object, the input on the array of text, stated in public documents and public offerings number of times it occurs.