RU2829065C1 - System and method for training machine learning models for ranking search results - Google Patents

Abstract

FIELD: information technology.

SUBSTANCE: invention relates to a method and a server for ranking digital documents on a digital platform. Method comprises receiving a search query sent to a digital platform, generating a first vector representation characterizing the search query, finding a plurality of digital candidate documents satisfying the search query, obtaining a second vector representation for each digital candidate document from the plurality of digital candidate documents, finding, in each candidate document from the plurality of digital candidate documents, at least one candidate phrase lexically associated with the search query, generating a third vector representation characterizing the at least one candidate phrase, determining, based on the first, second and third vector representations for each candidate document from a plurality of digital candidate documents, a ranking parameter value and ranking the plurality of digital candidate documents depending on the associated ranking parameter values.

EFFECT: high relevance of ranked digital documents to a search query.

20 cl, 9 dwg

Description

Field of technology to which the invention relates

[001] The present technology relates to machine learning methods, in particular to methods and systems for training and using machine learning models for the purpose of ranking search results.

State of the art

[002] Web search is currently a key technology, processing billions of user queries daily. Modern web search engines typically rank search results based on their relevance to the search query. Machine learning algorithms (MLA) trained to evaluate various relevance metrics based on a variety of features are often used to determine the relevance of search results to a query. This determination of relevance can be viewed, at least in part, as a language understanding task, since the relevance of a document to a search query is partly a matter of semantic understanding of both the query and the search results, even in cases where the query and results do not share common words or where the results are images, music, or other non-textual content.

[003] Recent advances in neural natural language processing include the use of Transformer-based machine learning models, as described in Vaswani et al., “Attention Is All You Need,” Advances in neural information processing systems, pages 5998–6008, 2017. A Transformer is a deep learning model (i.e., an artificial neural network or other machine learning model with multiple layers) that uses an “attention” mechanism to give more weight to some parts of the input data than others. In natural language processing, such an attention mechanism provides context for words in the input data, since the same word can have different meanings depending on the context. Transformer-based machine learning models are thus able to take into account the semantics (i.e., meaning) of words in a particular context, such as keywords in a search query. Transformers are also able to process many words or tokens of natural language in parallel, enabling parallel learning.

[004] Transformers have also formed the basis for other advances in natural language processing, in particular pre-trained systems that can be trained on a large dataset in advance and then optimized for use on specific tasks. Examples of such systems include Bidirectional Encoder Representations from Transformers (BERT), described in Devlin et al., “BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding,” Proceedings of NAACL-HLT 2019, pages 4171–4186, 2019, and the Generative Pre-trained Transformer (GPT), described in Radford et al., “Improving Language Understanding by Generative Pre-Training,” 2018.

[005] In general, to perform search ranking tasks, transformers may be trained to determine the relevance parameters of search results returned by a digital platform (e.g., a search engine) to a particular user. For example, such relevance parameters may be represented by values of the probability of a user interacting with the search results (e.g., clicking a mouse button). In particular, in response to a user submitting a search query, the digital platform may find a set of digital documents (e.g., web documents) that satisfy this search query. In addition, both the search query and the set of digital documents may be fed to a transformer-based machine learning model trained using specially organized training data to perform ranking.

[006] Although Transformer-based machine learning models demonstrate high performance in selecting relevant search results based on the semantic relationships between them and the search query, capturing the lexical relationships between the search query and digital documents can be challenging for such models. In other words, because the attention mechanism assigns more weight to some search query keywords than others, Transformer-based machine learning models may ignore (or assign low ranks to) digital documents that contain words that exactly match the search query keywords.

[007] For example, a user may submit a search query with the text “Macarons recipe Cedric Grolet” to a digital platform, and the digital platform may be able to find a plurality of digital documents that satisfy the search query. In addition, the transformer-based machine learning model may rank the digital documents in such a way that the first (best) search results on the search engine results page (SERP) of the digital platform include digital documents (e.g., text documents, images, videos, etc.) that contain information related only to recipes for traditional French desserts “macarons” and/or information related only to the famous French pastry chef Cedric Grolet. However, the transformer-based machine learning model may assign a relatively low priority to digital documents that contain information about recipes for macarons specifically from Cedric Grolet, as a result of which such digital documents may not appear on the search results page. As a result, interaction with the digital platform may leave a negative impression on the user.

[008] Thus, there is a need in this area to take into account both semantic and lexical aspects of the relationships between a search query and digital documents when ranking digital documents. A number of methods have been proposed in known technical solutions to solve this technical problem.

[009] Gao et al., “COIL: Revisit Exact Lexical Match in Information Retrieval with Contextualized Inverted List,” Carnegie Mellon University, 2021, present an architecture for contextualized exact matching that establishes lexical-semantic matches based on a contextualized inverted list (COIL). The COIL list score is formed based on contextualized representations of partially matching tokens in queries and documents. The architecture stores contextualized representations of tokens in inverted lists, combining the efficiency of exact matching with the rich representation capabilities of deep language models.

[010] US11475067B2, "Systems, apparatuses, and methods to generate synthetic queries from customer data for training of document querying machine learning models," issued October 18, 2022, to Amazon Technologies Inc., discloses methods for generating synthetic queries from customer data for the purpose of training machine learning models to query documents as a service. Such a service may receive one or more documents from a user, generate a plurality of question and answer pairs from the one or more user documents using a machine learning model trained to predict the question from the answer, and store the plurality of question and answer pairs generated from the one or more user documents. The question and answer pairs may be used to train another machine learning model, such as a document ranking model, a snippet ranking model, a question-answer model, or a Frequently Asked Question (FAQ) model.

Disclosure of invention

[011] The developers of this technology have found that the efficiency of taking into account both semantic and lexical relationships between search queries and ranked digital documents is increased if the input data of the transformer-based machine learning model, in addition to the search query and the digital documents themselves being ranked, includes a number of phrases from candidate digital documents that have been identified as lexically related to the search query.

[012] In particular, various aspects and embodiments of the present technology relate to methods and systems that provide for (a) receiving a search query, (b) finding in a search index of a digital platform a plurality of candidate digital documents that match the search query and are semantically related thereto, and (c) finding in the plurality of candidate digital documents a plurality of candidate phrases that are lexically related to the search query, that is, that contain at least one word that either completely matches one of the keywords of the search query or is a grammatical derivative thereof. Furthermore, in accordance with some non-limiting embodiments of the present technology, the search query and the phrases thus found that are lexically related thereto are fed to a transformer-based machine learning model trained to generate vector representations of input phrases, in order to generate a first vector representation of the search query and a second vector representation of the plurality of candidate phrases.

[013] Furthermore, the methods and systems described herein relate to feeding the vector representations generated in this manner, along with the vector representations of the documents and the search query, to a machine learning model for ranking that is specially pre-trained to determine the priorities of digital documents based on such vector representations. Accordingly, when ranking digital documents for their presentation on the search results page of the digital platform, these methods and systems make it possible to take into account not only the semantic relationships between the search query and the content of the digital documents that satisfy it, but also the vocabulary of the search keywords. Thus, in addition to digital documents that are semantically (i.e., in meaning) relevant to the search query, the search results page of the digital platform may also present digital documents that are lexically relevant to the search query, i.e., containing words that completely match one or more keywords of the search query or are derivatives of them. This helps to improve the impressions of users interacting with the digital platform.

[014] In particular, in accordance with the first broad aspect of the present technology, a computer method for ranking digital documents on a digital platform is implemented. The method involves receiving a search query sent by a user to a digital platform, forming a first vector representation characterizing the search query, finding a plurality of candidate digital documents satisfying the search query in a search index of the digital platform, obtaining a second vector representation formed before the search query is received for each candidate digital document from the plurality of candidate digital documents, finding for each candidate document from the plurality of candidate digital documents at least one candidate phrase lexically associated with at least one keyword of the search query, forming a third vector representation characterizing at least one candidate phrase, determining, based on the first, second and third vector representations, for each candidate document from the plurality of candidate digital documents a ranking parameter value characterizing the relevance of this candidate document from the plurality of candidate digital documents to the search query, and ranking the plurality of candidate digital documents based on the ranking parameter values associated with them.

[015] In some embodiments of the method, finding a plurality of candidate digital documents involves using a ranking function.

[016] In some embodiments of the method, the ranking function is the Okapi BM25 ranking function.

[017] In some embodiments of the method, finding at least one candidate phrase involves forming a vector representation of the phrase for each phrase of the candidate document from a plurality of candidate digital documents, determining in the space of vector representations a distance value between the first vector representation characterizing the search query and the vector representation of the phrase, ranking phrases from the plurality of candidate digital documents depending on the distance values associated with the vector representations of the phrases in order to form a ranked list of phrases for the search query, and selecting a specified number of priority phrases from the ranked list of phrases.

[018] In some embodiments of the method, the formation of a vector representation of a phrase involves determining for each keyword of the search query the value of the ratio of the keyword frequency to the inverse document frequency (TF-IDF, Term Frequency-Inverse Document Frequency) in a candidate document from a plurality of digital candidate documents.

[019] In some embodiments of the method, the formation of a vector representation of a phrase involves applying a vector representation algorithm to it.

[020] In some embodiments of the method, the text vector representation algorithm is the FastText word vector representation algorithm.

[021] In some embodiments of the method, determining the value of the ranking parameter includes transmitting the first, second and third vector representations to a joint machine learning model trained to determine the value of the ranking parameter for each document of the plurality of digital documents based on vector representations of (a) a search query used to find the plurality of digital documents, (b) each document of the plurality of digital documents satisfying this search query, and (c) at least one candidate phrase determined in the plurality of digital documents as lexically related to at least one keyword of this search query.

[022] In some embodiments of the method, it further comprises training the combined machine learning model by generating a training data set comprising a plurality of training digital objects, each of which comprises (a) a training vector representation of a training search query, (b) training vector representations of a plurality of candidate training digital documents satisfying the training search query, (c) a training vector representation of at least one candidate training phrase found in the plurality of training digital documents, lexically related to at least one keyword from the training search query, and (d) a label for each training document from the plurality of training digital documents, characterizing the degree of relevance of this document from the plurality of training digital documents to the training search query, transmitting the plurality of training digital objects to the combined machine learning model and minimizing, at each training iteration, the discrepancy between the current training prediction generated by the combined machine learning model and the label.

[023] In some embodiments of the method, the label is specified by a human evaluator.

[024] In some embodiments of the method, the label is generated by a machine learning model, pre-trained based on labels generated by human evaluators, to determine the degree of relevance of the digital document to the search query.

[025] In some embodiments of the method, the combined machine learning model comprises a deep machine learning model for determining semantic similarity.

[026] In some embodiments of the method, obtaining a second vector representation involves obtaining a second vector representation from a second machine learning model trained to generate vector representations of input digital documents.

[027] In some embodiments of the method, it further provides for training a second machine learning model by transmitting to it a plurality of digital documents from a search index of the digital platform.

[028] In some embodiments of the method, it further provides for reducing the volume of representations for each of the first, second and third vector representations before determining the value of the ranking parameter.

[029] In some embodiments of the method, the formation of the first, second and third vector representations involves using a transformer-based machine learning model, and the reduction in the volume of representations involves sequentially truncating the output data of each intermediate layer of the transformer-based machine learning model to a given length of each of the first, second and third vector representations.

[030] In some embodiments of the method, each of the first, second and third vector representations is assigned its own length, different from the others.

[031] In some embodiments of the method, the formation of the first and third vector representations is carried out independently of the formation of the second vector representation.

[032] In some embodiments of the method, the formation of a first vector representation characterizing a search query and a third vector representation characterizing at least one candidate phrase involves the use of a machine learning model trained to form vector representations of input phrases.

[033] In some embodiments of the method, it further comprises training the machine learning model by generating a training data set comprising a plurality of training digital objects, each of which comprises a training search query and a plurality of training candidate phrases found in a plurality of training digital documents that satisfy the training search query and are lexically associated with at least one keyword from the training search query, and transmitting the plurality of training digital objects to the machine learning model.

[034] In some embodiments of the method, the machine learning model is a transformer-based machine learning model.

[035] In some embodiments of the method, in the process of finding at least one candidate phrase lexically related to at least one keyword from the search query for each candidate document from the plurality of candidate digital documents, the method further comprises using an unprocessed version of this candidate document from the plurality of candidate digital documents.

[036] In some embodiments of the method, the raw version is located in a search index.

[037] According to a second broad aspect of the present technology, a server for ranking digital documents on a digital platform is implemented. The server comprises at least one processor and at least one non-volatile machine-readable memory storing executable commands, the execution of which by at least one processor on the server ensures the receipt of a search query sent by the user to the digital platform, the formation of a first vector representation characterizing the search query, finding in the search index of the digital platform a plurality of candidate digital documents that satisfy the search query, obtaining for each candidate digital document from the plurality of candidate digital documents a second vector representation formed before the search query is received, finding for each candidate document from the plurality of candidate digital documents at least one candidate phrase lexically associated with at least one keyword of the search query, the formation of a third vector representation characterizing at least one candidate phrase, determining on the basis of the first, second and third vector representations for each candidate document from the plurality of candidate digital documents a ranking parameter value characterizing the relevance of this a candidate document from a plurality of candidate digital documents to a search query, and ranking the plurality of candidate digital documents based on the ranking parameter values associated with them.

[038] As used herein, a "server" is a computer program running on appropriate hardware that is capable of receiving requests (e.g., from client devices) over a network, executing those requests, or initiating their execution. Such hardware may be implemented as a single physical computer or a single physical computer system, which is not essential to the present technology. As used herein, the use of the expression "server" does not mean that any particular task or all tasks (e.g., received commands or requests) are received, executed, or started on the same server (i.e., by the same software and/or hardware), but rather that any number of software or hardware may participate in receiving, transmitting, executing, or initiating the execution of any tasks or requests, or the results of any tasks or requests, and all of these software and hardware may represent a single server or multiple servers, and the expression "at least one server" covers both of these cases.

[039] In the context of the present description, a "client device" is any computer hardware capable of running software suitable for solving the task at hand. Thus, non-limiting examples of client devices include personal computers (desktop computers, laptops, netbooks, etc.), smartphones and tablets, as well as network equipment such as routers, switches and gateways. It should be noted that a device acting as a client in this context may act as a server for other client devices. The use of the expression "client device" does not exclude the use of multiple client devices to receive/transmit, perform or initiate the performance of any tasks or requests, the results of any tasks or requests, or the steps of any method described herein.

[040] As used herein, the term "database" means any structured collection of data, regardless of its specific structure, software for managing the database, and computer hardware for storing such data, using it, or otherwise enabling its use. A database may be located on the same hardware as the process that enables the storage or use of the information stored in the database, or on separate hardware, such as a dedicated server or group of servers.

[041] In the context of the present description, the expression "information" includes information of any kind or type capable of being stored in a database. Thus, information includes, among other things, audiovisual works (images, films, sound recordings, presentations, etc.), data (location data, numerical data, etc.), text (reviews, comments, questions, messages, etc.), documents, spreadsheets, word lists, etc.

[042] In the context of this description, the expression "component" means software (suitable for specific hardware) necessary and sufficient to perform the specific functions mentioned.

[043] In the context of the present description, the expression "computer storage medium" is intended to refer to media of any kind and type, including RAM, ROM, disks (CD-ROM, DVD, floppy disks, hard disks, etc.), USB drives, solid state drives, tape drives, etc.

[044] As used herein, the numerals "first," "second," "third," etc., serve merely to distinguish between the nouns to which they refer, and do not describe any particular relationships between those nouns. Thus, it should be understood that, for example, the terms "first server" and "third server" do not imply the existence of any particular order, type, chronology, hierarchy, or ranking of servers, and the use of these terms (in and of themselves) does not necessarily imply the presence of any "second server" in any particular situation. Furthermore, as occurs in other contexts herein, references to a "first" element and a "second" element do not exclude the possibility that the two elements may in fact be the same element. For example, in some cases the “first” and “second” servers may represent the same software and/or hardware, and in other cases, different software and/or hardware.

[045] Each embodiment of the present technology relates to at least one of the above-mentioned objectives and/or one of the above-mentioned aspects, but not necessarily to all of them. It should be understood that some aspects of the present technology associated with an attempt to achieve the above-mentioned objective may not correspond to this objective and/or may correspond to other objectives not explicitly mentioned herein.

[046] Additional and/or alternative features, aspects and advantages of embodiments of the present technology are contained in the following description, in the attached drawings and in the claims.

Brief description of the drawings

[047] These and other features, aspects and advantages of the present technology are contained in the following description, in the appended claims and in the following drawings.

[048] Fig. 1 is a schematic representation of an example of a computer system suitable for implementing certain non-limiting embodiments of systems and/or methods in accordance with the present technology.

[049] Fig. 2 illustrates a networked computer environment suitable for ranking digital documents on a digital platform, in accordance with some non-limiting embodiments of the present technology.

[050] Fig. 3 is a block diagram of the architecture of a machine learning model running on a server within the networked computing environment shown in Fig. 2, in accordance with some non-limiting embodiments of the present technology.

[051] Fig. 4 is a schematic diagram of an example digital document containing a plurality of candidate phrases that have been determined by a server within the networked computing environment shown in Fig. 2 to be lexically related to a search query used to locate an illustrative digital document, in accordance with some non-limiting embodiments of the present technology.

[052] Fig. 5 is a schematic diagram of a first machine learning model implemented based on the machine learning model architecture shown in Fig. 3 for generating vector representations of input phrases, in accordance with some non-limiting embodiments of the present technology.

[053] Fig. 6 is a schematic diagram of a second machine learning model implemented based on the machine learning model architecture shown in Fig. 3 for generating vector representations of input digital documents, in accordance with some non-limiting embodiments of the present technology.

[054] Fig. 7 is a schematic diagram of a process for training a combined machine learning model capable of generating ranking parameter values for digital documents based on vector representations generated by the first and second machine learning models shown in Figs. 5 and 6, respectively, in accordance with some non-limiting embodiments of the present technology.

[055] Fig. 8 is a schematic representation of the operation of the combined machine learning model shown in Fig. 7, in accordance with some non-limiting embodiments of the present technology.

[056] Fig. 9 is a flow chart of a method for ranking digital documents using the combined machine learning model shown in Fig. 8, in accordance with some non-limiting embodiments of the present technology.

Implementation of the invention

[057] The examples and conventional language provided herein are intended to provide a better understanding of the principles of the present technology, and not to limit its scope to such specifically provided examples and conditions. It is obvious that those skilled in the art are capable of devising various methods and devices that are not explicitly described or shown, but implement the principles of the present technology within its spirit and scope.

[058] Furthermore, in order to facilitate better understanding, the following description may contain simplified embodiments of the present technology. Those skilled in the art will understand that various embodiments of the present technology may be significantly more complex.

[059] In some cases, supposedly useful examples of modifications of the present technology are given. They are intended only to facilitate understanding and also do not define the scope or boundaries of the present technology. The list of modifications provided is not exhaustive and a person skilled in the art is able to develop other modifications within the scope of the present technology. In addition, if in some cases modifications are not described, this does not mean that they are impossible and/or that the description contains the only possible embodiment of a particular element of the present technology.

[060] In this case, the description of the principles, aspects and embodiments of the present technology, as well as their specific examples, are intended to cover their structural and functional equivalents, regardless of whether they are currently known or will be developed in the future. For example, those skilled in the art should understand that all block diagrams provided herein correspond to conceptual representations of illustrative circuit diagrams implementing the principles of the present technology. It should also be understood that all block diagrams, process diagrams, state transition diagrams, pseudocodes, etc. correspond to various processes that can be represented on a machine-readable physical information carrier and can be executed by a computer or processor, regardless of whether such a computer or processor is shown explicitly or not.

[061] The functions of the various elements shown in the drawings, including all functional blocks designated as a "processor" or "graphics processor", may be implemented using specialized hardware, as well as hardware capable of supporting the execution of the corresponding software. If a processor is used, these functions may be performed by one dedicated processor, one shared processor, and/or several separate processors, some of which may be used together. In some embodiments of the present technology, the processor may be a general-purpose processor, such as a central processing unit (CPU), or a processor dedicated to solving a specific task, such as a graphics processing unit (GPU). In addition, explicit use of the term "processor" or "controller" shall not be construed as referring solely to hardware capable of executing software, and may include, but is not limited to, hardware of a digital signal processor (DSP), a network processor, an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), ROM for storing software, RAM, and/or non-volatile memory. Other hardware, general purpose and/or custom, may also be included.

[062] Software modules or simply modules, the implementation of which is supposed to be based on software, may be represented here as any combination of flow chart elements or other elements indicating the execution of process steps and/or containing a text description. Such modules may be executed by hardware, shown explicitly or implied.

[063] Taking into account the above principles, some non-limiting examples are discussed below to illustrate various embodiments of aspects of the present technology.

Computer system

[064] Fig. 1 shows a computer system 100 suitable for use in some embodiments of the present technology. The computer system 100 includes various hardware elements, including one or more single-core or multi-core processors, collectively represented by a processor 110, a graphics processor 111, a solid-state drive 120, RAM 130, a display interface 140, and an input/output interface 150.

[065] Communication between elements of the computer system 100 may be accomplished via one or more internal and/or external buses 160 (such as a PCI bus, a USB bus, an IEEE 1394 FireWire bus, a SCSI bus, a Serial-ATA bus, etc.) to which various hardware elements are electronically connected.

[066] The input/output interface 150 may be connected to a touch screen 190 and/or to one or more internal and/or external buses 160. The touch screen 190 may be part of a display. In some non-limiting embodiments of the present technology, the touch screen 190 is a display. The touch screen 190 may also be referred to as a screen 190. In the embodiments shown in FIG. 1, the touch screen 190 comprises touch means 194 (for example, touch elements embedded in the display layer and providing for recording the physical interaction between the user and the display) and a touch input/output controller 192 providing communication with the display interface 140 and/or with one or more internal and/or external buses 160. In some embodiments, the input/output interface 150 may be connected to a keyboard (not shown), a mouse (not shown), or a touch pad (not shown), which provide for user interaction with the computer system 100 in addition to or instead of the touch screen 190.

[067] It should be noted that in some non-limiting embodiments of the present technology, some elements of the computer system 100 may be missing. For example, the touch screen 190 may be missing, including, but not limited to, in cases where the computer system is implemented as a server.

[068] In accordance with embodiments of the present technology, the solid-state drive 120 stores software instructions suitable for loading into RAM 130 and execution by the processor 110 and/or the graphics processor 111. For example, the software instructions may be part of a library or an application.

Networked computing environment

[069] Fig. 2 is a schematic representation of a networked computing environment 200 suitable for use with some non-limiting embodiments of systems and/or methods in accordance with the present technology. The networked computing environment 200 comprises a server 202 coupled to an electronic device 204 via a communications network 208. In some non-limiting embodiments of the present technology, the electronic device 204 may be coupled to a user 216.

[070] In some non-limiting embodiments of the present technology, the electronic device 204 may be any computer hardware capable of running software suitable for solving the task at hand. In this regard, the electronic device 204 may comprise some or all of the elements of the computer system 100 shown in Fig. 1. Non-limiting examples of the electronic device 204 include personal computers (desktop computers, laptops, netbooks, etc.), smartphones, and tablets. It will be appreciated that in some non-limiting embodiments of the present technology, the electronic device 204 may not be the only electronic device associated with the user 216, and the user 216 may also be associated with other electronic devices (not shown in Fig. 2) connected to the server 202 via the communication network 208 without departing from the spirit and scope of the present technology.

[071] In some non-limiting embodiments of the present technology, the server 202 is implemented as a conventional computer server and may also include some or all of the elements of the computer system 100 shown in Fig. 1. In a specific non-limiting example, the server 202 is implemented as a Dell™ PowerEdge™ server running a Microsoft™ Windows Server™ operating system, but it may also be implemented on any other suitable types of hardware, software, and/or firmware, or a combination thereof. In the shown non-limiting embodiments of the present technology, the server 202 is a single server. In other non-limiting embodiments of the present technology (not shown), the functions of the server 202 may be distributed among multiple servers.

[072] In some non-limiting embodiments of the present technology, a digital platform 210 may be hosted on the server 202. In general, the digital platform 210 is a web resource capable of managing, accessing, presenting, and interacting with a plurality of digital documents hosted on the digital platform 210. In general, the types of digital documents hosted on the digital platform 210 depend on the specific implementation thereof. For example, in some non-limiting embodiments of the present technology, the digital platform 210 is a streaming audio platform, such as the Spotify™, Yandex™, and other streaming audio platforms. Music™, etc., and the plurality of digital documents may include various digital audio documents, such as audio tracks, audio books, podcasts, etc. In another example, where the digital platform 210 is a video hosting platform or a video streaming platform, such as the YouTube™ video hosting platform or the Netflix™ video streaming platform, the plurality of digital documents may include various digital video documents, such as video clips, films, news videos, etc. In yet another example, where the digital platform is implemented as an online classifieds platform, such as the Yandex™ Market™ online classifieds platform, the Avito™ online classifieds platform, etc., the plurality of digital documents may include advertisements for the sale of various products, such as goods and services. In another example, the digital platform 210 may be implemented as a search engine, such as the Google™ search engine, the Yandex™ search engine, etc., and the plurality of digital documents may be a web document that may contain digital documents of all the types mentioned above. Obviously, other embodiments of the digital platform 210 are also possible, as well as other types of digital documents placed on it.

[073] Accordingly, to provide access to a plurality of digital documents to users of digital platform 210, such as user 216, digital platform 210 may provide search capability by allowing user 216 to submit search queries to digital platform 210 (e.g., via a special user interface), in response to which digital platform 210 may provide a selection of corresponding sets of digital documents.

[074] According to some non-limiting embodiments of the present technology, the server 202 may be coupled to the search index database 206 to provide storage of a plurality of digital documents potentially accessible via the communication network 208. In this regard, in those embodiments in which the digital platform 210 includes a streaming platform or an online classifieds platform, the search index database 206 may be pre-loaded with links to a plurality of digital documents from digital document providers such as musicians, record labels, and merchandise sellers. However, in those embodiments in which the digital platform 210 is implemented as a search engine, the links to a plurality of digital documents may be pre-loaded into the search index database 206 using a process called "content crawling," which in some non-limiting embodiments of the present technology may, for example, also be implemented by the server 202. Furthermore, although in the embodiments shown in FIG. 2, the search index database 206 is depicted as a single entity, it should be understood that in other non-limiting embodiments of the present technology, the functions of the search index database 206 may be distributed among several databases. In some non-limiting embodiments of the present technology, the search index database 206 may be accessed by the server 202 via the communication network 208, rather than via a direct communication channel (not separately indicated), as shown in Fig. 2.

[075] It is evident that in some non-limiting embodiments of the present technology, the search index database 206 may further provide for storing pre-generated vector (i.e., numeric) representations of each digital document of the plurality of digital documents. In some non-limiting embodiments of the present technology, the vector representations may be generated by a third-party server (not shown) that runs, for example, a specially trained machine learning model. In some non-limiting embodiments of the present technology, the vector representations may be generated by the server 202, as described in detail below.

[076] Thus, in accordance with some non-limiting embodiments of the present technology, a user 216, using an electronic device 204, may send a search query 212 to a digital platform 210, and the digital platform 210 may be able to locate in a search index database 206 a set 214 of digital documents that satisfy the search query 212.

[077] According to some non-limiting embodiments of the present technology, the server 202 is capable of finding the set 214 of digital documents using a mathematical model capable of evaluating the relevance of a document from a plurality of digital documents hosted on the digital platform 210 to the search query 212. In some non-limiting embodiments of the present technology, the mathematical model may perform a ranking function. For example, the ranking function may be implemented based on a bag-of-words model and include the Okapi BM25 ranking function. It should be noted that there are no limitations on the method by which the server 202 finds the set 214 of digital documents. For example, in other non-limiting embodiments of the present technology, the server 202 may use machine learning approaches, such as a decision tree machine learning model, to find the set 214 of digital documents.

[078] In addition, after the server 202 finds the set 214 of digital documents, it is able to rank the digital documents from the set 214 of digital documents, for example, depending on the degree of their relevance to the search query 212, in order to help the user 216 of the digital platform 210 navigate the set 214 of digital documents. Thus, the server 202 is able to form a ranked set 220 of digital documents.

[079] The developers of the present technology took into account that the degree of satisfaction of the user 216 with the set 214 of digital documents can increase in those cases when the ranks in the set 214 of digital documents are determined based on both lexical and semantic relationships between the digital documents from the set 214 of digital documents and the keywords of the search query 212.

[080] In the context of the present technology, the term "semantic links" between the keyword of the search query 212 and the digital document from the set 214 of digital documents denotes the correspondence of the digital document to the semantics (i.e., the meaning) of the keyword of the search query 212. In particular, the semantic links between the keyword and the digital document can be determined by whether the digital document contains words from the semantic field that includes the keyword of the search query 212, i.e., from the lexical set of semantically grouped words related to the corresponding topic. For example, the semantic field of the word "movement" can include the words "run", "go", "fly", "drive", "walk", "fast", etc., including their various derivatives in all parts of speech. A specific example of a semantic relationship between a digital document and a keyword of the search query 212 may be the presence of synonyms or antonyms of this keyword in this digital document. However, the term "semantic relationships" should not be interpreted as being limited to semantic relationships between words. For example, a digital document containing an image of a cat or a video clip of a cat is considered to be semantically related to the keyword of the search query 212 "cat" and other keywords from the same semantic field, such as "cat litter", "catnip", "kitten", and vice versa, images or video clips of such objects are considered to be semantically related to the word "cat". In another example, an audio recording of a cat meowing is also considered to be semantically related to the word "cat".

[081] In the context of the present description, the term "lexical relationships" between the search query keyword 212 and the digital document denotes the correspondence of the digital document to the vocabulary of the search query keyword 212, that is, its linguistic form recorded in the dictionary. In particular, the semantic relationships between the keyword and the digital document may be determined by (a) whether the digital document contains words that completely match the search query keyword 212, (b) whether the digital document contains one or more grammatical forms of the search query keyword 212, for example, forms that differ in the categories of number and case (or declension, for example, кат - кошки - кошки) for nouns and in the categories of person, number and tense (or conjugation, for example, спишет - спишет - спишет) for verbs, and (c) whether the digital document contains cognate words formed from the search query keyword 212 within a given part of speech, for example, декция - активие - экзание (noun).

[082] In some non-limiting embodiments of the present technology, in order to determine the degree of relevance of a digital document from the set 214 of digital documents to the search query 212 in terms of semantics and lexicon, the server 202 is configured to train and then apply a machine learning algorithm 218, including a combined machine learning model 602, the training and use of which is described in detail below with reference to Figs. 7 and 8. In particular, in accordance with some non-limiting embodiments of the present technology, the combined machine learning model 602 is configured to determine a ranking parameter value for each digital document from the set 214 of digital documents based on the degree of relevance of this digital document to the search query 212 in terms of lexicon and semantics. As explained in the description below, for this, the unified machine learning model 602 may receive (a) a first vector representation of the search query 212, for example, the vector representation 512 of the search query shown in Fig. 5, (b) a second vector representation of the digital document, for example, the vector representation 516 of the document for the illustrative digital document 402 shown in Fig. 6, and (c) a third vector representation of at least one candidate phrase from the set 214 of digital documents lexically related to at least one keyword of the search query 212, for example, the vector representation 514 of the phrases shown in Fig. 5. The method of causing the server 202 to receive the first, second and third vector representations for transmitting them to the unified machine learning model 602 is described in detail below with reference to Figs. 4-6.

[083] In some non-limiting embodiments of the present technology, the unified machine learning model 602 may be implemented based on a neural network architecture. For example, in such embodiments, the unified machine learning model 602 may be a machine learning model based on a deep semantic similarity model (DSSM), a machine learning model based on a long short-term memory (LS™) neural network, or a machine learning model based on a transformer.

[084] It should be understood that other types of machine learning models can be used to implement the combined machine learning model 602, such as, among others, a decision tree-based machine learning model, a gradient boosted decision tree-based machine learning model, an association rule-based machine learning model, an inductive logic programming-based machine learning model, a support vector machine-based machine learning model, a clustering-based machine learning model, Bayesian networks, a reinforcement learning machine learning model, a representation learning machine learning model, a metric and proximity learning machine learning model, a sparse dictionary learning machine learning model, a genetic algorithm-based machine learning model, etc.

[085] In addition, using the values of the ranking parameter determined in the specified manner, the server 202 can provide a ranking of the digital documents in the set 214 of digital documents, forming a ranked set 220 of digital documents. In this way, the server 202 is able to provide the user 216 with more relevant digital documents, which contributes to an increase in the degree of satisfaction of the user 216 from interaction with the digital platform 210.

[086] It can be stated that, in general, the server 202 performs two processes with respect to the combined machine learning model 602 in the machine learning algorithm 218. The first of these two processes is a training process in which the server 202 trains the combined machine learning model 602 based on a training data set in order to determine a ranking parameter value for each digital document in the set 214 of digital documents. A description of the training process is given below with reference to Fig. 7. The second process is a using process in which the server 202 applies the combined machine learning model 602 to rank the digital documents in the set 214 of digital documents, as described below with reference to Fig. 8, in accordance with some non-limiting embodiments of the present technology.

[087] An example of a machine learning model architecture that may be used to implement the machine learning algorithm 218, including the combined machine learning model 602, is described below with reference to Fig. 3.

Communication network

[088] In some non-limiting embodiments of the present technology, the communication network 208 is the Internet. In other non-limiting embodiments of the present technology, the communication network 208 may be implemented as any suitable local area network (LAN), wide area network (WAN), private communication network, etc. It should be understood that the embodiments of the communication network are given for illustrative purposes only. The method of implementing the communication links (not separately indicated) connecting the server 202 and the electronic device 204 to the communication network 208 depends, among other things, on the implementation of the server 202 and the electronic device 204. For example, among other things, it can be noted that in those embodiments of the present technology where the electronic device 204 is implemented as a wireless communication device, such as a smartphone, the communication link can be implemented as a wireless communication link. Examples of wireless communication links can be, among other things, a 3G communication network channel, a 4G communication network channel, etc. The communication network 208 may also use a wireless connection to the server 202.

Machine learning model architecture

[089] Fig. 3 is a block diagram of a machine learning model architecture 300 used to implement a machine learning algorithm 218 in accordance with some non-limiting embodiments of the present technology. As noted above, in some non-limiting embodiments of the present technology, the machine learning model architecture 300 may be built based on the BERT machine learning model, as described, for example, in the aforementioned article by Devlin et al. Similar to BERT, the machine learning model architecture 300 comprises a transformer stack 302 comprising transformer blocks, such as transformer blocks 304, 306, and 308.

[090] Each of the transformer blocks 304, 306, and 308 comprises a transformer encoder block, as described, for example, in the aforementioned article by Vaswani et al. Each of the transformer blocks 304, 306, and 308 has a multi-headed attention layer 320 (for clarity, only the transformer block 304 is shown here) and a feedforward neural network layer 322 (for clarity, only the transformer block 304 is also shown here). The transformer blocks 304, 306, and 308 have the same overall structure, but (after training) they are assigned different weights. In the multi-headed attention layer 320, there are dependencies between the inputs of the transformer block, which can be used, for example, to convey contextual information to each input depending on the data at all other inputs of the transformer block. In the feedforward neural network layer 322, such dependencies are generally not present, so the input data of the feedforward neural network layer 322 can be processed in parallel. It should be understood that in Fig. 3 only three transformer units (transformer units 304, 306 and 308) are shown, but in practical embodiments of the technology described herein, there may be many more such transformer units in the transformer stack 302. For example, in some embodiments, 12 transformer units may be used in the transformer stack 302.

[091] Input data 330 of the transformer stack 302 comprises tokens, such as token 332 [CLS] and tokens 334. Tokens 334 may represent, for example, words or parts of words. Token 332 [CLS] serves as a representation for classifying the entire set of tokens 334. Each of token 334 and token 332 [CLS] is represented as a vector. In some embodiments, each of these vectors may have a length of, for example, 768 floating point values. It should be understood that various compression methods can be used to effectively reduce the size (dimensionality) of the vectors.

[092] In some non-limiting embodiments of the present technology, the number of tokens 334 used as input data 330 for the transformer stack 302 may be fixed. For example, in some non-limiting embodiments of the present technology, 1024 tokens may be used, and in other embodiments, the transformer stack 302 may be capable of holding 512 tokens (in addition to the [CLS] token 332). Those input data 330 whose length is less than the fixed number of tokens 334 may be expanded to the fixed length, for example, by adding padding tokens.

[093] In some embodiments, input data 330 may be generated based on training digital object 336 using tokenizer 338. The architecture of tokenizer 338 typically depends on training digital object 336, which serves as input data for tokenizer 338. For example, in some non-limiting embodiments of the present technology, tokenizer 338 may use known encoding techniques, such as byte pair encoding, as well as pre-trained neural networks to generate input data 330.

[094] In some non-limiting embodiments of the present technology, the tokenizer 338 may be implemented based on a WordPiece byte pair encoding scheme, such as that used in BERT training models with a sufficiently large vocabulary. In particular, in some non-limiting embodiments of the present technology, the vocabulary size may be approximately 120,000 tokens. In some non-limiting embodiments of the present technology, the input data 330 may be pre-processed before the tokenizer 338 is applied. For example, all words in the input data 330 may be converted to lowercase, after which Unicode NFC normalization may be performed. A WordPiece byte pair encoding scheme that may be used in some implementations to create a token dictionary is described, for example, in Rico Sennrich et al., “Neural Machine Translation of Rare Words with Subword Units,” Proceedings of the ^54th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 1715–1725, 2016.

[095] Furthermore, in some non-limiting embodiments of the present technology, in addition to or instead of the tokenizer 338, the server 202 may use a vector representation algorithm to generate the input data 330. In general, the vector representation algorithm is capable of transforming the input data, such as the digital object 336, into a numeric vector in the vector representation space. In this case, a specific implementation of the vector representation algorithm typically depends on the data type of the digital object 336. For example, in those non-limiting embodiments of the present technology in which the digital object 336 comprises one of the past queries and a plurality of past candidate phrases associated with it, as described below, the vector representation algorithm may be a text vector representation algorithm.

[096] In some non-limiting embodiments of the present technology, the text vector representation algorithm may be, among others, one of the text vector representation algorithms Word2Vec, GloVe, FastText, etc., which the server 202 may use to generate input data 330 for the machine learning model architecture 300.

[097] In another example, in those non-limiting embodiments of the present technology, in which the digital object 336 comprises an audio recording, the vector representation algorithm may be implemented as an audio vector representation algorithm. According to some non-limiting embodiments of the present technology, the audio vector representation algorithm may be, among other things, a Seq2Seq Autoencoder audio vector representation algorithm, a convolutional vector regression audio vector representation algorithm, a letter n-gram-based audio vector representation algorithm, a long-term and short-term memory-based neural network audio vector representation algorithm, and the like.

[098] In another example, in those non-limiting embodiments of the present technology, in which the digital object 336 comprises an image, the vector representation algorithm may be implemented as an image vector representation algorithm. Examples of an image vector representation algorithm include algorithms implemented based on deep neural networks, such as convolutional neural networks, including, among others, the InceptionV3 image vector representation algorithm, the SqueezeNet image vector representation algorithm, and the DeepLoc image vector representation algorithm.

[099] In further non-limiting embodiments of the present technology, after using the tokenizer 338, generating the input data 330 may further include using by the server 202 a position embedding algorithm (not shown) capable of recording position data in portions of the input training digital object 336. For example, if the input training digital object 336 comprises a text phrase, the position embedding algorithm may generate a vector indicating position data among words in the text phrase. There is no limitation on the method of implementing the position embedding algorithm and such an algorithm may be, among other things, for example, a sinusoidal positional embedding algorithm, a frame stacking positional embedding algorithm, or a convolutional positional embedding algorithm.

[0100] The output data 350 of the transformer stack 302 comprises the output data 352 [CLS] and the output data vector 354 comprising the output value for each of the tokens 334 contained in the input data 330 of the transformer stack 302. The output data 350 can then be sent to the task module 370. In some embodiments, as shown in Fig. 3, the task module 370 uses only the output data 352 [CLS], which serves as a representation of the entire output data vector 354. This may be most appropriate in cases where the task module 370 is used as a classifier or for outputting a label or value characterizing the entire input training digital object 336, for example, for the purpose of generating a relevance score, in particular a ranking parameter value for a digital document from the set 214 of digital documents, as described above.

[0101] In some non-limiting embodiments of the present technology (not shown in FIG. 3), some or all of the values of the output data vector 354 and possibly the output data 352 [CLS] may serve as input to the task module 370. This may be most useful in cases where the task module 370 is used to generate labels or values for each of the tokens 334 of the input data 330, in particular for predicting a masked or missing token or for recognizing named entities. In some non-limiting embodiments of the present technology, the task module 370 may comprise a feedforward neural network (not shown) that generates a task-oriented output 380, such as a relevance score or a click-through probability. Other models may also be used in the task module 370. For example, the task module 370 itself may be a transformer or another type of neural network. Additionally, the task-oriented 380 result can serve as input to other models, such as the CatBoost model described in Dorogush et al., “CatBoost: gradient boosting with categorical features support,” NIPS, 2017.

[0102] It should be understood that in order to facilitate understanding of some non-limiting embodiments of the present technology, the architecture of the machine learning model 300 described above with reference to Fig. 3 has been simplified. For example, in a practical embodiment of the architecture of the machine learning model 300, each of the transformer units 304, 306 and 308 may also perform layer normalization operations, and the task module 370 may perform a softmax logistic normalization function, etc. One skilled in the art will understand that such operations are widely used in neural networks and in deep learning models, such as a model with the architecture of the machine learning model 300.

Formation of vector representations

[0103] As noted above, the lexical relationship between the digital document and the search query 212 may be determined by the presence in the digital document of at least one word that is represented in the same linguistic form as at least one keyword of the search query 212.

[0104] In this regard, in order to rank digital documents in the set 214 of digital documents using the combined machine learning model 602, in some non-limiting embodiments of the present technology, the server 202 may first ensure that each digital document in the set 214 of digital documents contains a plurality of candidate phrases that are lexically related to at least one keyword of the search query 212. In other words, as noted above, each candidate phrase of the plurality of candidate phrases may comprise words that (a) exactly match the at least one keyword of the search query, (b) are represented in one or more grammatical forms of at least one keyword of the search query 212, and/or (c) include one or more cognates of at least one keyword of the search query 212 within a single part of speech.

[0105] Fig. 4 is a schematic diagram of an illustrative digital document 402 of a set 214 of digital documents comprising a plurality 404 of candidate phrases, in accordance with some non-limiting embodiments of the present technology.

[0106] In the example of Fig. 4, it is assumed that the digital platform 210 is an online classifieds platform, to access which the user 216 needs to enter the corresponding URL of the online classifieds platform in the address bar of the browser application on the electronic device 204. The user 216 can send a search query 212 to the digital platform 210 with the text "Macaron Recipe Pierre Herme". The server 202, for example, using the above-mentioned mathematical model, is able to find in the search index database 206 storing a plurality of digital documents a set 214 of digital documents that satisfy this search query. Moreover, in an illustrative digital document 402 of the set 214 of digital documents, the server 202 is able to find a plurality 404 candidate phrases that contain at least one word lexically related to at least one keyword of the search query 212.

[0107] According to some non-limiting embodiments of the present technology, the server 202 is capable of finding a plurality 404 of candidate phrases in the illustrative digital document 402 by calculating values of a proximity metric between the search query 212 and individual phrases of the illustrative digital document 402. In some non-limiting embodiments of the present technology, the proximity metric is determined by the distance between vectors representing the search query 212 and a phrase of the illustrative digital document 402 in the space of vector representations. In this regard, the proximity metric may include, for example, a cosine proximity metric. In some non-limiting embodiments of the present technology, the server 202 is configured to determine individual phrases in the illustrative digital document 402 as candidate phrases for inclusion in the set 404 candidate phrases if their values of the metric of proximity to the search query 212 exceed a predetermined threshold of proximity. In other non-limiting embodiments of the present technology, the server 202 is configured to rank the phrases of the illustrative digital document 402 according to their values of the metric of proximity and select N priority phrases for inclusion in the set 404 candidate phrases.

[0108] In this case, there are no limitations on the method of generating vectors by the server 202 representing the search query 212 and individual phrases of the illustrative digital document 402. For example, in some non-limiting embodiments of the present technology, the server 202 is capable of generating such vectors by determining for each keyword of the search query 212 a value of the ratio of the keyword frequency to the inverse document frequency (TF-IDF) with respect to each phrase in the illustrative digital document 402. In other non-limiting embodiments of the present technology, the server 202 is capable of applying one of the above-mentioned algorithms for vector representation of text to the search query and each phrase of the illustrative digital document 402.

[0109] Accordingly, the server 202 may, for example, select the first candidate phrase 401 because it contains words that exactly match the keywords “Pierre Herme” (highlighted) from the search query 212. In addition, the server 202 may select the second candidate phrase 403 because it contains the plural form of the keyword “Macaron” from the search query 212. Similarly, the server 202 may select the third and fourth candidate phrases 405, 407. Finally, the server 202 may select the fifth candidate phrase 409 because it contains the possessive form of the keywords “Pierre Herme,” the plural form of the keyword “Macaron,” and a word that exactly matches the keyword “Macaron.”

[0110] In some non-limiting embodiments of the present technology, the set 404 candidate phrases for subsequent use in ranking the set 214 of digital documents may contain a predetermined number of candidate phrases, such as 3, 5, 10, 100, or 147,000. In these embodiments, the server 202 is configured to select from the illustrative digital document 402 for inclusion in the set 404 candidate phrases only those candidate phrases that contain the greatest number of words lexically related to the keywords of the search query 212. For example, if the predetermined number of candidate phrases is 3, the server 202 may include in the set 404 candidate phrases the fifth candidate phrase 409, the second candidate phrase 403, and one of the first, third, and fourth candidate phrases 401, 405, and 407.

[0111] In some non-limiting embodiments of the present technology, each candidate phrase of the plurality 404 candidate phrases may contain at least a predetermined target number of words lexically related to at least one keyword of the search query 212. In particular, in such embodiments, if the predetermined target number of candidate phrases is, for example, two, the server 202 may exclude from the plurality 404 candidate phrases all candidate phrases except the second and fifth candidate phrases 403, 409.

[0112] Thus, using an approach similar to that described above with respect to the exemplary digital document 402, the server 202 is capable of analyzing other documents from the set 214 of digital documents to select candidate phrases for inclusion in the set 404 of candidate phrases. In some non-limiting embodiments of the present technology, the server 202 may analyze all digital documents from the set 214 of digital documents. In other non-limiting embodiments of the present technology, the server 202 may analyze only a portion of the set 214 of digital documents to select candidate phrases for the set 404 of candidate phrases. The server 202 is capable of determining such a portion of the set 214 of digital documents as the N priority digital documents that best satisfy the search query 212.

[0113] In some other non-limiting embodiments of the present technology, for determining the plurality 404 candidate phrases, the server 202 is configured to (a) determine for each phrase in the set 214 digital documents a value of a proximity metric to the search query 212, as described above, (b) rank the phrases depending on the values of the proximity metric, and (c) select N (e.g., 3, 5, or 20) priority phrases from the set 214 digital documents for inclusion in the plurality 404 candidate phrases.

[0114] Thus, in accordance with some non-limiting embodiments of the present technology, the server 202 is capable of obtaining information about lexical relationships between the search query 212 and the set 214 of digital documents from their raw versions stored in the search index database 206 when it is populated, for example, by crawling the content of the communication network 208 by the server 202, as indicated above.

[0115] In addition, in some non-limiting embodiments of the present technology, for transmitting the plurality 404 candidate phrases and the search query 212 to the combined machine learning model 602, the server 202 is capable of determining vector representations thereof. In some non-limiting embodiments of the present technology, the server 202 is capable of determining vector representations of the plurality 404 candidate phrases and the search query 212 using one of the text vector representation algorithms indicated above by way of example in relation to the description of the architecture 300 of the machine learning model. However, in other non-limiting embodiments of the present technology, the server 202 is capable of determining such vector representations using a specially trained machine learning model.

[0116] Thus, in accordance with some non-limiting embodiments of the present technology, the machine learning algorithm 218 may include a first model 502, schematically illustrated in Fig. 5. In accordance with some non-limiting embodiments of the present technology, the first machine learning model 502 may be implemented as a deep neural network, such as a long-short-term memory neural network or a recurrent neural network. In some non-limiting embodiments of the present technology, the first machine learning model 502 may be implemented as a transformer-based machine learning model. In this regard, the first machine learning model 502 may include some or all of the elements of the machine learning model architecture 300 described above.

[0117] According to some non-limiting embodiments of the present technology, the first machine learning model 502 is capable of generating vector representations of input phrases. For example, the first machine learning model 502 may receive a search query 212 to generate a vector representation 512 of the search query and receive a plurality 404 candidate phrases to generate a vector representation 514 of the phrases.

[0118] To train the first machine learning model 502 to generate vector representations of input phrases, the server 202 may generate a first training data set comprising a first plurality of training digital objects, each of which comprises a training phrase. Like the search query 212 and the plurality 404 of candidate phrases, in some non-limiting embodiments of the present technology, this training phrase may comprise one of the training search queries (for example, one of the past queries sent by users of the digital platform 210) and a training set of candidate phrases selected like the plurality 404 of candidate phrases for the search query 212. However, in other non-limiting embodiments of the present technology, the training phrase may be any other phrase that the server 202 is capable of extracting, for example, from the entire plurality of digital documents located on the digital platform 210. In another example, the server 202 is capable of extracting training phrases for training the first machine learning model 502 from other resources of the communication network 208, including any text information.

[0119] In addition, the server 202 is capable of transmitting each training digital object from the first plurality of training digital objects to the first machine learning model 502, training the first machine learning model 502 to generate vector representations of the input phrases. As indicated above in connection with the description of the architecture 300 of the machine learning model, the server 202 is capable of transmitting the training phrase to the tokenizer 338 to generate the input data 330, and also transmitting the input data 330 to the transformer units 304, 306 and 308 to generate the output data 350 comprising a numeric representation of the training phrase taking into account the context of the words in the training phrase. In other words, the output data 350 of the first machine learning model 502 may comprise a vector representation of the training phrase, including the aggregate semantics of the training phrase. Thus, by transmitting the first plurality of training digital objects to the first machine learning model 502, the server 202 can determine the weights of the transformer blocks 304, 306 and 308, training the first machine learning model 502 to form vector representations of input phrases, such as the search query 212 and the plurality 404 of candidate phrases.

[0120] In addition, as most clearly shown in Fig. 5, in accordance with some non-limiting embodiments of the present technology, the server 202 is capable of reducing the amount of representations (that is, the number of values in the output data vector 354) in the output data 350 of the first machine learning model 502. In this regard, in some non-limiting embodiments of the present technology, the server 202 is capable of truncating the values in the output data 350 to a desired specified length. In other non-limiting embodiments of the present technology, the server 202 is capable of reducing the volume of the representations of the output data 350 of the first machine learning model 502 by successively truncating the values in the output data of the intermediate layers, that is, each of the transformer blocks 304, 306 and 308, until the required specified length of the output data 350 is reached. In various non-limiting embodiments of the present technology, the server 202 is capable of selecting the required specified length of the output data 350 for the first machine learning model 502 and the truncated lengths of the output data of the transformer blocks 304, 306 and 308 based on a trade-off between the speed and quality of generating the output data 350 for subsequent transmission to one of the inputs of the combined machine learning model 602. For example, the required specified length of the output data 350 after truncation can be 8, 16 or 32 values in the corresponding vector representations.

[0121] In some non-limiting embodiments of the present technology, the server 202 is capable of reducing the size of the representations in the output data 350 of the first machine learning model 502 for different vector representations in different ways. In other words, to generate the vector representation 512 of the search query, the server 202 can cause the values of the output data 350 of the first machine learning model 502 to be truncated to a first necessary specified length, which can be, for example, 8 values. To generate the vector representation 514 of phrases, the server 202 can cause the values of the output data 350 of the first machine learning model 502 to be truncated to a second necessary specified length, which can be, for example, 16 values.

[0122] Thus, the server 202 is capable of using the first machine learning model 502 to generate vector representations of the search queries and the corresponding sets of candidate phrases, which can provide further training of the combined machine learning model 602 and its use for ranking digital documents, such as the set 214 of digital documents, taking into account their lexical relationships with the search query 212, as described below.

[0123] In addition, to determine semantic relationships between a digital document from the set 214 of digital documents and a search query 212 using the combined machine learning model 602, the server 202, in accordance with some non-limiting embodiments of the present technology, is capable of receiving a vector representation of the digital document.

[0124] As indicated above, in some non-limiting embodiments of the present technology, a vector representation of a digital document may be pre-generated by a third-party server and stored in the search index database 206 in association with the corresponding digital documents of the digital platform 210. In some non-limiting embodiments of the present technology, the server 202 is capable of independently generating the vector representation of the document. In some non-limiting embodiments of the present technology, the server 202 is capable of generating the vector representation of the document depending on the type of this digital document (i.e., what data it contains - text, audio, video, etc.), using one of the plurality of implementations of the vector representation algorithm indicated above in relation to the description of the architecture 300 of the machine learning model.

[0125] In other non-limiting embodiments of the present technology for generating vector representations of documents, the machine learning algorithm 218 located on the server 202 may include a second machine learning model 504, schematically illustrated in Fig. 6.

[0126] According to some non-limiting embodiments of the present technology, the second machine learning model 504, like the first machine learning model 502, may be implemented as a deep neural network, such as a transformer-based machine learning model. In this regard, the second machine learning model 504 may include some or all of the elements of the machine learning model architecture 300 described above.

[0127] For example, the second machine learning model 504 is capable of receiving an illustrative digital document 402 and generating a vector representation 516 of the document that takes into account contextual information for elements of the illustrative digital document 402. To this end, in some non-limiting embodiments of the present technology, the server 202 may pre-train the second machine learning model to generate vector representations of various digital documents.

[0128] In this regard, according to some non-limiting embodiments of the present technology, the server 202 is configured to (a) generate a second training data set comprising a second plurality of training digital objects, each of which comprises one of the plurality of digital documents hosted on the digital platform 210, (b) transmit each of the second plurality of training digital objects to the tokenizer 338 to generate input data 330 for the second machine learning model 504, (c) transmit the input data 330 to the transformer units 304, 306 and 308 to ensure the generation of output data 350 comprising a numerical representation of the digital document. Thus, by transmitting all digital documents from the plurality of digital documents of the digital platform 210, the server 202 is able to determine the weights of the transformer blocks 304, 306 and 308 of the second machine learning model 504, training the second machine learning model 504 to generate vector representations of documents, for example, vector representation 516 of a document for an illustrative digital document 402.

[0129] It is evident that when using the second machine learning model 504 to rank the set 214 of digital documents, generating vector representations for each document from the set 214 of digital documents in real time may be a very resource-intensive task for the server 202. Accordingly, in some non-limiting embodiments of the present technology, the server 202 is configured to (a) cause the second machine learning model 504 to pre-generate vector representations of documents for a plurality of digital documents and store them in association with the corresponding digital documents, for example, in the search index database 206, and (b) if necessary use the thus-generated document representation for a digital document from the set 214 of digital documents as input to the combined machine learning model 602, as described below. This allows saving the computing resources of the server 202 in real time.

[0130] In addition, as in the case of the first machine learning model 502, in some non-limiting embodiments of the present technology, the server 202 is capable of causing the second machine learning model 504 to reduce the volume of representations of its output data 350 using one of the truncation methods described above. It should be noted that the resulting length of the vector representation 516 of the document may either coincide with the lengths of the vector representations 512, 514 of the search query and phrases, or differ from them.

[0131] Thus, the server 202 is capable of generating vector representations of documents and then, based on them, training the combined machine learning model 602 and using it to rank the set 214 digital documents taking into account the semantic relationship of the digital document from the set 214 digital documents with the search query 212.

[0132] In other words, after receiving vector representations for (a) search queries, (b) sets of candidate phrases lexically related to the search queries, and (c) digital documents, such as by training and using the first and second machine learning models 502, 504, the server 202 is capable of training the combined machine learning model 602 to rank the digital documents based on their lexical and semantic relationships with the corresponding search queries, as described below.

The learning process

[0133] Fig. 7 is a schematic diagram of a process for training a combined machine learning model 602 to rank a set 214 of digital documents in accordance with some non-limiting embodiments of the present technology. As noted above, in some non-limiting embodiments of the present technology, the combined machine learning model 602 may be implemented as a machine learning model based on a deep model for determining semantic similarity. In other non-limiting embodiments of the present technology, the combined machine learning model 602 may be a transformer-based machine learning model. In such embodiments, the combined machine learning model 602 may include some or all of the elements of the machine learning model architecture 300 described above.

[0134] According to some non-limiting embodiments of the present technology, to train the combined machine learning model 602 to generate ranking parameter values, the server 202 is configured to generate a third training data set comprising a third plurality of training digital objects, each of which comprises (a) a vector representation 613 of a training search query for a training search query 603 (e.g., submitted by one of the users of the digital platform 210 in the past), (b) vector representations of training documents from a set 616 of training digital documents that have been identified in the search index database 206 as satisfying the training search query 603, such as a vector representation 617 of a training document for a training digital document (not separately indicated), (c) a label 619 characterizing the degree of relevance of the training digital document represented by the vector representation 617 of the training document to the training search query 603, and (d) a vector representation 615 of training phrases for the plurality of training 605 candidate phrases lexically related to training search query 603.

[0135] As indicated above, the server 202 is capable of generating a vector representation 613 of the training search query and a vector representation 615 of the training phrases using the first machine learning model 502 in an online mode, that is, in real time. At the same time, the server 202 is capable of receiving vector representations of training documents for a set 616 of training digital documents from the search index database 206, which the server 202 can generate using the second machine learning model 504, before generating the vector representations 613, 615 of the training search query and phrases.

[0136] In addition, according to some non-limiting embodiments of the present technology, the label 619 may be provided by a human evaluator who, for example, may be associated with a crowdsourcing platform (not shown), such as the Amazon™ Mechanical Turk™ crowdsourcing platform or the Yandex™ Toloka™ crowdsourcing platform, to which the server 202 is connected via the communication network 208. For example, such a human evaluator may receive, via the interface of the crowdsourcing platform, an instruction for generating labels for assigning the label 619 to the training digital document represented by the vector representation 617 of the training document. According to some non-limiting embodiments of the present technology, the label 619 comprises a ranking parameter value that can be selected, for example, in the range from 0 to 1, where 0 means the irrelevance of the training digital document to the training search query 603, and 1 means the relevance of the training digital document to the training search query 603. For example, the corresponding label 619 can take values of 0.1, 0.25, 0.37, etc.

[0137] In some non-limiting embodiments of the present technology, the label 619 for the training digital document may be automatically generated by the server 202, for example, using an additional machine learning model trained to determine the degree of relevance of the training digital document to the training search query 603 based on high-quality labels generated by human raters. For example, high-quality labels generated by human raters may not be provided by human raters from a crowdsourcing platform, but by experts who generate labels of higher quality than those of such human raters.

[0138] In addition, according to some non-limiting embodiments of the present technology, the server 202 is capable of finding a plurality of training 605 candidate phrases from a set 616 of training digital documents in the same manner as described above with reference to Fig. 4 with respect to selecting a plurality of candidate phrases 404 in an illustrative digital document 402. In addition, by transmitting the plurality of training 605 candidate phrases to the first machine learning model 502, the server 202 is capable of generating a vector representation 615 of the training phrases.

[0139] In some non-limiting embodiments of the present technology, to provide information about lexical relationships between the training search query 603 and the set 616 of training digital documents, instead of the vector representation 615 of training phrases, the server 202 may use raw text data of the training search query 603. In particular, in some non-limiting embodiments of the present technology, the server 202 is capable of including in the third plurality of training digital objects the number of occurrences of a particular keyword from the training search query 603 in the corresponding document from the set 616 of training digital documents. In other words, in such embodiments, each object of the third plurality of training digital objects comprises (a) a vector representation 613 of a training search query for a training search query 603, (b) vector representations of training documents for a set 616 of training digital documents, for example, a vector representation 617 of a training document for some training digital document, (c) a label 619 characterizing the degree of relevance of the training digital document represented by the vector representation 617 of the training document to the training search query 603, and (d) integer values characterizing the number of occurrences of keywords from the training search query 603 in the training digital document.

[0140] In some non-limiting embodiments of the present technology, the server 202 is configured to use raw text data from the set 616 of training digital documents to generate a third training data set in a different manner. In particular, in such alternative embodiments, instead of determining the frequency of occurrence of each full keyword from the training search query 603 in the training digital document, the server 202 is configured to (a) partition the keywords from the training search query 603 into N-grams, such as bigrams or trigrams, (b) determine the frequency of occurrence of each N-gram of the training search query in the training digital document, and (c) include in the third plurality of training digital objects integer values characterizing the frequency of occurrence of each N-gram from the training search query 603 in the training digital document from the set 616 of training digital documents.

[0141] In other words, in contrast to embodiments that are oriented toward using the vector representation 615 of training phrases to provide information about lexical relationships between training digital documents from the set 616 of training digital documents and the training search query 603, in some non-limiting embodiments of the present technology, the server 202 is capable of using “raw” integer values that characterize the frequencies of occurrence of either entire keywords or N-grams of the training search query 603 in each training digital document from the set 616 of training digital documents.

[0142] In addition, to train the combined machine learning model 602 based on the third training data set, the server 202 is capable of (a) at this training iteration transmitting a training digital object from the third plurality of training digital objects to the combined machine learning model 602, causing the combined machine learning model 602 to generate a current predicted value of the ranking parameter, and (b) minimizing the discrepancy between the (actual) value of the ranking parameter represented by the label 619 and the current predicted value generated by the combined machine learning model 602 at this training iteration by adjusting the weights of the nodes in the combined machine learning model 602.

[0143] For example, in some non-limiting embodiments of the present technology, the server 202 is capable of determining the discrepancy between the actual and the current predicted values of the ranking parameter using a loss function, such as a cross-entropy loss function. Obviously, in other non-limiting embodiments of the present technology, other embodiments of the loss function are possible, which may include, for example, among other things, a mean squared error loss function, a Huber loss function, a piecewise linear loss function, etc. However, there is no limitation on the method for minimizing the loss function by the server 202, and in some non-limiting embodiments of the present technology, it typically depends on the differentiability of the loss function. For example, if the loss function is continuously differentiable, a gradient descent algorithm, a Newton optimization algorithm, etc., among other things, can be used to minimize it. In those embodiments where the loss function is not differentiable, the server 202 can use, for example, direct algorithms, stochastic algorithms, and/or population algorithms to minimize it.

[0144] Thus, feeding the combined machine learning model 602 a vector representation 617 of a training document as part of an object from the third plurality of training digital objects allows the combined machine learning model 602 to determine how a digital document from the set 214 of digital documents is semantically (i.e., by meaning) related to the search query 212. On the other hand, feeding the combined machine learning model 602 a vector representation 615 of training phrases for the plurality of training 605 candidate phrases lexically related to the training search query 603 allows the combined machine learning model 602 to determine how a digital document is lexically (i.e., by linguistic forms) related to the search query 212.

[0145] It is evident that the third plurality of training digital objects may comprise thousands, tens of thousands, or even hundreds of thousands of training digital objects generated based on various training search queries obtained, for example, from historical data on user interactions with the digital platform 210. Thus, by feeding all of the training digital objects to the combined machine learning model and minimizing the discrepancy between the actual and predicted values of the ranking parameter, the server 202 is able to train the combined machine learning model 602 to generate values of the ranking parameter for digital documents selected for search queries in use mode, such as the set 214 of digital documents for the search query 212, taking into account both semantic and lexical relationships between them. The following describes how the server 202 can use the combined machine learning model 602 after completing the training process described above.

Process of use

[0146] Fig. 8 is a schematic representation of a process for using a combined machine learning model 602 to rank digital documents in accordance with some non-limiting embodiments of the present technology.

[0147] In particular, according to some non-limiting embodiments of the present technology, in order to use the combined machine learning model 602 for ranking digital documents, the server 202 is configured to (a) receive a search query 212 sent by a user 216 to the digital platform 210, (b) generate, using the first machine learning model 502, a vector representation 512 of the search query as described above, (c) find, using, for example, a mathematical model, in the search index database 206 a set 214 of digital documents that satisfy the search query 212 and their corresponding vector representations of documents, such as a vector representation 516 of a document for an illustrative digital document 402, (d) find in the set 214 of digital documents a plurality 404 of candidate phrases lexically related to the search query 212, and (e) generate, using the first machine learning model 502, vector representation of 514 phrases characterizing a set of 404 candidate phrases.

[0148] As noted above, in some non-limiting embodiments of the present technology, the server 202 is capable of generating vector representations of documents for individual documents from a plurality of digital documents on the digital platform 210 using the second machine learning model 504, prior to running the combined machine learning model 602 in use mode and storing such vector representations of documents in the search index database 206 for subsequent use. Thus, after the server 202 selects a set of digital documents for ranking, for example, a set 214 of digital documents, the server 202 is capable of receiving corresponding vector representations of the documents without the need to generate them in real time. This allows saving the computing resources of the server 202 during the operation of the combined machine learning model 602, which increases the efficiency of generating the ranked set 220 of digital documents.

[0149] In addition, the server 202 is capable of transmitting (a) the vector representation 512 of the search query, (b) the vector representation 514 of the phrases, and (c) the vector representations of the documents for the set 214 of digital documents to the combined machine learning model 602, causing the combined machine learning model 602 to generate ranking parameter values 702 for the set 214 of digital documents. In addition, the server 202 is capable of ranking the set 214 of digital documents depending on the ranking parameter values 702, generating a ranked set 220 of digital documents. Finally, the server 202 is capable of transmitting the ranked set 220 of digital documents to the electronic device 204 for presentation to the user 216.

[0150] As indicated above in the description of the training process, in some non-limiting embodiments of the present technology for providing information about lexical relationships between the search query 212 and the set 214 of digital documents, instead of the vector representation 514 of phrases for the plurality 404 candidate phrases, the server 202 is capable of generating and further transmitting to the combined machine learning model 602 frequency vectors in use mode, each of which contains values characterizing the number of occurrences of the keywords of the search query 212 in the illustrative digital document 402. For example, as shown in FIG. 4, where the search query 212 with the text “Macaron Recipe Pierre Herme” is presented, the frequency vector in the use mode for the illustrative digital document 402 may contain the following values: [5, 0, 2], since (a) linguistic forms of the keyword “Macaron” occur in the illustrative digital document 402 five times, (b) any forms of the keyword “Recipe” are absent in the document, and (c) different forms of the keywords “Pierre Herme” occur twice in the illustrative digital document 402.

[0151] Thus, transmitting the vector representation 514 of phrases or frequency vectors in the mode of use together with the vector representations of documents for the set 214 of digital documents allows the combined machine learning model 602 to generate the values 702 of the ranking parameter, taking into account not only the semantic relationships between individual documents from the set 214 of digital documents and the search query 212, but also the lexical relationships between them. In other words, transmitting the vector representation 514 of phrases to the combined machine learning model 602 allows the combined machine learning model 602 to be reoriented from determining the degree of relevance of each of the set 214 of digital documents to the search query 212 solely on the basis of the semantic relationships between them to determining the degree of relevance of each of the set 214 of digital documents to the search query 212 on the basis of both semantic and lexical relationships between them.

[0152] This allows the set 214 of digital documents to be ranked such that digital documents that are related to the search query 212 both semantically and lexically are assigned a higher ranking parameter value than those that are related to the search query 212 only semantically or only lexically. As a result, the degree of satisfaction of the user 216 with the ranked set 220 of digital documents can be increased.

Way

[0153] Taking into account the above-described architecture and the examples given, it is possible to implement a method for ranking digital documents, such as the set 214 of digital documents. Fig. 9 shows a flow chart of a method 800 in accordance with some non-limiting embodiments of the present technology. The method 800 can be implemented by the server 202.

Step 802: Receive the search query submitted by the user to the digital platform.

[0154] The method 800 begins with step 802, in which the server 202 may receive a search query 212 that the user 216 of the electronic device 204 sent to the digital platform 210.

[0155] The next step in method 800 is step 804.

Step 804: Forming the first vector representation characterizing the search query.

[0156] In step 804, in accordance with some non-limiting embodiments of the present technology, the server 202 may generate a vector representation 512 of the search query for the search query 212 received in step 802. To do this, in accordance with some non-limiting embodiments of the present technology, the server 202 may use a first machine learning model 502 trained to generate vector representations of input phrases, as described above with reference to Fig. 5.

[0157] The next step in method 800 is step 806.

Step 806: finding in the search index of the digital platform a plurality of candidate digital documents that satisfy the search query.

[0158] In step 806, in accordance with some non-limiting embodiments of the present technology, the server 202 may find in the database 206 of the search index of the digital platform 210 a set 214 of digital documents that satisfy the search query 212. In this regard, as described above, the server 202 may apply a mathematical model that may include, for example, the Okapi BM25 ranking function.

[0159] The next step in method 800 is step 808.

Step 808: obtaining for each candidate digital document from the plurality of candidate digital documents a second vector representation generated prior to receiving the search query.

[0160] At step 808, in accordance with some non-limiting embodiments of the present technology, the server 202 may obtain, for each digital document of the set 214 of digital documents, a vector representation of the document, such as a vector representation 516 of the document for an illustrative digital document 402 of the set 214 of digital documents.

[0161] As noted above, prior to receiving the search query 212 in step 802, in accordance with some non-limiting embodiments of the present technology, the server 202 may generate a vector representation 516 of the document using a second machine learning model 504 trained to generate vector representations of input digital documents, as described above with reference to Fig. 6. In addition, the server 202 may store the vector representation 516 of the document in the search index database 206 in association with the illustrative digital document 402 for subsequent use in ranking digital documents.

[0162] The next step in method 800 is step 810.

Step 810: finding for each candidate document of the plurality of candidate digital documents at least one candidate phrase lexically related to at least one keyword of the search query.

[0163] At step 810, in accordance with some non-limiting embodiments of the present technology, the server 202 may find in the digital document from the set 214 of digital documents at least one candidate phrase lexically related to the search query 212, for example, a plurality 404 of candidate phrases found in the illustrative digital document 402, as described above with reference to Fig. 4.

[0164] In particular, in some non-limiting embodiments of the present technology, the server 202 may (a) generate, for each phrase of the illustrative digital document 402, a vector comprising a numeric representation of the phrase in a vector representation space, (b) determine, using such vectors, for each phrase of the illustrative digital document 402 a value of a metric of closeness to the search query 212, (c) rank the phrases of the illustrative digital document 402 depending on the values of the metric of closeness, and (d) select for inclusion in the set of candidate phrases N priority phrases lexically related to the search query 212. As noted above, in some non-limiting embodiments of the present technology, the closeness metric may characterize the distance between the vector representing the phrase of the illustrative digital document 402 and the vector representing the search query 212 in the space of vector representations. In some non-limiting embodiments of the present technology, the proximity metric may be a cosine proximity metric.

[0165] The next step in method 800 is step 812.

Step 812: forming a third vector representation characterizing at least one candidate phrase.

[0166] In step 812, in accordance with some non-limiting embodiments of the present technology, the server 202 may generate a vector representation 514 of phrases for the plurality 404 candidate phrases lexically related to the search query 212. To do this, the server 202 may use a first machine learning model 502 trained to generate vector representations of input phrases, as described above with reference to Fig. 5.

[0167] In other non-limiting embodiments, instead of the vector representation 514 of phrases, the server 202 is configured to generate frequency vectors in use mode, each of which contains values characterizing the number of occurrences of the keywords of the search query 212 in the illustrative digital document 402, as described above with reference to Figs. 4 and 8.

[0168] The next step in method 800 is step 814.

Step 814: determining, based on the first, second and third vector representations, a ranking parameter value for each candidate document from the plurality of candidate digital documents.

[0169] At step 814, based on the vector representation 512 of the search query, the vector representations of the documents for each document from the set 214 of digital documents, and the vector representation 514 of phrases (or vectors of frequency of occurrence of keywords from the search query 212 in use mode) for the plurality 404 candidate phrases lexically related to the search query 202, the server 202 may rank the set 214 of digital documents taking into account their lexical and semantic relationships with the search query 212.

[0170] To this end, in accordance with some non-limiting embodiments of the present technology, the server 202 may use a combined machine learning model 602 trained as described above with reference to Fig. 7. In particular, as described above with reference to Fig. 8, the server 202 may feed (a) a vector representation 512 of the search query, (b) a vector representation 514 of the phrases, and (c) vector representations of the documents for the set 214 of digital documents to the combined machine learning model 602, causing the combined machine learning model 602 to generate ranking parameter values 702 for the set 214 of digital documents. As indicated above, the ranking parameter value characterizes the degree of relevance of the digital document to the search query 212.

Step 816: Ranking a plurality of candidate digital documents based on the ranking parameter values associated with them.

[0171] Finally, in step 816, in accordance with some non-limiting embodiments of the present technology, the server 202 may rank the set 214 of digital documents depending on the ranking parameter values 702 determined in step 814, forming a ranked set 220 of digital documents. In addition, the server 202 may transmit the ranked set 220 of digital documents to the electronic device 204 for presentation to the user 216.

[0172] This completes the implementation of method 800.

[0173] Thus, some embodiments of the method 800 make it possible to input into the combined machine learning model 602 information characterizing lexical relationships between digital documents and the search query 212, which further allows ranking the set 214 of digital documents so that digital documents related to the search query 212 both semantically and lexically are assigned a higher ranking parameter value than digital documents related to the search query 212 only semantically or only lexically. This allows increasing the degree of satisfaction of the user 216 with the ranked set 220 of digital documents.

[0174] It should also be understood that the embodiments presented herein are described with reference to specific features and structures, but various modifications and combinations thereof are possible without departing from the spirit of this description. For example, various optimizations applicable to neural networks, including transformers and/or BERT, can be applied to the described technology. In addition, optimizations that speed up the determination of relevance in use mode can be used. For example, in some embodiments, the transformer model can be divided so as to ensure that some transformer blocks are distributed between the tasks of query processing and document processing, allowing document representations to be pre-computed offline and stored in a document search index. The description and, accordingly, the drawings should be considered only as illustrating the presented embodiments or implementations and their principles defined by the appended claims, while it is intended that they cover all modifications, variations, combinations and equivalents within the spirit and scope of the present invention.

Claims (45)

1. A computer method for ranking digital documents on a digital platform, which provides for: - receiving a search query sent by a user to a digital platform; - formation of the first vector representation characterizing the search query; - finding in the search index of the digital platform a set of digital documents-candidates that satisfy the search query; - obtaining for each digital document candidate from a set of digital documents candidates a second vector representation formed before the search query was received; - finding for each candidate document from a plurality of candidate digital documents at least one candidate phrase lexically related to at least one keyword of the search query; - formation of a third vector representation characterizing at least one candidate phrase; - determining, on the basis of the first, second and third vector representations, for each candidate document from the plurality of candidate digital documents, the value of a ranking parameter characterizing the relevance of this candidate document from the plurality of candidate digital documents to the search query; and - ranking a set of candidate digital documents based on their associated ranking parameter values. 2. The method according to claim 1, wherein finding a plurality of candidate digital documents involves using a ranking function. 3. The method according to claim 2, wherein the ranking function is the Okapi BM25 ranking function. 4. The method according to claim 1, wherein finding at least one candidate phrase comprises: - formation of a vector representation of the phrase for each phrase of a candidate document from a set of digital candidate documents; - determining in the space of vector representations the value of the distance between the first vector representation characterizing the search query and the vector representation of the phrase; - ranking phrases from a set of candidate digital documents based on distance values associated with the vector representations of the phrases to form a ranked list of phrases for a search query; and - selection of a specified number of priority phrases from a ranked list of phrases. 5. The method according to claim 4, wherein the formation of the vector representation of the phrase involves determining for each keyword of the search query the value of the ratio of the frequency of the keyword to the inverse frequency of the document in the candidate document from a plurality of digital candidate documents. 6. The method according to paragraph 4, in which the formation of a vector representation of a phrase involves applying to it an algorithm for vector representation of text. 7. The method according to claim 6, wherein the text vector representation algorithm is the FastText word vector representation algorithm. 8. The method of claim 1, wherein determining the value of the ranking parameter comprises transmitting the first, second and third vector representations to a combined machine learning model trained to determine the value of the ranking parameter for each document of the plurality of digital documents based on vector representations of (a) a search query used to find the plurality of digital documents, (b) each document of the plurality of digital documents that satisfy the search query, and (c) at least one candidate phrase identified in the plurality of digital documents as lexically related to at least one keyword of the search query. 9. The method according to paragraph 8, further comprising training the combined machine learning model by: - forming a training data set containing a plurality of training digital objects, each of which contains (a) a training vector representation of a training search query, (b) training vector representations of a plurality of candidate training digital documents satisfying the training search query, (c) a training vector representation of at least one candidate training phrase found in the plurality of training digital documents, lexically associated with at least one keyword from the training search query, and (d) a label for each training document from the plurality of training digital documents, characterizing the degree of relevance of this training document from the plurality of training digital documents to the training search query; - feeding a set of training digital objects into a unified machine learning model; and - minimizing the discrepancy between the current training prediction generated by the combined machine learning model and the label at each training iteration. 10. The method according to claim 9, wherein the label is generated by a machine learning model, pre-trained on the basis of labels generated by a human evaluator, to determine the degree of relevance of a digital document to a search query. 11. The method of claim 8, wherein the combined machine learning model comprises a deep machine learning model for determining semantic similarity. 12. The method according to claim 1, wherein obtaining the second vector representation involves obtaining the second vector representation from a second machine learning model trained to generate vector representations of input digital documents. 13. The method according to claim 12, further comprising training the second machine learning model by transmitting to it a plurality of digital documents from the search index of the digital platform. 14. The method according to claim 1, further comprising reducing the volume of representations for each of the first, second and third vector representations before determining the value of the ranking parameter. 15. The method according to claim 14, wherein the formation of the first, second and third vector representations involves the use of a transformer-based machine learning model, and the reduction in the volume of representations involves sequentially truncating the output data of each intermediate layer of the transformer-based machine learning model to a given length of each of the first, second and third vector representations. 16. The method according to claim 1, wherein the formation of the first and third vector representations is carried out independently of the formation of the second vector representation. 17. The method according to claim 1, wherein the formation of the first vector representation characterizing the search query and the third vector representation characterizing at least one candidate phrase involves using a machine learning model trained to form vector representations of input phrases. 18. The method according to paragraph 17, further comprising training the machine learning model by: - forming a training data set containing a plurality of training digital objects, each of which contains a training search query and a plurality of training candidate phrases found in a plurality of training digital documents that satisfy the training search query and are lexically associated with at least one keyword from the training search query; and - transferring a set of training digital objects to a machine learning model. 19. The method of claim 1, wherein in the process of finding at least one candidate phrase lexically related to at least one keyword from the search query for each candidate document from the plurality of candidate digital documents, the method further comprises using an unprocessed version of this candidate document from the plurality of candidate digital documents. 20. A server for ranking digital documents on a digital platform, comprising at least one processor and at least one long-term machine-readable memory storing executable commands, the execution of which by at least one processor on the server ensures: - receiving a search query sent by a user to a digital platform; - formation of the first vector representation characterizing the search query; - finding in the search index of the digital platform a set of digital documents-candidates that satisfy the search query; - obtaining for each digital document candidate from a set of digital documents candidates a second vector representation formed before the search query was received; - finding for each candidate document from a plurality of candidate digital documents at least one candidate phrase lexically related to at least one keyword of the search query; - formation of a third vector representation characterizing at least one candidate phrase; - determining, on the basis of the first, second and third vector representations, for each candidate document from the plurality of candidate digital documents, the value of a ranking parameter characterizing the relevance of this candidate document from the plurality of candidate digital documents to the search query; and - ranking a set of candidate digital documents based on their associated ranking parameter values.