WO2021178020A1 - Systems and methods for interpreting natural language search queries - Google Patents
Systems and methods for interpreting natural language search queries Download PDFInfo
- Publication number
- WO2021178020A1 WO2021178020A1 PCT/US2020/065307 US2020065307W WO2021178020A1 WO 2021178020 A1 WO2021178020 A1 WO 2021178020A1 US 2020065307 W US2020065307 W US 2020065307W WO 2021178020 A1 WO2021178020 A1 WO 2021178020A1
- Authority
- WO
- WIPO (PCT)
- Prior art keywords
- word
- term
- natural language
- terms
- search query
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Ceased
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/30—Information retrieval; Database structures therefor; File system structures therefor of unstructured textual data
- G06F16/33—Querying
- G06F16/3331—Query processing
- G06F16/3332—Query translation
- G06F16/3334—Selection or weighting of terms from queries, including natural language queries
-
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/30—Information retrieval; Database structures therefor; File system structures therefor of unstructured textual data
- G06F16/33—Querying
- G06F16/3331—Query processing
- G06F16/334—Query execution
- G06F16/3344—Query execution using natural language analysis
Definitions
- This disclosure relates to processing search queries and, more particularly, interpreting natural language search queries.
- Natural language search queries are normally processed by simply applying a fdter, such as content type, to the query, and returning any results that match the query within that filter.
- a fdter such as content type
- many natural language search queries include words that are contextually relevant to the search query but are ignored by the processing systems because they are not associated with any keyword or genre by themselves. Thus, the results of the search do not provide content for which the user was searching.
- a natural language search query is received, either as a voice input, a text input, or a transcribed voice-to-text input, and a plurality of terms in the natural language search query are identified. Each term is associated with a respective part of speech, and a frequency of occurrence of each term in content metadata is determined. A relevance of each term is then determined based on its respective part of speech and frequency.
- the natural language search query is then interpreted based on the importance or relevance of each term. Search results are retrieved based on the interpreted search query, and the results are then generated for display.
- a search query for “poison movies” may be received, and the user may intend to search for movies in which a character is poisoned, or in which poison is a major plot point. While the word “movies” may normally be identified as a keyword indicating the desired type of content, the word “poison” is not associated with any genre, actor, or other identifying information that could narrow a search for movies to those that are about poison or have poison as a major plot point or plot device. However, the system processes the word “poison” to identify that it is a noun and determines its frequency of occurrence to be low.
- the word “poison” is marked as a keyword and the search query is interpreted as a query for movies whose metadata contain the word “poison,” such as in a plot summary.
- Other examples may include searches for “movies that will make me cry” or “videos where a boy falls from his bike.” These searches identify the primary type of content (“movies” or “videos”) but the remaining words do not match up with any preexisting content identifiers that would allow for a meaningful search. Identifying terms such as “make me cry” as uncommon search terms results in a determination that the term is relevant to the search query.
- the natural language interpreter may also be trained using a training data set compiled from previous natural language searches that have been annotated.
- a frequency of occurrence for each term in the training data set is determined in relation to the entire training data set.
- a relational data structure is generated that associates each term in the training data with its respective frequency. Any term that has a frequency below a threshold frequency is then added to a list of relevant words.
- a natural language search query is received, a plurality of terms in the natural language search query are identified and compared with the list of relevant words. If any term of the natural language search query is included in the relevant words list, that term is identified as a keyword.
- the natural language search query is then interpreted based on any identified keywords. As above, search results are retrieved based on the interpreted search query and generated for display to the user.
- the training data may include a total of ten thousand words, and the threshold frequency may be one percent.
- the threshold frequency may be one percent.
- the relevance of the word “poison” can be determined by checking if the word “poison” appears on the relevant words list. If the word “poison” appears on the relevant words list, then it is identified as a keyword, and the natural language search query is interpreted as a query for movies whose metadata contain the word “poison,” such as in a plot summary.
- a query type can be determined based on the structure of the natural language search query.
- the natural language search query is processed to determine whether it is a complete sentence. If so, a number of terms are identified in the natural language search query and each term is associated with a part of speech.
- a type of query is determined, and the natural language search query is interpreted in the context of the query type based on the parts of speech of each term in the natural language search query. For example, the sentence “Show me movies of Tom Cruise where he is flying” may be received as a natural language search query.
- the search query is identified as a complete sentence, and a sequence labeling algorithm such as Hidden Markov Model or Conditional Random Field identifies each part of the sentence.
- a sequence labeling algorithm such as Hidden Markov Model or Conditional Random Field identifies each part of the sentence.
- Each part of the sentence is labelled with a part of speech, and the term “show me movies” is used to identify that the natural language search query is a query for multimedia content, specifically movies.
- “Tom Cruise” is identified as a proper noun, and “where” is identified as a filter trigger word.
- He is identified as a pronoun referring to Tom Cruise as the previously identified proper noun subject of the sentence.
- Is is a stop word, which indicates that the words which follow it are the parameters of the previously triggered filter.
- “flying” is identified as a verb and the parameter for the filter to be applied to the search for movies starring Tom Cruise.
- the natural language search query is interpreted to be a search for movies starring the actor Tom Cruise and containing scenes in which he is flying. Search results are retrieved based on the interpretation, and the results are generated for display to the user.
- the natural language search query can also be interpreted through use of machine learning, such as using one or more neural networks. After identifying a number of terms in the natural language search query, a vector is generated for each term describing a relationship between each term and a plurality of other terms. Each vector is then input into a trained neural network that generates an output based on the input vectors. The natural language search query is then interpreted based on the output of the neural network. For example, for the query “Movies of Tom Cruise where he is flying,” a vector for “Tom Cruise” may be generated that represents degrees of connection between Tom Cruise and other terms, such as co-stars, movie titles, or other terms.
- a vector for “flying” may be generated that represents degrees of connection between “flying” and other terms, such as “airplane,” “helicopter,” “jet,” and “falling.” These vectors may be input into a neural network which processes each input vector and outputs an interpretation of the search query. For example, the vector for Tom Cruise may indicate a connection with Val Kilmer, and both Tom Cruise and Val Kilmer have a connection to “flying.” The neural network may then output an interpretation of the search query that focuses on the movie “Top Gun” starring Tom Cruise and Val Kilmer.
- Other examples may include searches for “movies that will make me cry” or “videos where a boy falls from his bike.” These searches identify the primary type of content (“movies” or “videos”) but the remaining words do not match up with any preexisting content identifiers that would allow for a meaningful search. Identifying terms such as “make me cry” as uncommon search terms results in a determination that the term is relevant to the search query, and a vector connecting this term to a particular genre of movie would result in an interpretation that accounts for the relevance of the term. Search results are then retrieved based on the interpreted search query and generated for display to the user.
- FIG. 1 shows a first system for interpreting a natural language search query, in accordance with some embodiments of the disclosure
- FIG. 2 shows a second system for interpreting a natural language search query, in accordance with some embodiments of the disclosure
- FIG. 3 shows a third system for interpreting a natural language search query, in accordance with some embodiments of the disclosure
- FIG. 4 shows a fourth system for interpreting a natural language search query in accordance with some embodiments of the disclosure
- FIG. 5 is a block diagram show components and data flow therebetween of a device for interpreting a natural language search query, in accordance with some embodiments of the disclosure
- FIG. 6 is a block diagram showing components and data flow therebetween of a device for enabling interpretation of a natural language search query, in accordance with some embodiments of the disclosure
- FIG. 7 is a flowchart representing a first process for interpreting a natural language search query, in accordance with some embodiments of the disclosure.
- FIG. 8 is a flowchart representing a process for identifying a plurality of terms in a natural language search query, in accordance with some embodiments of the disclosure.
- FIG. 9 is a flowchart representing a process for determining a frequency with which terms in a natural language search query occur in metadata, in accordance with some embodiments of the disclosure;
- FIG. 10 is a flowchart representing a process for determining a relevance of terms in a natural language search query, in accordance with some embodiments of the disclosure.
- FIG. 11 is a flowchart representing a second process for interpreting a natural language search query, in accordance with some embodiments of the disclosure;
- FIG. 12 is a flowchart representing a process for determining a frequency with which terms in a natural language search query occur in a training data set, in accordance with some embodiments of the disclosure
- FIG. 13 is a flowchart representing a process for generating a relational data structure associating terms of a natural language search query with respective frequencies of occurrence, in accordance with some embodiments of the disclosure
- FIG. 14 is a flowchart representing a third process for interpreting a natural language search query, in accordance with some embodiments of the disclosure.
- FIG. 15 is a flowchart representing a fourth process for interpreting a natural language search query, in accordance with some embodiments of the disclosure.
- FIG. 16 is a flowchart representing a process for generating vectors for terms in a natural language search query for input into a neural network, in accordance with some embodiments of the disclosure.
- FIG. 1 shows a first system for interpreting a natural language search query, in accordance with some embodiments of the disclosure.
- Natural language search query 100 may be received from a user or from an input device.
- Voice-user interface 102 may capture spoken words representing the natural language search uttered by the user and transmit a digital representation of the spoken natural language search query to a user device 104.
- User device 104 processes the words of the natural language search query and generates interpretation 106 of the natural language search query.
- User device 104 may transmit the interpreted query, via a communications network 108, to server 110, which provides search results back to user device 104.
- User device 104 may also request or retrieve metadata describing content items from server 110 and use it to determine the relevance of each word or term of the natural language search query.
- Interpretation 106 of the natural language search query may be based on the relevance of each word or term.
- natural language search query 100 may be the words “poison movies.”
- User device 104 determines, based on the metadata, that the word “poison” is an infrequent word, and must therefore be relevant to the query.
- User device 104 also determines that the word “movies” is a type of content for which a search should be performed. Based on this information, user device 104 interprets the natural language search query and generates a corresponding query in a format that can be understood by server 110, such as an SQL “SELECT” command.
- the command shown in interpretation 106 is an SQL command to select all records from a “movies” table of a content database where any of a summary, a title, or a plot synopsis contains the word “poison.”
- FIG. 2 shows a second system for interpreting a natural language search query, in accordance with some embodiments of the disclosure.
- a training data set 200 may be provided to, or accessed by, server 202.
- Server 202 may use the training data set to determine a list of relevant words 204.
- a natural language search query 206 is received, via voice-user interface 208, at user device 210.
- User device 210 may request or retrieve, via communications network 212, relevant words list 204 from server 202.
- User device 210 may compare each word or term of natural language search query 206 to relevant words list 204 to determine relevant words or terms of natural language search query 206.
- the relevant words of natural language search query 206 are identified as keywords, and user device 210 interprets natural language search query 206 based on the identified keywords.
- natural language search query 206 may be the word “poison movies.”
- server 202 determines that “poison” is an infrequent word and adds it to the relevant words list 204.
- User device 210 compares the word “poison” to the relevant words list 204 received from server 202 and finds that the word “poison” is included therein. Based on this, user device 210 identifies “poison” as a keyword. As above, user device 210 identifies the word “movies” as a type of content for which a query should be performed, and generates interpretation 214, which may be a SQL command as described above.
- FIG. 3 shows a third system for interpreting a natural language search query, in accordance with some embodiments of the disclosure.
- Natural language search query 300 is received by user device 302.
- natural language search query 300 may be the sentence “Show me movies where Tom Cruise is flying.”
- the user device splits natural language search query 300 into a plurality of terms 304a-304f using natural language processing.
- User device 302 associates each term with a part of speech.
- Term 304a (“show me”) is identified as a query trigger; term 304b (“movies”) is identified as a query type; term 304c (“where”) is identified as a filter trigger, indicating that at least one term which follows the filter trigger should be applied as a filter to the query; term 304d (“Tom Cruise”) is identified as a proper noun and/or a person’s name; term 304e (“is”) is identified as a stop word, which is determined based on context to be an additional filter trigger for an additional filter parameter; and term 304f (“flying”) is identified as a verb and as the second filter parameter.
- user device 302 Based on these associations, user device 302 generates interpretation 306, such as an SQL command, in which each of terms 304a-304f are included as corresponding portions 308a-308f of the SQL command.
- portion 308a which initializes a search query, corresponds to term 304a
- portion 308b which identifies what records to select and from which table the records should be selected, corresponds to term 304b
- portion 308c which initializes a search filter, corresponds to term 304c
- portion 308d which represents a first filter parameter, corresponds to term 304d
- portion 308e which indicates an additional filter parameter, corresponds to term 304e
- portion 308f which represents the second filter parameter, corresponds to term 304f.
- FIG. 4 shows a fourth system for interpreting a natural language search query in accordance with some embodiments of the disclosure.
- Natural language search query 400 is received by a user device (not shown).
- the user device generates vectors for terms of the natural language search query.
- the natural language search query may be the sentence “Show me movies where Tom Cruise is flying.”
- the user device may identify “Tom Cruise” and “flying” as relevant word of the natural language search query.
- the user device then generates a vector 402, describing connections between “Tom Cruise” and other terms and the distance between “Tom Cruise” and each of the other terms, and vector 404, describing connections between “flying” and other terms and the distance between “flying” and each of the other terms.
- Vectors 402 and 404 are then inputted into trained neural network 406 which processes the terms “Tom Cruise” and “flying” based on the input vectors and outputs an interpretation of each term, which is used to generate interpretation 408.
- FIG. 5 is a block diagram show components and data flow therebetween of a device for interpreting a natural language search query, in accordance with some embodiments of the disclosure.
- a natural language search query may be received as voice input 500 using voice-user interface 502.
- Voice-user interface 502 may include a microphone or other audio capture device capable of capturing raw audio data and may convert raw audio data into a digital representation of voice input 500.
- Voice-user interface 502 may also include a data interface, such as a network connection using ethernet or Wi-Fi, a Bluetooth connection, or any other suitable data interface for receiving digital audio from another input device.
- Voice-user interface 502 transmits 504 the digital representation of the voice input to control circuitry 506, where it is received using natural language processing circuitry 508.
- Natural language processing circuitry may transcribe the audio representing the natural language search query to generate a corresponding text string or may process the audio data directly.
- a natural language search query may be received as text input 510 using text-user interface 512, which may include similar data interfaces to those described above in connection with voice-user interface 502.
- Text-user interface 512 transmits 514 the text input 510 to control circuitry 506, where it is received using natural language processing circuitry 508.
- Control circuitry 506 may be based on any suitable processing circuitry and comprises control circuits and memory circuits, which may be disposed on a single integrated circuit or may be discrete components.
- processing circuitry should be understood to mean circuitry based on one or more microprocessors, microcontrollers, digital signal processors, programmable logic devices, field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), etc., and may include a multi-core processor (e.g ., dual-core, quad-core, hexa-core, or any suitable number of cores).
- processing circuitry may be distributed across multiple separate processors or processing units, for example, multiple of the same type of processing units (e.g., two Intel Core i7 processors) or multiple different processors (e.g., an Intel Core i5 processor and an Intel Core i7 processor).
- multiple of the same type of processing units e.g., two Intel Core i7 processors
- multiple different processors e.g., an Intel Core i5 processor and an Intel Core i7 processor.
- Natural language processing circuitry 508 identifies a plurality of terms in the natural language search query. For example, natural language processing circuitry 508 may identify individual words in the natural language search query using spaces in text input 510 or pauses or periods of silence in voice input 500. Natural language processing circuitry 508 analyzes a first word and determines whether the first word can be part of a larger phrase.
- natural language processing circuitry 508 may request 516 a dictionary or other word list or phrase list from memory 518.
- Memory 518 may be any device for temporarily storing electronic data, such as random-access memory, hard drives, solid state devices, quantum storage devices, or any other suitable fixed or removable storage devices, and/or any combination of the same.
- natural language processing circuitry 508 Upon receiving 520 the dictionary or word list or phrase list from memory 520, natural language processing circuitry 508 determines if the first word can be followed by at least a second word. If so, natural language processing circuitry 508 analyzes the first word together the word immediately following the first word to determine if the two words together form a phrase. If so, the phrase is identified as a single term in the natural language search query. Otherwise, the first word alone is identified as a single term in the natural language search query.
- natural language processing circuitry 508 associates each term with a part of speech. Natural language processing circuitry 508 also determines a frequency with which each term occurs. For example, natural language processing circuitry 508 may request 522 metadata describing a plurality of content items from content metadata 524. Natural language processing circuitry 508 receives 526 the requested metadata and determines how many occurrences of each term there are in the metadata as a percentage of the total number of terms in the metadata. Using the part of speech and frequency of each term, natural language processing circuitry 508 determines a relevance for each term and interprets the natural language search query based on the relevance of each term.
- Natural language processing circuitry 508 transmits 528 the interpretation of the natural language search query to query construction circuitry 530 which constructs a search query corresponding to the natural language search query in a format that can be understood by, for example, a content database.
- Query construction circuitry 530 transmits 532 the constructed search query to transceiver circuitry 534, which transmits 536 the search query to, for example, content database 538.
- Transceiver circuitry 534 may be a network connection such as an Ethernet port, WiFi module, or any other data connection suitable for communicating with a remote server.
- Transceiver circuitry 534 then receives 540 search results from content database 538 and transmits 542 the search results to output circuitry 544.
- Output circuitry 544 then generates for display 546 the search results.
- Output circuitry 544 may be any suitable display driver or other graphic or video signal processing circuitry.
- a training data set is used to determine the relevance of each term.
- Training data 548 may be processing by control circuitry 506 or by a remote server to determine the relevance of a plurality of terms included in the training data.
- the resulting list of relevant terms is transmitted 550 to control circuitry 506, where it is received using transceiver circuitry 534.
- Transceiver circuitry 534 transmits 552 the received list of relevant terms to natural language processing circuitry 508 for use in determining the relevance of each term in the natural language search query.
- FIG. 6 is a block diagram showing components and data flow therebetween of a device for enabling interpretation of a natural language search query, in accordance with some embodiments of the disclosure.
- a natural language search query may be received as a voice input 600 using voice-user interface 602, which transmits 604 a digital representation of voice input 600 to control circuitry 606, where it is received by natural language processing circuitry 608, or as a text input 610 using text-user interface 612, which transmits 614 the natural language search query to control circuitry 606 where it is received by natural language processing circuitry 608.
- Control circuitry 606 may, like control circuitry 506, be based on any suitable processing circuitry and comprises control circuits and memory circuits, which may be disposed on a single integrated circuit or may be discrete components.
- processing circuitry should be understood to mean circuitry based on one or more microprocessors, microcontrollers, digital signal processors, programmable logic devices, field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), etc., and may include a multi-core processor (e.g ., dual-core, quad-core, hexa-core, or any suitable number of cores).
- processing circuitry may be distributed across multiple separate processors or processing units, for example, multiple of the same type of processing units (e.g., two Intel Core i7 processors) or multiple different processors (e.g., an Intel Core i5 processor and an Intel Core i7 processor).
- multiple of the same type of processing units e.g., two Intel Core i7 processors
- multiple different processors e.g., an Intel Core i5 processor and an Intel Core i7 processor.
- Natural language processing circuitry 608 after processing the natural language search query to identify a plurality of term therein and their relevance to the query, transmits 616 the relevant terms to vector generation circuitry 618.
- Vector generation circuitry 618 may access a knowledge graph or other data source to identify connections between each of the relevant terms and other terms, as well as the distance between each relevant term and the terms to which it is connected.
- Vector generation circuitry 618 transmits 620 the vectors for each relevant term to neural network 622 which may be trained used Hidden Markov Model or Conditional Random Field algorithms to process and interpret the relevant terms of the search query.
- Neural network 622 outputs interpretations of each relevant term and transmits 624 the interpretations to query construction circuitry 626.
- Query construction circuitry 626 uses the interpretations received from neural network 622, generates a corresponding search query in a format that can be understood by, for example, a content database.
- Query construction circuitry 626 transmits 628 the constructed query to transceiver circuitry 630, which in turn transmits 632 the constructed query to, for example, content database 634.
- transceiver circuitry 632 may be a network connection such as an Ethernet port, WiFi module, or any other data connection suitable for communicating with a remote server.
- Transceiver circuitry 630 then receives 636 search results from content database 634 and transmits 638 the search results to output circuitry 640.
- Output circuitry 640 then generates for display 642 the search results.
- Output circuitry 640 like output circuitry 544, may be any suitable display driver or other graphic or video signal processing circuitry.
- FIG. 7 is a flowchart representing a first illustrative process 700 for interpreting a natural language search query, in accordance with some embodiments of the disclosure.
- Process 700 may be implemented on control circuitry 506 or control circuitry 606.
- one or more actions of FIG. 7 may be incorporated into or combined with one or more actions of any other process or embodiment described herein.
- control circuitry receives a natural language search query.
- control circuitry 506 using natural language processing circuitry 508, identifies a plurality of terms in the natural language search query. This may be accomplished using methods described below in connection with FIG. 8.
- control circuitry 506 initializes a counter variable N, setting its value to one, and a variable T representing the total number of identified terms.
- control circuitry 506, using natural language circuitry 508, associates the N 111 term of the natural language search query with a part of speech.
- natural language processing circuitry 508 may access a dictionary or other word list or phrase list to identify a part of speech to which the N 111 term corresponds.
- control circuitry 506, using natural language circuitry 508 determines a frequency with which the N 111 term occurs in metadata describing content items. This may be accomplished using methods described below in connection with FIG. 9.
- natural language processing circuitry 508 determines a relevance for the N 111 term based on the part of speech and the frequency of the N* term. This may be accomplished using methods described below in connection with FIG. 10.
- control circuitry 506 determines whether N is equal to T, meaning that all terms of the natural language search query have been processed to determine their respective relevance. If N is not equal to T (“No” at 714), then, at 716, control circuitry 506 increments the value of N by one, and processing returns to step 708. If N is equal to T (“Yes” at 714), then, at 718, control circuitry 506 interprets the natural language search query based on the relevance of each term. This may be accomplished using methods described below in connection with FIG. 11.
- control circuitry 506 retrieves search results (e.g., from content database 540) based on the interpreted search query.
- control circuitry 506, using output circuitry 544 generates the search results for display to the user.
- FIG. 7 may be used with any other embodiment of this disclosure. In addition, the actions and descriptions described in relation to FIG. 7 may be done in suitable alternative orders or in parallel to further the purposes of this disclosure.
- FIG. 8 is a flowchart representing an illustrative process 800 for identifying a plurality of terms in a natural language search query, in accordance with some embodiments of the disclosure. Process 800 may be implemented on control circuitry 506 or control circuitry 606.
- control circuitry e.g ., control circuitry 506
- natural language processing circuitry 508 splits the natural language search query into a plurality of words.
- natural language processing circuitry 508 may identify pauses or periods of silence in audio data representing the natural language search query and split the audio data at each period of silence to separate the audio data into audio chunks, each representing a single word.
- natural language processing circuitry 508 may receive the natural language search query as text or may transcribe audio data into corresponding text. Natural language processing circuitry 508 may then split the text into individual words at every space.
- control circuitry 506 using natural language processing circuitry 508, determines whether a first word of the natural language search query can be part of a phrase. For example, natural language processing circuitry 508 may access a dictionary, word list, or phrase list, and identify any phrases that begin with the first word. If a phrase beginning with the first word is located (“Yes” at 804), then, at 806, natural language processing circuitry 508 determines whether the first word and a second word immediately following the first word form a phrase together. Natural language processing circuitry 508 may concatenate the first and second words to form a string representing a possible phrase formed by the first and second words together and compare the string to the dictionary, word list, or phrase list, as above.
- natural language processing circuitry 508 identifies the first and second word together as a single term. If the first and second words do not form a phrase together (“No” at 806) or if the first word cannot be part of a phrase at all (“No” at 804), then, at 810, natural language processing circuitry 508 identifies the first word as a single term.
- FIG. 9 is a flowchart representing an illustrative process 900 for determining a frequency with which terms in a natural language search query occur in metadata, in accordance with some embodiments of the disclosure.
- Process 900 may be implemented on control circuitry 506 or control circuitry 606.
- one or more actions of FIG. 9 may be incorporated into or combined with one or more actions of any other process or embodiment described herein.
- control circuitry retrieves metadata describing a plurality of content items.
- the metadata may be stored locally in memory 518 or may be stored at a remote server and retrieved using transceiver circuitry 534.
- control circuitry 506, using natural language processing circuitry 508, counts the number of words contained in the metadata.
- control circuitry 506 initializes a counter variable N, setting its value to one, and a variable T representing the total number of terms in the natural language search query.
- control circuitry 506, using natural language processing circuitry 508 determines the total number of occurrences of the N* term in the metadata.
- Control circuitry 506 then, at 910, calculates a percentage of the total number of words contained in the metadata corresponding to the total number of occurrences of the N 111 term. For example, if the metadata contains a total of ten thousand words, and the 111 term occurs one hundred times, control circuitry 506 will calculate that the N 111 term represents 0.1% of the words contained in the metadata. Thus, the N th term has a frequency of 0.001.
- control circuitry 506 determines whether N is equal to T, meaning that all the terms of the natural language search query have been processed to determine their respective frequency. If N is not equal to T (“No” at 912), then, at 914, control circuitry 506 increments the value of N by one, and processing returns to step 908. If N is equal to T (“Yes” at 912), then the process is complete.
- FIG. 10 is a flowchart representing an illustrative process 1000 for determining a relevance of terms in a natural language search query, in accordance with some embodiments of the disclosure.
- Process 1000 may be implemented on control circuitry 506 or control circuitry 606.
- one or more actions of FIG. 10 may be incorporated into or combined with one or more actions of any other process or embodiment described herein.
- control circuitry e.g., control circuitry 506 initializes a counter variable N, setting its value to one, and a variable T representing the total number of terms in the natural language search query.
- control circuitry 506 determines whether the frequency of the N 111 term meets or exceeds a threshold frequency. For example, a term having a high frequency, such as a frequency of 0 3, it may be a common term that is not relevant to the search query. However, if the frequency is low, such as 0.05, it may be an uncommon term and therefore may be relevant to the search query because the term would not otherwise normally appear in a search query.
- control circuitry 506 determines that the N 111 term is not relevant. However, if the frequency of the N 111 term does not exceed the threshold frequency (“No” at 1004), indicating that the 111 term is relatively uncommon, then, at 1008, control circuitry 506 determines a relevance factor for the N 111 term. For example, control circuitry 506 may divide the frequency of the N 111 term by the threshold frequency to determine a relevance factor. For example, if the frequency of the N* term is 0.05 and the threshold frequency is 0.25, then the relevance factor for the N 111 term is calculated to be 2.
- control circuitry 506 applies a weighting factor to the relevance factor based on the part of speech of the N 111 term. For example, a stop word or a filter trigger word may be less relevant to the search query than a proper noun or a verb.
- a weighting factor is used to adjust the overall relevance of the N lh term based on its part of speech.
- control circuitry 506 determines whether N is equal to T, meaning that all terms of the natural language search query have been processed to determine their respective relevance. If N is not equal to T (“No” at 1012), then, at 1014, control circuitry 506 increments the value of N by one, and processing returns to step 1004. If N is equal to T (“Yes” at 1012), then the process is complete.
- FIG. 11 is a flowchart representing a second illustrative process 1100 for interpreting a natural language search query, in accordance with some embodiments of the disclosure.
- Process 1100 may be implemented on control circuitry 506 or control circuitry 606.
- one or more actions of FIG. 11 may be incorporated into or combined with one or more actions of any other process or embodiment described herein.
- control circuitry e.g ., control circuitry 506 accesses training data comprising a first plurality of terms.
- the training data may comprise a set of natural language search queries that have been previously received and manually annotated.
- control circuitry 506 generates a relational data structure for associating terms with respective frequencies of occurrence in the training data.
- control circuitry 506 initializes a counter variable N, setting its value to one, a variable T t representing the total number of terms in the training data, at a data set ⁇ R ⁇ to contain a list of relevant terms.
- control circuitry 506 associates the N 111 term with the determined frequency in the relational data structure. For example, control circuitry 506 may add an entry to the relational data structure in which the N th term is a token, and the corresponding value is set to the determined frequency of the N 111 term.
- control circuitry 506 determines whether the frequency of the N 111 term is below a threshold frequency. If so (“Yes” at 1112), then, at 1114, control circuitry 506 adds the N* 11 term to ⁇ R ⁇ .
- control circuitry 506 After adding the N* term to ⁇ R ⁇ , or if the frequency of the N 111 term meets or exceeds the threshold frequency (“No” at 1112), at 1116, control circuitry 506 determines whether N is equal to T t , meaning that all the terms in the training data set have been processed. If N is not equal to T (“No” at 1116), then, at 1118, control circuitry 506 increments the value of N by one, and processing returns to step 1112. If N is equal to T t (“Yes” at 1116), then, at 1120, control circuitry 506 receives a natural language search query.
- control circuitry 506 using natural language processing circuitry 508, identifies a plurality of terms in the natural language search query. This may be accomplished using methods described above in connection with FIG. 8.
- control circuitry 506 initializes a counter variable K, setting its value to one, and a variable T representing the total number of identified terms in the natural language search query.
- control circuitry 506 determines whether the K* 1 ’ term is included in ⁇ R ⁇ , meaning that the K th term is a relevant term. If so (“Yes” at 1126), then, at 1128, control circuitry 506 identifies the K lh term as a keyword. After identifying the K lh term as a keyword, or if the K lh term is not included in ⁇ R ⁇ (“No” at 1126), at 1130, control circuitry 506 determines whether K is equal to T, meaning that all of the identified terms of the natural language search query have been processed. If K is not equal to T (“No” at 1130), then, at 1132, control circuitry 506 increments the value of K by one, and processing returns to step 1126.
- control circuitry 506 using natural language processing circuitry 508, interprets the natural language search query based on the identified keywords.
- control circuitry 506 retrieves search results (e.g ., from content database 540) based on the interpreted search query.
- control circuitry 506, using output circuitry 544, generates the search results for display to the user.
- FIG. 12 is a flowchart representing an illustrative process 1200 determining a frequency with which terms in a natural language search query occur in a training data set, in accordance with some embodiments of the disclosure
- Process 1200 may be implemented on control circuitry 506 or control circuitry 606.
- one or more actions of FIG. 12 may be incorporated into or combined with one or more actions of any other process or embodiment described herein.
- control circuitry e.g., control circuitry 506 counts the total number of words contained in the training data set.
- control circuitry 506 initializes a counter variable N, setting its value to one, at a variable T representing the total number of terms in the natural language search query.
- control circuitry 506 determines the total number of occurrences of the 111 term in the training data set. This may be accomplished using methods described above in connection with FIG. 9.
- control circuitry 506 calculates a percentage of the total number of words contained in the training data set corresponding to the total number of occurrences of the * term in the training data set. This may be accomplished using methods described above in connection with FIG. 9.
- control circuitry 506 determines whether N is equal to T, meaning that all the terms of the natural language search query have been processed. If N is not equal to T (“No” at 1210), then, at 1212, control circuitry 506 increments the value of N by one, and processing return to step 1206. If N is equal to T (“Yes” at 1210), then the process is complete.
- FIG. 12 is a flowchart representing an illustrative process 1300 generating a relational data structure associating terms of a natural language search query with respective frequencies of occurrence, in accordance with some embodiments of the disclosure.
- Process 1300 may be implemented on control circuitry 506 or control circuitry 606. In addition, one or more actions of FIG. 13 may be incorporated into or combined with one or more actions of any other process or embodiment described herein.
- control circuitry 506 creates a data structure comprising at least a token field and a corresponding value field.
- control circuitry 506 initializes a counter variable N, setting its value to one, and a variable T representing the total number of terms in the training data set.
- control circuitry 506 adds the N 111 term to the data structure as a token and, at 1318, sets the value corresponding to the token to the frequency of the N 111 term.
- control circuitry 506 determines whether N it equal to T, meaning that all the terms contained in the training data set have been processed. If N is not equal to T (“No” at 1310), then, at 1312, control circuitry 506 increments the value ofNby one, and processing return to step 1306. If N is equal to T (“Yes” at 1310), then the process is complete.
- FIG. 14 is a flowchart representing a third illustrative process 1400 interpreting a natural language search query, in accordance with some embodiments of the disclosure.
- Process 1400 may be implemented on control circuitry 506 or control circuitry 606.
- one or more actions of FIG. 14 may be incorporated into or combined with one or more actions of any other process or embodiment described herein.
- control circuitry receives a natural language search query.
- control circuitry 606 using natural language processing circuitry 608, determines whether the natural language search query comprises a complete sentence.
- natural language processing circuitry 608 may use Hidden Markov Model or Conditional Random Field algorithms or a grammar engine to determine the structure of the natural language search query. If the natural language search query does comprise a complete sentence (“Yes” at 1404), then, at 1406, control circuitry 606 identifies a plurality of terms in the natural language search query. This may be accomplished using methods described above in connection with FIG. 8.
- control circuitry 606 initializes a counter variable N, setting its value to one, and a variable T representing the total number of identified terms.
- control circuitry 606 using natural language processing circuitry 608, associates the N 111 term with a part of speech. This may be accomplished using methods described above in connection with FIG. 7.
- control circuitry 606 determines whether N is equal to T, meaning that all terms of the natural language search query have been associated with a part of speech. If N is not equal to T (“No” at 1412), then, at 1414, control circuitry 606 increments the value of N by one, and processing returns to step 1410.
- control circuitry 606 identifies, based on the sentence structure of the natural language search query, a query type. For example, is the natural language search query begins with “show me,” the query type will be a query for content items matching filter parameters contained in the remainder of the sentence. If the query beings with “what is,” then the query is an informational request which may return data other than content items.
- control circuitry 606 interprets the natural language search query, in the context of the query type, based on the parts of speech of each of the identified terms. This may be accomplished using methods described above in connection with FIG. 11.
- control circuitry 606 retrieves search results (e.g ., from content database 634) based on the interpreted search query.
- FIG. 15 is a flowchart representing a fourth illustrative process 1500 interpreting a natural language search query, in accordance with some embodiments of the disclosure.
- Process 1500 may be implemented on control circuitry 506 or control circuitry 606.
- one or more actions of FIG. 15 may be incorporated into or combined with one or more actions of any other process or embodiment described herein.
- control circuitry receives a natural language search query.
- control circuitry 606 using natural language processing circuitry 608, identifies a plurality of terms in the natural language search query. This may be accomplished using methods described above in connection with FIG. 8.
- control circuitry 606 initializes a counter variable N, settings its value to one, and a variable T representing the total number of identified terms.
- control circuitry 606, using vector generation circuitry 618 generates a vector for the N* 11 term describing a relationship between the N ,h term and a plurality of other terms. This may be accomplished using methods described below in connection with FIG. 16.
- control circuitry 606 determines whether N is equal to T, meaning that vectors have been generated for all terms of the natural language search query. If N is not equal to T (“No” at 1510), then, at 1512, control circuitry 606 increments the value of N by one, and processing returns to step 1508. If is equal to T (“Yes” at 1510), then, at 1514, control circuitry 606 inputs each vector into a trained neural network that generates an interpretation of each term for which a vector is input, and for the combination of terms for which vectors have been input. At 1516, control circuitry 606 interprets the natural language search query based on the output of the neural network.
- control circuitry 606 retrieves search results (e.g., from content database 634) based on the interpreted search query.
- FIG. 15 is a flowchart representing an illustrative process 1600 generating vectors for terms in a natural language search query for input into a neural network, in accordance with some embodiments of the disclosure.
- Process 1600 may be implemented on control circuitry 506 or control circuitry 606.
- one or more actions of FIG. 16 may be incorporated into or combined with one or more actions of any other process or embodiment described herein.
- control circuitry 606 using vector generation circuitry 618, accesses a knowledge graph associated with content metadata.
- the knowledge graph may contain nodes for every term in the content metadata and include connections between each node representing connections between each term in the content metadata, such as two terms included in the metadata describing a single content item.
- control circuitry 606 initializes a counter variable N, setting its value to one, and a variable T representing the total number of identified terms in the natural language search query, or the total number of relevant terms in the natural language search query.
- vector generation circuitry 618 identifies a plurality of terms to which the N 111 term is connected in the knowledge graph. For example, vector control circuitry 618 may count the number of nodes to which the node representing the N 111 term connects. At 1608, control circuitry 606 initializes another counter variable K, setting its value to one, and another variable TK representing the total number of terms to which the N* term connects. At 1610, vector generation circuitry 618 calculates a distance between the N 11 ' term and the K th connected term. For example, the K ,h term may connect directly to the node representing the N 111 term, or may connect indirectly through a number of intermediate nodes.
- control circuitry 606 determines whether K is equal to TK, meaning that a distance between the N 111 term and every term connected thereto has been calculated. If K is not equal to TK (“NO” at 1612), then, at 1614, control circuitry 606 increments the value of K by one, and processing returns to step 1610. If K is equal to TK (“Yes” at 1612), then, at 1616, vector generation circuitry 618 generates a vector for the N 111 term based on the connections of the N 111 term and the distance between the N 111 term and each connected term.
- a vector for the word “January” may include other months of the year with a close distance, and holidays that occur in the month of January with a farther distance.
- a vector for “Tom Cruise” may include other actors who have co-starred with Tom Cruise with close distances, genres in which Tom Cruise as acted with farther distances, and subgenres with even farther distances.
- control circuitry 606 determines whether N is equal to T, meaning that a vector for all terms of the natural language search query, or all relevant terms thereof, have been generated. If is not equal to T (“No” at 1618), then, at 1620, control circuitry 606 increments the value of N by one, and processing return to step 1606. IfN is equal to T (“Yes” at 1618), then the process is complete.
- FIG. 16 The actions or descriptions of FIG. 16 may be used with any other embodiment of this disclosure. In addition, the actions and descriptions described in relation to FIG. 16 may be done in suitable alternative orders or in parallel to further the purposes of this disclosure. [0085] The processes described above are intended to be illustrative and not limiting.
- a computer-implemented method for interpreting a natural language search query comprising using processing circuitry for: receiving the natural language search query; identifying a plurality of terms in the natural language search query; associating each terms of the plurality of terms with a respective part of speech; determining a respective frequency with which each term of the plurality of terms occurs in metadata describing a plurality of content items; determining a relevance for each term of the plurality of terms based on its respective part of speech and frequency; and interpreting the natural language search query based on of the relevance of each term.
- identifying a plurality of terms in the natural language search query comprises: splitting the natural language search query into a plurality of words; analyzing a first word of the plurality of words; determining, based on analyzing the first word, whether the first word can be part of a phrase; in response to determining that the first word can be part of a phrase, analyzing the first word together with a second word that immediately follows the first word; determining, based on analyzing the first word together with the second word, whether the first word and the second word form a phrase together; in response to determining that the first word and the second word form a phrase together, identifying the first and second word as a single term; and in response to determining that the first word does not form a phrase with the second word, identifying the first word as single term.
- determining a respective frequency with which each term of the plurality of terms occurs in metadata describing a plurality of content items comprises: retrieving the metadata describing a plurality of content items; counting the total number of words contained in the metadata; determining the total number of occurrences of each respective term of the plurality of terms; and calculating, for each term of the plurality of terms, a percentage of the total number of words contained in the metadata corresponding to the total number of occurrences of each respective term of the plurality of terms.
- determining a relevance of each term of the plurality of terms based on its respective part of speech and frequency comprises: determining whether the frequency meets or exceeds a threshold frequency; in response to determining that the frequency meets or exceeds the threshold frequency, determining that the respective term is not relevant; and in response to determining that the frequency is below the threshold frequency: determining a relevance factor to the respective term; and applying a weighting factor to the relevance factor based on the respective part of speech.
- determining a relevance factor to the respective term comprises dividing the frequency of the respective term by the threshold frequency.
- a system for interpreting a natural language search query comprising control circuitry configured to: receive the natural language search query; identify a plurality of terms in the natural language search query; associate each terms of the plurality of terms with a respective part of speech; determine a respective frequency with which each term of the plurality of terms occurs in metadata describing a plurality of content items; determine a relevance for each term of the plurality of terms based on its respective part of speech and frequency; and interpret the natural language search query based on of the relevance of each term.
- control circuitry configured to identify a plurality of terms in the natural language search query is further configured to: split the natural language search query into a plurality of words; analyze a first word of the plurality of words; determine, based on analyzing the first word, whether the first word can be part of a phrase; in response to determining that the first word can be part of a phrase, analyze the first word together with a second word that immediately follows the first word; determine, based on analyzing the first word together with the second word, whether the first word and the second word form a phrase together; in response to determining that the first word and the second word form a phrase together, identify the first and second word as a single term; and in response to determining that the first word does not form a phrase with the second word, identify the first word as single term.
- control circuitry configured to determine a respective frequency with which each term of the plurality of terms occurs in metadata describing a plurality of content items is further configured to: retrieve the metadata describing a plurality of content items; count the total number of words contained in the metadata; determine the total number of occurrences of each respective term of the plurality of terms; and calculate, for each term of the plurality of terms, a percentage of the total number of words contained in the metadata corresponding to the total number of occurrences of each respective term of the plurality of terms
- control circuitry configured to determine a relevance of each term of the plurality of terms based on its respective part of speech and frequency is further configured to: determine whether the frequency meets or exceeds a threshold frequency; in response to determining that the frequency meets or exceeds the threshold frequency, determine that the respective term is not relevant; and in response to determining that the frequency is below the threshold frequency: determine a relevance factor to the respective term; and apply a weighting factor to the relevance factor based on the respective part of speech.
- control circuitry configured to determine a relevance factor to the respective term is further configured to divide the frequency of the respective term by the threshold frequency.
- a non-transitory computer-readable medium having non-transitory computer-readable instructions encoded thereon for interpreting a natural language search query that, when executed by control circuitry, cause the control circuitry to: receive the natural language search query; identify a plurality of terms in the natural language search query; associate each terms of the plurality of terms with a respective part of speech; determine a respective frequency with which each term of the plurality of terms occurs in metadata describing a plurality of content items; determine a relevance for each term of the plurality of terms based on its respective part of speech and frequency; and interpret the natural language search query based on of the relevance of each term.
- execution of the instruction to identify a plurality of terms in the natural language search query further causes the control circuitry to: split the natural language search query into a plurality of words; analyze a first word of the plurality of words; determine, based on analyzing the first word, whether the first word can be part of a phrase; in response to determining that the first word can be part of a phrase, analyze the first word together with a second word that immediately follows the first word; determine, based on analyzing the first word together with the second word, whether the first word and the second word form a phrase together; in response to determining that the first word and the second word form a phrase together, identify the first and second word as a single term; and in response to determining that the first word does not form a phrase with the second word, identify the first word as single term.
- execution of the instruction to determine a relevance of each term of the plurality of terms based on its respective part of speech and frequency further causes the control circuitry to: determine whether the frequency meets or exceeds a threshold frequency; in response to determining that the frequency meets or exceeds the threshold frequency, determine that the respective term is not relevant; and in response to determining that the frequency is below the threshold frequency: determine a relevance factor to the respective term; and apply a weighting factor to the relevance factor based on the respective part of speech.
- a computer-implemented method for interpreting a natural language search query comprising using processing circuitry for: receiving the natural language search query; identifying a plurality of terms in the natural language search query; associating each terms of the plurality of terms with a respective part of speech; determining a respective frequency with which each term of the plurality of terms occurs in metadata describing a plurality of content items; determining a relevance for each term of the plurality of terms based on its respective part of speech and frequency; and interpreting the natural language search query based on of the relevance of each term.
- identifying a plurality of terms in the natural language search query comprises: splitting the natural language search query into a plurality of words; analyzing a first word of the plurality of words; determining, based on analyzing the first word, whether the first word can be part of a phrase; in response to determining that the first word can be part of a phrase, analyzing the first word together with a second word that immediately follows the first word; determining, based on analyzing the first word together with the second word, whether the first word and the second word form a phrase together; in response to determining that the first word and the second word form a phrase together, identifying the first and second word as a single term; and in response to determining that the first word does not form a phrase with the second word, identifying the first word as single term.
- determining a respective frequency with which each term of the plurality of terms occurs in metadata describing a plurality of content items comprises: retrieving the metadata describing a plurality of content items; counting the total number of words contained in the metadata; determining the total number of occurrences of each respective term of the plurality of terms; and calculating, for each term of the plurality of terms, a percentage of the total number of words contained in the metadata corresponding to the total number of occurrences of each respective term of the plurality of terms.
- determining a relevance of each term of the plurality of terms based on its respective part of speech and frequency comprises: determining whether the frequency meets or exceeds a threshold frequency; in response to determining that the frequency meets or exceeds the threshold frequency, determining that the respective term is not relevant; and in response to determining that the frequency is below the threshold frequency: determining a relevance factor to the respective term; and applying a weighting factor to the relevance factor based on the respective part of speech.
- determining a relevance factor to the respective term comprises dividing the frequency of the respective term by the threshold frequency.
- a computer-implemented method for interpreting a natural language search query based on training data comprising a first plurality of terms comprising, using processing circuitry for: determining a respective frequency with which each of the first plurality of terms occurs in the training data; generating a relational data structure associating each term of the first plurality of terms with its respective frequency; adding a term of the first plurality of terms to a list of relevant terms when its frequency is below a threshold; receiving the natural language search query; identifying a second plurality of terms in the natural language search query; determining, for each term of the second plurality of terms, whether the term is included in the list of relevant terms; in response to determining that a term of the second plurality of terms is included in the list of relevant terms, identifying the term as a keyword; and interpreting the natural language search query based on the identified keywords.
- determining a respective frequency with which each of the first plurality of terms occurs in the training data comprises: counting the total number of words contained in the training data; determining the total number of occurrences of each respective term of the plurality of terms in the training data; and calculating, for each term of the plurality of terms, a percentage of the total number of words contained in the training data corresponding to the total number of occurrences of each respective term of the plurality of terms in the training data.
- generating a relational data structure associating each term of the first plurality of terms with its respective frequency comprises: creating a data structure comprising at least a token field and a corresponding value field; adding to the data structure, for each term of the first plurality of terms, the respective term as a token; and setting the value corresponding to the token to the frequency of the respective term.
- adding a term of the plurality of terms to a list of relevant terms when its frequency is below a threshold comprises: determining whether the frequency of the term meets or exceeds a threshold frequency; in response to determining that the frequency meets or exceeds the threshold frequency, determining that the respective term is not relevant; and in response to determining that the frequency is below the threshold frequency, adding the term to the list of relevant terms.
- identifying a second plurality of terms in the natural language search query comprises: splitting the natural language search query into a plurality of words; analyzing a first word of the plurality of words; determining, based on analyzing the first word, whether the first word can be part of a phrase; in response to determining that the first word can be part of a phrase, analyzing the first word together with a second word that immediately follows the first word; determining, based on analyzing the first word together with the second word, whether the first word and the second word form a phrase together; in response to determining that the first word and the second word form a phrase together, identifying the first and second word as a single term; and in response to determining that the first word does not form a phrase with the second word, identifying the first word as single term.
- a system for interpreting a natural language search query based on training data comprising a first plurality of terms comprising control circuitry configured to: determine a respective frequency with which each of the first plurality of terms occurs in the training data; generate a relational data structure associating each term of the first plurality of terms with its respective frequency; add a term of the first plurality of terms to a list of relevant terms when its frequency is below a threshold; receive the natural language search query; identify a second plurality of terms in the natural language search query; determine, for each term of the second plurality of terms, whether the term is included in the list of relevant terms; in response to determining that a term of the second plurality of terms is included in the list of relevant terms, identify the term as a keyword; and interpret the natural language search query based on the identified keywords.
- control circuitry configured to determine a respective frequency with which each of the first plurality of terms occurs in the training data is further configured to: count the total number of words contained in the training data; determine the total number of occurrences of each respective term of the plurality of terms in the training data; and calculate, for each term of the plurality of terms, a percentage of the total number of words contained in the training data corresponding to the total number of occurrences of each respective term of the plurality of terms in the training data.
- control circuitry configured to generate a relational data structure associating each term of the first plurality of terms with its respective frequency is further configured to: create a data structure comprising at least a token field and a corresponding value field; add to the data structure, for each term of the first plurality of terms, the respective term as a token; and set the value corresponding to the token to the frequency of the respective term.
- control circuitry configured to add a term of the plurality of terms to a list of relevant terms when its frequency is below a threshold is further configured to: determine whether the frequency of the term meets or exceeds a threshold frequency; in response to determining that the frequency meets or exceeds the threshold frequency, determine that the respective term is not relevant; and in response to determining that the frequency is below the threshold frequency, add the term to the list of relevant terms.
- control circuitry configured to identify a second plurality of terms in the natural language search query is further configured to: split the natural language search query into a plurality of words; analyze a first word of the plurality of words; determine, based on analyzing the first word, whether the first word can be part of a phrase; in response to determining that the first word can be part of a phrase, analyze the first word together with a second word that immediately follows the first word; determine, based on analyzing the first word together with the second word, whether the first word and the second word form a phrase together; in response to determining that the first word and the second word form a phrase together, identify the first and second word as a single term; and in response to determining that the first word does not form a phrase with the second word, identify the first word as single term.
- a system for interpreting a natural language search query based on training data comprising a first plurality of terms comprising: means for determining a respective frequency with which each of the first plurality of terms occurs in the training data; means for generating a relational data structure associating each term of the first plurality of terms with its respective frequency; means for adding a term of the first plurality of terms to a list of relevant terms when its frequency is below a threshold; means for receiving the natural language search query; means for identifying a second plurality of terms in the natural language search query; means for determining, for each term of the second plurality of terms, whether the term is included in the list of relevant terms; means for, in response to determining that a term of the second plurality of terms is included in the list of relevant terms, identifying the term as a keyword; and means for interpreting the natural language search query based on the identified keywords.
- the means for determining a respective frequency with which each of the first plurality of terms occurs in the training data comprises: means for counting the total number of words contained in the training data; means for determining the total number of occurrences of each respective term of the plurality of terms in the training data; and means for calculating, for each term of the plurality of terms, a percentage of the total number of words contained in the training data corresponding to the total number of occurrences of each respective term of the plurality of terms in the training data.
- the means for generating a relational data structure associating each term of the first plurality of terms with its respective frequency comprises: means for creating a data structure comprising at least a token field and a corresponding value field; means for adding to the data structure, for each term of the first plurality of terms, the respective term as a token; and means for setting the value corresponding to the token to the frequency of the respective term.
- the means for adding a term of the plurality of terms to a list of relevant terms when its frequency is below a threshold comprises: means for determining whether the frequency of the term meets or exceeds a threshold frequency; means for, in response to determining that the frequency meets or exceeds the threshold frequency, determining that the respective term is not relevant; and means for, in response to determining that the frequency is below the threshold frequency, adding the term to the list of relevant terms.
- the means for identifying a second plurality of terms in the natural language search query comprises: means for splitting the natural language search query into a plurality of words; means for analyzing a first word of the plurality of words; means for determining, based on analyzing the first word, whether the first word can be part of a phrase; means for, in response to determining that the first word can be part of a phrase, analyzing the first word together with a second word that immediately follows the first word; means for determining, based on analyzing the first word together with the second word, whether the first word and the second word form a phrase together; means for, in response to determining that the first word and the second word form a phrase together, identifying the first and second word as a single term; and means for, in response to determining that the first word does not form a phrase with the second word, identifying the first word as single term.
- a non-transitory computer-readable medium having non-transitory computer-readable instructions encoded thereon for interpreting a natural language search query based on training data comprising a first plurality of terms that, when executed by control circuitry, cause the control circuitry to: determine a respective frequency with which each of the first plurality of terms occurs in the training data; generate a relational data structure associating each term of the first plurality of terms with its respective frequency; add a term of the first plurality of terms to a list of relevant terms when its frequency is below a threshold; receive the natural language search query; identify a second plurality of terms in the natural language search query; determine, for each term of the second plurality of terms, whether the term is included in the list of relevant terms; in response to determining that a term of the second plurality of terms is included in the list of relevant terms, identify the term as a keyword; and interpret the natural language search query based on the identified keywords.
- execution of the instruction to determine a respective frequency with which each of the first plurality of terms occurs in the training data further causes the control circuitry to: count the total number of words contained in the training data; determine the total number of occurrences of each respective term of the plurality of terms in the training data; and calculate, for each term of the plurality of terms, a percentage of the total number of words contained in the training data corresponding to the total number of occurrences of each respective term of the plurality of terms in the training data.
- execution of the instruction to generate a relational data structure associating each term of the first plurality of terms with its respective frequency further causes the control circuitry to: create a data structure comprising at least a token field and a corresponding value field; add to the data structure, for each term of the first plurality of terms, the respective term as a token; and set the value corresponding to the token to the frequency of the respective term.
- execution of the instruction to add a term of the plurality of terms to a list of relevant terms when its frequency is below a threshold further causes the control circuitry to: determine whether the frequency of the term meets or exceeds a threshold frequency; in response to determining that the frequency meets or exceeds the threshold frequency, determine that the respective term is not relevant; and in response to determining that the frequency is below the threshold frequency, add the term to the list of relevant terms.
- execution of the instruction to identify a second plurality of terms in the natural language search query further causes the control circuitry to: split the natural language search query into a plurality of words; analyze a first word of the plurality of words; determine, based on analyzing the first word, whether the first word can be part of a phrase; in response to determining that the first word can be part of a phrase, analyze the first word together with a second word that immediately follows the first word; determine, based on analyzing the first word together with the second word, whether the first word and the second word form a phrase together; in response to determining that the first word and the second word form a phrase together, identify the first and second word as a single term; and in response to determining that the first word does not form a phrase with the second word, identify the first word as single term.
- a computer-implemented method for interpreting a natural language search query based on training data comprising a first plurality of terms comprising, using processing circuitry for: determining a respective frequency with which each of the first plurality of terms occurs in the training data; generating a relational data structure associating each term of the first plurality of terms with its respective frequency; adding a term of the first plurality of terms to a list of relevant terms when its frequency is below a threshold; receiving the natural language search query; identifying a second plurality of terms in the natural language search query; determining, for each term of the second plurality of terms, whether the term is included in the list of relevant terms; in response to determining that a term of the second plurality of terms is included in the list of relevant terms, identifying the term as a keyword; and interpreting the natural language search query based on the identified keywords.
- determining a respective frequency with which each of the first plurality of terms occurs in the training data comprises: counting the total number of words contained in the training data; determining the total number of occurrences of each respective term of the plurality of terms in the training data; and calculating, for each term of the plurality of terms, a percentage of the total number of words contained in the training data corresponding to the total number of occurrences of each respective term of the plurality of terms in the training data.
- generating a relational data structure associating each term of the first plurality of terms with its respective frequency comprises: creating a data structure comprising at least a token field and a corresponding value field; adding to the data structure, for each term of the first plurality of terms, the respective term as a token; and setting the value corresponding to the token to the frequency of the respective term.
- adding a term of the plurality of terms to a list of relevant terms when its frequency is below a threshold comprises: determining whether the frequency of the term meets or exceeds a threshold frequency; in response to determining that the frequency meets or exceeds the threshold frequency, determining that the respective term is not relevant; and in response to determining that the frequency is below the threshold frequency, adding the term to the list of relevant terms.
- identifying a second plurality of terms in the natural language search query comprises: splitting the natural language search query into a plurality of words; analyzing a first word of the plurality of words; determining, based on analyzing the first word, whether the first word can be part of a phrase; in response to determining that the first word can be part of a phrase, analyzing the first word together with a second word that immediately follows the first word; determining, based on analyzing the first word together with the second word, whether the first word and the second word form a phrase together; in response to determining that the first word and the second word form a phrase together, identifying the first and second word as a single term; and in response to determining that the first word does not form a phrase with the second word, identifying the first word as single term.
- a computer-implemented method for interpreting a natural language search query comprising using processing circuitry for: receiving the natural language search query; determining, using natural language processing, that the natural language search query comprises a grammatical structure; identifying a plurality of terms in the natural language search query; associating each term of the plurality of terms with a respective part of speech; identifying, based on a the grammatical, a query type; interpreting the natural language search query, in the context of the query type, based on the parts of speech of each of the plurality of terms; retrieving search results based on the interpreted search query; and generating for display the search results.
- identifying a plurality of terms in the natural language search query comprises: splitting the natural language search query into a plurality of words; analyzing a first word of the plurality of words; determining, based on analyzing the first word, whether the first word can be part of a phrase; in response to determining that the first word can be part of a phrase, analyzing the first word together with a second word that immediately follows the first word; determining, based on analyzing the first word together with the second word, whether the first word and the second word form a phrase together; in response to determining that the first word and the second word form a phrase together, identifying the first and second word as a single term; and in response to determining that the first word does not form a phrase with the second word, identifying the first word as single term.
- a system for interpreting a natural language search query comprising control circuitry configured to: receive the natural language search query; determine, using natural language processing, that the natural language search query comprises a grammatical structure; identify a plurality of terms in the natural language search query; associate each term of the plurality of terms with a respective part of speech; identify, based on the grammatical structure, a query type; interpret the natural language search query, in the context of the query type, based on the parts of speech of each of the plurality of terms; retrieve search results based on the interpreted search query; and generate for display the search results.
- control circuitry configured to identify a plurality of terms in the natural language search query is further configured to: split the natural language search query into a plurality of words; analyze a first word of the plurality of words; determine, based on analyzing the first word, whether the first word can be part of a phrase; in response to determining that the first word can be part of a phrase, analyze the first word together with a second word that immediately follows the first word; determine, based on analyzing the first word together with the second word, whether the first word and the second word form a phrase together; in response to determining that the first word and the second word form a phrase together, identify the first and second word as a single term; and in response to determining that the first word does not form a phrase with the second word, identify the first word as single term.
- a system for interpreting a natural language search query comprising: means for receiving the natural language search query; means for determining, using natural language processing, that the natural language search query comprises a grammatical structure; means for identifying a plurality of terms in the natural language search query; means for associating each term of the plurality of terms with a respective part of speech; means for identifying, based on a grammatical structure, a query type; means for interpreting the natural language search query, in the context of the query type, based on the parts of speech of each of the plurality of terms; means for retrieving search results based on the interpreted search query; and means for generating for display the search results. 59.
- the means for identifying a plurality of terms in the natural language search query comprises: means for splitting the natural language search query into a plurality of words; means for analyzing a first word of the plurality of words; means for determining, based on analyzing the first word, whether the first word can be part of a phrase; means for, in response to determining that the first word can be part of a phrase, analyzing the first word together with a second word that immediately follows the first word; means for determining, based on analyzing the first word together with the second word, whether the first word and the second word form a phrase together; means for, in response to determining that the first word and the second word form a phrase together, identifying the first and second word as a single term; and means for, in response to determining that the first word does not form a phrase with the second word, identifying the first word as single term.
- a non-transitory computer-readable medium having non-transitory computer-readable instructions encoded thereon for interpreting a natural language search query that, when executed by control circuitry, causes the control circuitry to: receive the natural language search query; determine, using natural language processing, that the natural language search query comprises a grammatical structure; identify a plurality of terms in the natural language search query; associate each term of the plurality of terms with a respective part of speech; identify, based on a grammatical structure, a query type; interpret the natural language search query, in the context of the query type, based on the parts of speech of each of the plurality of terms; retrieve search results based on the interpreted search query; and generate for display the search results.
- execution of the instruction for identifying a plurality of terms in the natural language search query further causes the control circuitry to: split the natural language search query into a plurality of words; analyze a first word of the plurality of words; determine, based on analyzing the first word, whether the first word can be part of a phrase; in response to determining that the first word can be part of a phrase, analyze the first word together with a second word that immediately follows the first word; determine, based on analyzing the first word together with the second word, whether the first word and the second word form a phrase together; in response to determining that the first word and the second word form a phrase together, identify the first and second word as a single term; and in response to determining that the first word does not form a phrase with the second word, identify the first word as single term.
- a computer-implemented method for interpreting a natural language search query comprising using processing circuitry for: receiving the natural language search query; identifying a plurality of terms in the natural language search query; generating a respective vector for each term of the plurality of terms, wherein each vector describes a relationship between the respective term and a second plurality of terms; inputting the respective vector for each term into a trained neural network that generates an output; interpreting the natural language search query based on the output; retrieving search results based on the interpreted search query; and generating for display the search results.
- identifying a plurality of terms in the natural language search query comprises: splitting the natural language search query into a plurality of words; analyzing a first word of the plurality of words; determining, based on analyzing the first word, whether the first word can be part of a phrase; in response to determining that the first word can be part of a phrase, analyzing the first word together with a second word that immediately follows the first word; determining, based on analyzing the first word together with the second word, whether the first word and the second word form a phrase together; in response to determining that the first word and the second word form a phrase together, identifying the first and second word as a single term; and in response to determining that the first word does not form a phrase with the second word, identifying the first word as single term.
- generating a respective vector for each term of the plurality of terms comprises: accessing a knowledge graph associated with content metadata; identifying plurality of terms to which each term of the plurality of terms connects in the knowledge graph; calculating a distance between each respective term and each term connected to the respective term; and generating a vector for each term based on the connections of each respective term and the distance between each respective term and each term to which each respective term is connected.
- a system for interpreting a natural language search query comprising control circuitry configured to: receive the natural language search query; identify a plurality of terms in the natural language search query; generate a respective vector for each term of the plurality of terms, wherein each vector describes a relationship between the respective term and a second plurality of terms; input the respective vector for each term into a trained neural network that generates an output; interpret the natural language search query based on the output; retrieve search results based on the interpreted search query; and generate for display the search results.
- control circuitry configured to identify a plurality of terms in the natural language search query is further configured to: split the natural language search query into a plurality of words; analyze a first word of the plurality of words; determine, based on analyzing the first word, whether the first word can be part of a phrase; in response to determining that the first word can be part of a phrase, analyze the first word together with a second word that immediately follows the first word; determine, based on analyzing the first word together with the second word, whether the first word and the second word form a phrase together; in response to determining that the first word and the second word form a phrase together, identify the first and second word as a single term; and in response to determining that the first word does not form a phrase with the second word, identify the first word as single term.
- control circuitry configured to generate a respective vector for each term of the plurality of terms is further configured to: access a knowledge graph associated with content metadata; identify plurality of terms to which each term of the plurality of terms connects in the knowledge graph; calculate a distance between each respective term and each term connected to the respective term; and generate a vector for each term based on the connections of each respective term and the distance between each respective term and each term to which each respective term is connected.
- a system for interpreting a natural language search query comprising: means for receiving the natural language search query; means for identifying a plurality of terms in the natural language search query; means for generating a respective vector for each term of the plurality of terms, wherein each vector describes a relationship between the respective term and a second plurality of terms; means for inputting the respective vector for each term into a trained neural network that generates an output; means for interpreting the natural language search query based on the output; means for retrieving search results based on the interpreted search query; and means for generating to display the search results.
- the means for identifying a plurality of terms in the natural language search query comprises: means for splitting the natural language search query into a plurality of words; means for analyzing a first word of the plurality of words; means for determining, based on analyzing the first word, whether the first word can be part of a phrase; means for, in response to determining that the first word can be part of a phrase, analyzing the first word together with a second word that immediately follows the first word; means for determining, based on analyzing the first word together with the second word, whether the first word and the second word form a phrase together; means for, in response to determining that the first word and the second word form a phrase together, identifying the first and second word as a single term; and means for, in response to determining that the first word does not form a phrase with the second word, identifying the first word as single term.
- the means for generating a respective vector for each term of the plurality of terms comprises: means for accessing a knowledge graph associated with content metadata; means for identifying plurality of terms to which each term of the plurality of terms connects in the knowledge graph; means for calculating a distance between each respective term and each term connected to the respective term; and means for generating a vector for each term based on the connections of each respective term and the distance between each respective term and each term to which each respective term is connected.
- a non-transitory computer-readable medium having non-transitory computer-readable instructions encoded thereon for interpreting a natural language search query that, when executed by control circuitry, cause the control circuitry to: receive the natural language search query; identify a plurality of terms in the natural language search query; generate a respective vector for each term of the plurality of terms, wherein each vector describes a relationship between the respective term and a second plurality of terms; input the respective vector for each term into a trained neural network that generates an output; interpret the natural language search query based on the output; retrieve search results based on the interpreted search query; and generate for display the search results.
- execution of the instruction to identify a plurality of terms in the natural language search query causes the control circuitry to: split the natural language search query into a plurality of words; analyze a first word of the plurality of words; determine, based on analyzing the first word, whether the first word can be part of a phrase; in response to determining that the first word can be part of a phrase, analyze the first word together with a second word that immediately follows the first word; determine, based on analyzing the first word together with the second word, whether the first word and the second word form a phrase together; in response to determining that the first word and the second word form a phrase together, identify the first and second word as a single term; and in response to determining that the first word does not form a phrase with the second word, identify the first word as single term.
- execution of the instruction to generate a respective vector for each term of the plurality of terms causes the control circuitry to: access a knowledge graph associated with content metadata; identify plurality of terms to which each term of the plurality of terms connects in the knowledge graph; calculate a distance between each respective term and each term connected to the respective term; and generate a vector for each term based on the connections of each respective term and the distance between each respective term and each term to which each respective term is connected.
- a computer-implemented method for interpreting a natural language search query comprising using processing circuitry for: receiving the natural language search query; identifying a plurality of terms in the natural language search query; generating a respective vector for each term of the plurality of terms, wherein each vector describes a relationship between the respective term and a second plurality of terms; inputting the respective vector for each term into a trained neural network that generates an output; interpreting the natural language search query based on the output; retrieving search results based on the interpreted search query; and generating for display the search results.
- identifying a plurality of terms in the natural language search query comprises: splitting the natural language search query into a plurality of words; analyzing a first word of the plurality of words; determining, based on analyzing the first word, whether the first word can be part of a phrase; in response to determining that the first word can be part of a phrase, analyzing the first word together with a second word that immediately follows the first word; determining, based on analyzing the first word together with the second word, whether the first word and the second word form a phrase together; in response to determining that the first word and the second word form a phrase together, identifying the first and second word as a single term; and in response to determining that the first word does not form a phrase with the second word, identifying the first word as single term.
- generating a respective vector for each term of the plurality of terms comprises: accessing a knowledge graph associated with content metadata; identifying plurality of terms to which each term of the plurality of terms connects in the knowledge graph; calculating a distance between each respective term and each term connected to the respective term; and generating a vector for each term based on the connections of each respective term and the distance between each respective term and each term to which each respective term is connected.
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Artificial Intelligence (AREA)
- Computational Linguistics (AREA)
- Data Mining & Analysis (AREA)
- Databases & Information Systems (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
Description
Claims
Priority Applications (2)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CA3143945A CA3143945A1 (en) | 2020-03-03 | 2020-12-16 | Systems and methods for interpreting natural language search queries |
| EP20842092.7A EP4115303A1 (en) | 2020-03-03 | 2020-12-16 | Systems and methods for interpreting natural language search queries |
Applications Claiming Priority (8)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| US16/807,422 | 2020-03-03 | ||
| US16/807,421 US20210280176A1 (en) | 2020-03-03 | 2020-03-03 | Systems and methods for enabling interpretation of a natural language search query |
| US16/807,421 | 2020-03-03 | ||
| US16/807,415 US11594213B2 (en) | 2020-03-03 | 2020-03-03 | Systems and methods for interpreting natural language search queries |
| US16/807,422 US20210279264A1 (en) | 2020-03-03 | 2020-03-03 | Systems and methods for interpreting a natural language search query using a neural network |
| US16/807,415 | 2020-03-03 | ||
| US16/807,419 US11914561B2 (en) | 2020-03-03 | 2020-03-03 | Systems and methods for interpreting natural language search queries using training data |
| US16/807,419 | 2020-03-03 |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| WO2021178020A1 true WO2021178020A1 (en) | 2021-09-10 |
| WO2021178020A8 WO2021178020A8 (en) | 2021-10-21 |
Family
ID=74184900
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| PCT/US2020/065307 Ceased WO2021178020A1 (en) | 2020-03-03 | 2020-12-16 | Systems and methods for interpreting natural language search queries |
Country Status (3)
| Country | Link |
|---|---|
| EP (1) | EP4115303A1 (en) |
| CA (1) | CA3143945A1 (en) |
| WO (1) | WO2021178020A1 (en) |
Citations (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20120136649A1 (en) * | 2010-11-30 | 2012-05-31 | Sap Ag | Natural Language Interface |
-
2020
- 2020-12-16 WO PCT/US2020/065307 patent/WO2021178020A1/en not_active Ceased
- 2020-12-16 CA CA3143945A patent/CA3143945A1/en active Pending
- 2020-12-16 EP EP20842092.7A patent/EP4115303A1/en not_active Ceased
Patent Citations (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20120136649A1 (en) * | 2010-11-30 | 2012-05-31 | Sap Ag | Natural Language Interface |
Also Published As
| Publication number | Publication date |
|---|---|
| CA3143945A1 (en) | 2021-09-10 |
| WO2021178020A8 (en) | 2021-10-21 |
| EP4115303A1 (en) | 2023-01-11 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US12062366B2 (en) | Systems and methods for interpreting natural language search queries | |
| US11740863B2 (en) | Search and knowledge base question answering for a voice user interface | |
| US12499099B2 (en) | Systems and methods for interpreting natural language search queries using training data | |
| US20210279264A1 (en) | Systems and methods for interpreting a natural language search query using a neural network | |
| US11626113B2 (en) | Systems and methods for disambiguating a voice search query | |
| US10872601B1 (en) | Natural language processing | |
| US12367345B2 (en) | Identifying high effort statements for call center summaries | |
| AU2019219717B2 (en) | System and method for analyzing partial utterances | |
| US20230214382A1 (en) | Systems and methods for interpreting natural language search queries | |
| JP2007115145A (en) | Conversation control device | |
| US20210280176A1 (en) | Systems and methods for enabling interpretation of a natural language search query | |
| Kaushik et al. | Automatic audio sentiment extraction using keyword spotting. | |
| US20250036617A1 (en) | Systems and methods for processing negation in natural language search queries | |
| US12204866B1 (en) | Voice based searching and dialog management system | |
| US20250239255A1 (en) | Systems and methods for improving content discovery in response to a voice query using a recognition rate which depends on detected trigger terms | |
| WO2021178020A1 (en) | Systems and methods for interpreting natural language search queries | |
| HK40055458B (en) | Answer generation method, device, equipment and storage medium based on artificial intelligence |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| 121 | Ep: the epo has been informed by wipo that ep was designated in this application |
Ref document number: 20842092 Country of ref document: EP Kind code of ref document: A1 |
|
| ENP | Entry into the national phase |
Ref document number: 3143945 Country of ref document: CA |
|
| NENP | Non-entry into the national phase |
Ref country code: DE |
|
| ENP | Entry into the national phase |
Ref document number: 2020842092 Country of ref document: EP Effective date: 20221004 |
|
| WWW | Wipo information: withdrawn in national office |
Ref document number: 2020842092 Country of ref document: EP |