JP2016531375A - Local and remote speech processing - Google Patents

Abstract

The user device may be configured to detect the trigger expression emitted by the user and respond by interpreting subsequent words or phrases as commands. The command may be recognized by sending audio containing words or phrases to a remote service configured to perform speech recognition. Certain commands may be designated as local commands and may be detected locally rather than relying on remote services. Upon detecting the trigger expression, the audio is streamed to the remote service and analyzed locally to detect local command utterances. When a local command is detected, the corresponding function is started immediately and subsequent activity or response by the remote service is canceled or ignored.

Description

  The present invention relates to local and remote speech processing.

Cross-reference of related applications
This application claims priority to US Patent Application No. 14 / 033,302, filed on September 20, 2013, and is named “Local and Remote Speech Processing”. , Incorporated herein by reference in its entirety.

  Homes, offices, cars and public spaces are more wired and associated with the proliferation of computing devices such as laptops, tablets, entertainment systems and portable communication devices. As computing devices evolve, the way users interact with these devices continues to evolve. For example, a person can communicate with a computing device (eg, keyboard, mouse, etc.), an electrical device (eg, touch screen, touchpad, etc.), and an optical device (eg, motion detector, camera, etc.) via a mechanical device. Can interact. Another way to interact with a computing device is through an audio device that captures and reacts to human speech.

The detailed description is described with reference to the accompanying drawings. In the figure, the leftmost digit (s) of a reference number identifies the drawing in which the reference number first appears. The use of the same reference numbers in different drawings indicates similar or identical components or features.
1 is a block diagram of an exemplary voice interaction computing architecture including a local audio device and a remote voice processing service. 2 is a flow diagram illustrating an exemplary process for detecting a representation of a command that can be executed by a local audio device in connection with a remote voice processing service. 2 is a flow diagram illustrating an exemplary process for detecting a representation of a command that can be executed by a local audio device in connection with a remote voice processing service. 2 is a flow diagram illustrating an exemplary process for detecting a representation of a command that can be executed by a local audio device in connection with a remote voice processing service.

  The present disclosure relates generally to a speech interface system that provides or facilitates speech-based interaction with a user. The system includes a local device having a microphone that captures audio including user speech. Spoken user commands may be attached with keywords called trigger expressions or awakening expressions. The audio following the trigger expression may be streamed to a remote service for speech recognition, and the service may provide a response by performing a function or a command executed by the audio device.

  Communication with remote services may in most cases introduce response latencies that can be kept to an acceptable minimum. Some colloquial commands, however, may be invoked for low latency. As an example, colloquial commands associated with a particular type of media rendering, such as stop, pause, hang up, etc. may need to be executed with a low latency perceptual amount.

  According to various embodiments, a local command, or a specific command expression referred to herein as a local command expression, is detected by or at the local device rather than being detected by a remote service. More specifically, the local device is configured to detect a trigger or warning expression that indicates that subsequent speech is intended by the user to form a command. It is configured to detect a trigger, or warning expression, that indicates that subsequent audio is intended by the user to form a command. In detecting the trigger expression, the local device initiates streaming to indicate a communication session with the remote service and receive audio to the service. In response, the remote service performs speech recognition of the received audio and attempts to identify the user's intention based on the recognized speech. In response to recognition of the user's intention, the remote service may perform a corresponding function. In some examples, the function may be performed in connection with a local device. For example, the remote service may send a command to the local device indicating that the local device should execute a command to perform the corresponding function.

  Concurrent with the remote service activity, the local device monitors or analyzes the audio that detects the occurrence of the local command expression following the trigger expression. Upon detecting a local command representation in the audio, the local device immediately implements the corresponding function. In addition, further actions by the remote service are stopped or canceled to avoid duplicate actions for a single user utterance. The action by the remote service is any command specified by the remote service by notifying the explicit remote service where the utterance was acted on locally by the termination or cancellation of the communication session and / or in response to remote recognition of the user speech It may be canceled by execution of matching.

  FIG. 1 shows an example of a voice interaction system 100. System 100 may utilize or include a local audio-based audio device 102 that may be located within an environment 104 such as a home and that may be used to interact with user 106. The voice interaction system 100 is also configured to receive audio that recognizes speech in audio and to perform a function, so that a remote network-based speech command referred to as a service identification function in response to the recognized speech. A service 108 may be utilized or included. The service identification function may be implemented by the speech command service 108 independent of the audio device and / or by providing a command to the audio device 102 for local execution.

  In certain embodiments, the primary mode of audio device 102 and user interaction may be via speech. For example, the audio device 102 may receive a spoken command expression from the user 106 and may provide a service in response to the command. The user may speak a predefined awakening or trigger expression (eg, awake) or continue with a command or command (eg, I want to go to a movie. Please tell me what you have.) The provided services perform actions or activities, media rendering, acquisition and / or provision of information, provide information via generated or synthesized speech via the audio device 102, on behalf of the user 106, etc. It may include starting a base service.

  Local audio device 102 and speech command service 108 are configured to interact and interact with each other to receive and respond to a command representation from user 106. The command representation may include a command representation that is detected and performed by the independent local device 102 of the speech command service 108. The command representation also includes commands that are interpreted and executed by or in association with the remote speech command service 108.

  Audio service 102 may include one or more microphones 110 and one or more speakers or transducers 112 that facilitate interaction with user 106. Microphone 110 contains sounds or representations made by user 106 and represents the audio from environment 104 and generates a microphone signal referenced as an audio signal input.

  In some cases, the microphone 110 may include a microphone array used in connection with an audio beamforming technique that produces an audio signal input that is concentrated in a selected direction. Similarly, a multi-directional microphone 110 may be used to generate an audio signal corresponding to one of a plurality of available directions.

  Audio service 102 includes processing logic, which in many cases may include processor 114 and memory 116. The processor 114 may include multiple processors having multiple cores and / or a single processor. The processor 114 may also include or comprise a digital single processor that processes audio signals.

  Memory 116 includes the special functions described below, and may include applications and programs in the formation of computer-executed instructions executed by processor 114 that performs acts or activities that perform the required functions of audio device 102. . The memory 116 may be a computer readable storage media type and may include volatile and non-volatile memory. And the memory 116 may include, but is not limited to, RAM, ROM, EEPROM, flash memory, or other memory technology.

  Audio device 102 may include a plurality of applications, services and / or functions 118 referred to below collectively as functional components 118 executed by processor 114 for providing services and functions. Applications and other functional components 118 may include media playback services such as music players. Other services or operations provided or performed by applications and other functional components 118 include, for example, request or consumption entertainment (eg, games, music discovery performances, movies or other content), personal management (eg, , Calendaring, taking notes, etc.), online shopping, financial transactions, database inquiries, person-to-person voice communications, and the like.

  In some embodiments, the functional component 118 may be pre-installed on the audio device 102 or implemented in the core functionality of the audio device 102. In other embodiments, one or more applications or other functional components 118 may be installed by the user 106, or may be installed after the audio device 102 utilized by the user 106, and the user It may be implemented as a request by 106 or in a customized function.

  The processor 114 may be configured with audio processing of a function or component 120 that processes the audio signal input generated by the audio signal output supplied to the microphone 110 and / or the speaker 112. As an example, the audio processing component 120 may perform acoustic echo cancellation that reduces audio echo generated by the acoustic coupling between the microphone 110 and the speaker 112. Audio processing component 120 may also perform noise reduction to reduce noise in the received audio signal, such as elements of the audio signal input other than user speech. In certain embodiments, the audio processing component 120 may include one or more audio beam generators that are responsive to a plurality of microphones 110 from which user speech is generated in an audio signal focused in a direction from detection.

  Audio device 102 may be configured to execute one or more expression detectors or speech recognition component 122 that may be used to detect a trigger expression in speech captured by microphone 110. The term “trigger expression” is used herein to indicate a word, phrase or other utterance used in the signal of the audio device 102 intended by the user for which subsequent user speech is interpreted as a command.

  One or more recognition components 122 may also be used to detect commands or command expressions in speech captured by the microphone 110. The term “command expression” refers to a word, phrase or other that corresponds to or is associated with a function performed by the audio device 102 or by a service or other service accessible to the audio service 102 such as the speech command service 108. Used here to show the utterances. For example, the words “stop”, “pause”, and “hang-up” may be used as command expressions. The “stop”, “pause” command expressions may indicate that the media playback activity is interrupted. The “hang-up” command expression may indicate that the current person-to-person communication should be terminated. Other command expressions corresponding to different functions may also be used. The command expression may include an interactive command such as “find a nearby Italian restaurant”.

  The command representation may include a local command representation that should be interpreted by the audio device 102 without resorting to the speech command service 108. In general, a local command expression is a related short expression such as a single language or a short phrase that can be easily detected by the audio device 102. The local command representation may be responsive to device functions for which a relatively low response latency is required, such as media processing or media playback processing functions. The service of the speech command service 108 may be utilized for other command expressions because larger response latencies are accepted. The command representation to be acted upon by the speech command service is referred to herein as the remote command representation.

  In some cases, the speech recognition component 122 may be implemented using automatic speech recognition (ASR) technology. For example, a large amount of speech recognition technology may be used for keyword reduction, and the speech recognition output may be monitored for the appearance of keywords. As an example, speech recognition may use a Hidden Markov model and a Gaussian mixture model to provide a continuous word stream corresponding to speech input and to recognize speech. The word stream may then be monitored to detect one or more special languages and expressions.

  Alternatively, the speech recognition component 122 may be implemented by one or more keyword spotters. A keyword spotter is a functional component or algorithm that evaluates an audio signal to detect the presence of one or more predefined words or expressions in the audio signal. In general, keyword spotters use a limited number of languages rather than a simplified ASR technique for detecting specific words or recognizing a fairly large vocabulary. For example, a keyword spotter may provide notification if a special language is detected in the audio signal rather than providing text or word-based output. Keyword spotters using these techniques may compare to another word based on a Hidden Markov Model (HMM) that represents the word as a series of states. In general, utterances are analyzed by comparing the model against a keyword model and a background model. Comparing the utterance model with the keyword model obtains a score representing the likelihood that the utterance corresponds to the keyword. Comparing the utterance model with the keyword model obtains a score that represents the utterance corresponding to a common word other than the keyword. The two scores can be compared to determine if the keyword was spoken.

  The audio device 102 further includes a control function 124, referred to herein as a controller or control logic that is configured to interact with other components of the audio device 102 to implement the logical function of the audio device 102. Is done.

  Control logic 124, audio processing component 120, speech recognition component 122, and functional component 118 may include executable instructions, programs, and / or program modules that are stored and executed by processor 114 in memory 116.

  The speech command service 108 may be part of a network accessible computing platform that, in some examples, is maintained and accessible via a network 126 such as the Internet. Such network accessible computing platforms are “software as a service (SaaS)”, “on-demand computing”, “platform computing”, “network access platform”, “cloud service”, “data center”. Reference may be made using terms such as.

  Audio device 102 and / or speech command service 108 may be wired technology (eg, wire, universal serial bus (USB), fiber optic cable, etc.), wireless technology (eg, radio frequency (RF), mobile phone, mobile phone network, Satellite, Bluetooth, etc.) may be communicatively coupled to the network 126, or other connection technology. Network 126 represents any type of communication network including data and / or voice networks, and wired infrastructure (eg, coaxial cable, fiber optic cable, etc.), wireless infrastructure (eg, RF, cellular, micro, etc.). Wave, satellite, Bluetooth, etc.) and / or other connection technologies may be used.

  However, the audio device 102 is described herein as a voice control or speech-based interface device, and the techniques described herein are various such as communication devices and components, hands-free devices, entertainment devices, media playback devices, etc. It may be implemented in conjunction with different types of devices.

  The speech command service 108 typically receives an audio stream from the audio device 102, recognizes speech in the audio stream, determines user intent from the recognized speech, and performs an activity or service depending on the user intent. Provides functions to execute. The provided activities may be performed in connection with the audio device 102 in some cases, in which the speech command service 108 returns a response to the audio device 102 indicating the command to be performed by the audio device 102. May be.

  The speech command service 108 includes processing logic that may often include one or more servers, computers, and / or processors 128. The speech command service 108 may also include applications in the form of instructions executed by the processor 128 to perform actions or activities that implement the required functionality of the speech command service, including the functions specifically described herein. You may have the memory 130 containing a program. Memory 130 may be a type of computer storage media and may include volatile and non-volatile memory. Thus, the memory 130 is not limited, and may include RAM, ROM, EEPROM, flash memory, or other memory technology.

  The speech command service 108 may include a speech recognition component 132, among other logical and physical components not specifically shown. The speech recognition component 132 may include a human speech recognition on an audio signal, an automatic speech recognition (ASR) function.

  The speech command service 108 may also include a natural language understanding component (NLU) 134 that determines user intent based on the recognized speech.

  The speech command service 108 may include a command interpreter and action dispatcher 136 (simply referred to below as the command interpreter 136) that determines functions or commands according to the user's intention. In some cases, the commands may correspond to functions that are at least partially performed by the audio device 102, and the command interpreter 136 may send commands to the audio device 102 that indicate commands to implement such functions. A response may be provided in those cases. Examples of commands or functions that may be executed by an audio device in response to a command from the command interpreter 136 include generating audible audio through the speaker 112 and a particular type of communication with a user of a similar device. And playback of music or other media that increases / decreases the volume of the speaker 112 may be included.

  It should be noted that the speech command service 108 may perform the function of responding to speech recognized from received audio with entities or devices not shown in FIG. For example, the speech command service 108 may interact with other network-based services to obtain information or services on behalf of the user 106. Further, the speech command service 108 may itself have various components and functions that can respond to the user's 106 speech.

  In operation, the microphone 110 of the audio device 102 captures or receives audio including the user's 106 speech. The audio is processed by the audio processing component 120 and the processed audio is received by the speech recognition component 122. The speech recognition component 122 analyzes the audio that detects the occurrence of a trigger expression in the speech included in the audio. Upon detecting the trigger expression, the controller 124 transmits or streams the received audio to the speech command service 108 in response to the request of the speech command service 108 to interpret and recognize the user speech, and responds to certain interpretation intentions. Start to start function.

  At the same time, sending audio to the speech command service 108, the speech recognition component 122 continues to analyze the received audio to detect the occurrence of local command expressions in the user speech. Upon detection of the local command expression, the controller 124 initiates or executes a device function corresponding to the local command expression. For example, in response to the local command expression “stop”, the controller 124 may initiate a function to stop media playback. The controller 124 may interact with one or more functional components 118 when initiating or executing a function.

  On the other hand, the speech command service 108 responds to the reception of the audio, and at the same time, determines the service identification function realized according to the user's intention, and recognizes the speech to recognize the user's intention. Is analyzed. However, after detecting and acting locally on the local command representation, the audio device 102 may take action to cancel, disable, or disable any service identification function that may eventually be initiated by the speech command service 108. It may be taken. For example, the audio device 102 may cancel the previous request by sending a cancel message to the speech command service 108 and / or by stopping streaming audio to the speech command service 108. As another example, the audio device may ignore or discard any response or service specific command received from the speech command service 108 in response to a previous request. In some cases, the audio device may notify the speech command service 108 of actions performed locally in response to the local command representation, and the speech command service 108 may perform subsequent actions based on the information. May be changed. For example, the speech command service 108 may refrain from actions that may otherwise be recognized in the received audio and that are performing in response to the speech.

  FIG. 2 illustrates an example method 200 that may be performed by the audio device 102 in connection with the speech command service 108 to recognize and respond to user speech. The method 200 is described in the context of the system 100 of FIG. 1, nevertheless, the method 200 may be performed in other environments and may be implemented in different ways.

  The actions on the left side of FIG. 2 are performed by and by the local audio device 102. The actions on the right side of FIG. 2 are also performed by the remote speech command service 108 and by the remote speech command service 108.

  Action 202 is associated with microphone 110 and includes receiving an audio signal captured by microphone 110. The audio signal includes audio from the environment 104 and may include representations and user speech. The audio signal may be an analog electrical signal or may include a digital signal such as a digital audio stream.

  Action 204 includes detecting the occurrence of a trigger expression in the received audio and / or user speech. This may include a keyword spotter in some embodiments, and may be performed by the speech recognition component 122 as described above. If no trigger expression is detected, action 204 is repeated to continuously monitor the generation of the trigger expression. The remaining actions shown in FIG. 2 are performed in response to detecting the trigger expression.

If a trigger expression is detected in action 204, action 206 is executed along with service request 208 for speech command service 108 that recognizes speech in audio and implements a function in response to the recognized speech. Transmission following the audio received by the speech command service 108. Such functions initiated by the speech command service 108 are referred to herein as service identification functions and may be performed in connection with the audio device 102 in special cases. For example, the function may be initiated by sending a command to the audio device 102.

  Transmission 206 includes other transmissions of digital audio stream 210 to streaming or speech command service 108 and may represent or include audio received from microphone 110 following detection of a trigger expression. In certain embodiments, action 206 may include opening or initiating a communication session between audio device 102 and speech command service 108. Specifically, the request 208 understands the intent, establishes a communication session with the speech command service 108 for the purpose of speech recognition, and determines the action or function to be performed in response to the user speech. May be used. Request 208 may be followed or accompanied by streamed audio 210. In some cases, the audio stream 210 provided to the speech command service 108 may include a portion of the received audio that starts just before the trigger expression latency.

  The communication session may be associated with a communication or session identifier (ID) that identifies a communication session established between the audio device 102 and the speech command service 108. The session ID may be included or used in a particular user utterance or future communication associated with the audio stream. In some cases, the session ID may be generated by audio device 102 and provided in request 208 for speech command service 108. Alternatively, the session ID may be generated by the speech command service 108 and provided by the speech command service 108 upon acceptance of the request 208. The term “request (ID)” is used herein to indicate a request with a particular session ID. Responses from the speech command service 108 associated with the same session request or audio stream may be indicated by the term “response (ID)”.

  In certain embodiments, each communication session and corresponding session ID may correspond to a single user utterance. For example, the audio device 102 may establish a session upon detecting the trigger expression. Audio device 102 may stream audio to speech command service 108 as part of the same session until the end of the user's utterance and then continue. The speech command service 108 may provide a response to the audio device 102 via the session using the same session ID. The response that may be in some cases may indicate a command to be executed by the audio device 102 in response to the speech recognized by the speech command service 108 on the received audio 210. The communication session may remain open until the audio device 102 receives a response from the speech command service 108 and until the audio device 102 cancels the request.

  The speech command service 108 receives the request 208 and the audio stream 210 at action 212. In response, the speech command service 108 recognizes the received audio and uses the speech recognition service 132 and the natural language understanding component 134 of the speech command service 108 to determine the user's intention as represented by the recognized speech. The action 214 is executed. Action 214 performed by command interpreter 136 includes identifying and initiating a service identification function that satisfies the determined user intent. The service identification function may be performed by the independent speech command service 108 of the audio device 102 in some cases. In other cases, the speech command service 108 may identify functions to be performed by the audio device 102 and may send commands corresponding to the audio device 102 for execution by the audio device 102.

  Concurrent with the action performed by the speech command service 108, the local audio device 102 determines whether the user is issuing a local command expression and whether to perform a corresponding local function in response to such an utterance local command expression. Take further action to make a decision. Specifically, action 218 performed in response to detecting the trigger expression in action 204 is an action 202 that detects the occurrence of a local command expression in the user speech that is subsequently or immediately followed by the trigger expression in the received user speech. Includes analysis of received audio. This may be performed by the speech recognition component 122 of the audio device 102 as described above, and may include a keyword spotter in some embodiments.

  In response to detecting the local command representation at action 218, action 220 immediately initiates the device function associated with the local command representation. For example, “stop” in the local command expression may be associated with a function to stop media playback.

  Also, in response to detecting the local command expression at action 218, audio device 102 performs action 222 to stop or cancel request 208 for speech command service 108. This may include canceling or disabling the implementation of the service identification function that may be implemented by the speech command service 108 in response to the received request 208 and with the audio 210.

  In a specific implementation, action 222 sends an explicit notification or command to speech command service 108, and speech command service 108 requests service request 208 to cancel any further cognitive activity. And / or canceling the implementation of a service identification function that may be initiated in response to speech recognized as otherwise. Alternatively, the audio device 102 may simply notify the speech command service 108 regarding any function that is performed locally in response to local recognition of the local command representation, and the speech command service 108 may send a service request 208. You may respond by canceling or by performing other actions that are appropriate.

  In certain implementations, the speech command service 108 may perform a service identification function by identifying commands executed by the audio device 102. The speech command service 108 may forego sending a command to the audio device 102 in response to receiving a notification that the service request 208 is cancelled. Alternatively, the speech command service may complete its processing and send the command to the audio device 102, so that the audio device 102 may ignore the command or suspend execution of the command. .

  In some implementations, the speech command service may be configured to notify the audio device 102 before initiating the service identification function and implements the service identification function until a permission is received from the audio device 102. You may delay. In this case, the audio device 102 may be configured to deny such permission when the local command representation is recognized as local.

  The various approaches described above may be used in situations where different amounts of command latency are desired. For example, waiting for communication from the speech command service introduces a relatively high latency, which may not be acceptable in some situations. Communication prior to realizing such a function may be prevented against overlapping or unintended actions. The immediate realization of local recognition command representation and the request to ignore subsequent commands from the speech command service or subsequently cancel the speech command service is more appropriate in situations where less latency is desired. There should be.

  It should be noted that the actions of the speech command service 108 shown in FIG. 2 are performed in parallel and asynchronously with the actions 218, 220 and 222 of the audio device 102. This envisions some implementations in which the audio device 102 can detect and act on local command representations relatively quickly, which may perform an action 222 that cancels the request 208. , And subsequent processing by the speech command service 108 prior to the service identification function of action 216 has been implemented or performed.

  FIG. 3 illustrates that the speech command service 108 returns a command to the audio device 102, and the audio device 102 ignores the command or suspends execution of the command in situations where a local command representation is already detected and acted on by the audio device 102. An example method 300 configured as described is shown. The initial action is similar or identical to that described above. The actions performed by the audio device 102 are shown on the left and the actions performed by the speech command service 108 are shown on the right.

  Action 302 includes receiving an audio signal that includes user speech. Action 304 includes analyzing the audio signal to detect a trigger expression in the user speech. The subsequent actions shown in FIG. 3 are performed in response to detecting the trigger expression.

  Action 306 includes sending a request 308 and audio 310 to the speech command service 108. Action 312 includes receiving request 308 and audio 310 at speech command service 108. Action 314 includes recognizing user speech and determining user intent based on the recognized user speech.

  In response to the determined user intention, the speech command service 108 sends a command 318 to the audio device 102 for execution by the audio device 102 to implement a service identification function corresponding to the recognized user intention. The action 316 is executed. For example, the command may include a “stop” command indicating that the audio device 102 stops playing music.

  Action 320 performed by audio device 102 includes receiving and executing a command. Action 320 is shown in a dashed box to indicate that it is conditionally executed based on whether a local command representation is detected and acted upon by audio device 102. Specifically, if a local command representation is detected by audio device 102, action 320 is not performed.

  In parallel with the actions performed by the speech command service 108, the audio device 102 performs an audio reception analysis action 322 that detects the occurrence of a local command expression following or immediately following the trigger expression in the received user speech. To do. In response to detecting the local command representation, action 324 immediately occurs to initiate a local device function associated with the local command representation.

  Also, in response to the detection of the local command expression in action 322, the audio device 102 performs execution of matching the received command 318 in action 326. More specifically, commands received from the speech command service 108 in response to the request 308 are discarded or ignored. The response and command corresponding to the request 308 may be identified by a session ID associated with the response.

  If no local command representation is detected at action 322, the audio device performs action 320 for the command 318 being received and executed from the speech command service 108.

  FIG. 4 illustrates an example method 400 that is configured for the audio device 102 to actively cancel a request to the speech command service 108 after detecting a local command representation locally. The initial action is similar or identical to those described above. The actions performed by the audio device 102 are displayed on the left and the actions performed by the speech command service 108 are shown on the right.

  Action 402 includes receiving an audio signal that includes user speech. Action 404 includes analyzing the audio signal to detect a trigger expression in the user speech. The subsequent actions shown in FIG. 4 are performed in response to detecting the trigger expression.

  Action 406 includes sending request 408 and audio 410 to speech command service 108. Action 412 includes receiving request 408 and audio 410 at speech command service 108. Action 414 includes recognizing user speech and determining user intent based on the recognized user speech.

  Action 416 includes determining whether request 408 has been canceled by audio device 102. As an example, the audio device 102 may send a cancel message or may terminate the current communication session to cancel the request. If the request is canceled by the audio device 102, no further action is obtained by the speech command service. If the request has not been canceled, action 418 is executed, which sends a command 420 to the audio device 102 for execution by the audio device 102 to implement a service identification function corresponding to the recognized user intent. Including that.

  Action 422 performed by audio device 102 includes receiving and executing a command. Action 422 depends on whether a command has been sent and received from the speech command service 108, which then indicates that the audio device 102 will be executed conditionally depending on whether the request 408 has been canceled. Is indicated by a dashed box.

  In parallel with the actions performed by the speech command service 108, the audio device 102 performs an audio reception analysis action 424 that detects the occurrence of a local command expression following or immediately following the trigger expression in the received user speech. To do. In response to detecting the representation of the local command, action 426 immediately occurs to initiate a local device function associated with the local command representation.

  Also, in response to detecting the local command representation at action 424, the audio device 102 cancels the request 408 to the speech command service 108 and / or recognizes speech recognized by the speech command service 108 from the audio device 102. An action 428 is executed that requests canceling the implementation of any service identification function that may be performed in response to. This may include communication with such a speech command service 108 by sending a notification or request cancellation.

  In some cases, the cancellation may include a reply to a communication or notification from the speech command service 108 of the implementation that suspends the service identification function by the speech command. In response to receiving such a notification, audio device 102 may reply and request cancellation of the pending implementation. Instead, the audio device 102 may cancel the implementation of any function that may otherwise have been performed in response to detection of the local command representation, and speech to proceed with the implementation of the pending function. The command service 108 may be commanded.

  If no local command representation is detected at action 424, audio device 102 performs an action 422 of executing command 420 received from speech command service 108. Action 422 may occur asynchronously upon receiving command 420 from the speech command service.

  The above-described embodiments may be realized by a program such as a computer, a processor, a digital signal processor, an analog processor, and the like. In other embodiments, however, one or more of the components, functions, or elements may be implemented using special or dedicated circuitry including analog circuitry and / or digital logic circuitry. The term “component” as used herein is intended to include any hardware, software, logic, or combination of the foregoing that is used to implement the functionality attributable to the component.

  Although the subject matter is described in specific language for structural features, it is to be understood that the subject matter defined in the appended claims is not necessarily limited to the specific features described. It is. Rather, the specific features are disclosed as forms of implementing the claims.

Article 1. One or more non-transitory computer-readable media having computer-executable instructions stored thereon, wherein the computer-executable instructions, when executed, receive audio including user speech on one or more processors. Receiving, detecting a trigger expression in the user speech, streaming the received audio to a remote speech command service in response to detecting the trigger expression in the user speech, and analyzing the received audio In response to detecting a local command expression associated with a device function following the trigger expression in the user speech and detecting the local command expression following the trigger expression in the user speech; Starting the device function Receiving a response from the remote speech command service, wherein the response indicates a command to be executed in response to speech recognized by the remote speech command service on the streamed audio; and Executing a command indicated by the response if a local command expression is not detected following the trigger expression in the user speech; and the local command expression follows the trigger expression in the user speech. Suspending execution of the command indicated by the response if detected;
One or more non-transitory computer readable media that cause an operation comprising:

2. The one or more computer-readable media of clause 1, wherein the streaming is associated with a communication identifier and the response indicates the communication identifier.

3. The one or more computer-readable media of clause 1, wherein the device functionality includes media control functionality.

4). The one or more computer-readable media of clause 1, wherein the operation further comprises stopping the streaming of the received audio in response to detecting the command representation.

5. Receiving audio including user speech; detecting a trigger expression in the user speech; and recognizing the speech in the received audio in response to detection of the trigger expression in the user speech. To perform the first function corresponding to the recognized speech, send the received audio to a speech command service, analyze the received audio, and follow the trigger expression on the received audio. In response to detecting a local command expression associated with the function of the first command and detecting the local command expression following the trigger expression in the received audio, and initiating the second function, A method that includes canceling execution of a function.

6). 6. The method of clause 5, wherein canceling execution of the first function comprises requesting the speech command service to cancel execution of the first function.

7). Receiving further communication indicating the pending execution of the first function from the speech command service, wherein canceling the execution of the first function is the pending execution of the first function; 6. The method of clause 5, comprising requesting the speech command service to cancel.

8). The method further includes receiving a command corresponding to the first function from the speech command service, and canceling execution of the first function is to suspend execution of the received command from the speech command service. The method of clause 5, including.

9. 6. The method of clause 5, further comprising notifying the speech command service that the second function has been initiated.

10. 6. The method of clause 5, wherein canceling execution of the first function includes notifying the speech command service that the second function has been initiated.

11. 6. The method of clause 5, wherein the second function includes a media control function.

12 In response to detecting the trigger representation in the audio, further comprising establishing a communication session with the speech command service, and canceling execution of the first function includes stopping the communication session. The method according to clause 5.

13. Associating an identifier with the received audio;
Receiving a response indicating the identifier and a command corresponding to the first function from the speech command service, and canceling the execution of the first function is to suspend execution of the command. The method of clause 5, including.

14 One or more speech recognition components configured to recognize user speech in received audio, detect a trigger expression in the user speech, and detect a local command expression in the user speech; and Control logic configured to perform an action in response to detecting the trigger expression in the user speech by or more speech recognition components, the action comprising the audio Transmitting the audio to a speech command service for recognizing speech and performing a first function corresponding to the recognized speech, and the user speech by the one or more speech recognition components Responds to detection of local command expression in Te, and to cancel at least one execution of (a) the local command that identifies a second function corresponding to the representation and (b) said first and second functions,
Including the system.

15. 15. The system of clause 14, wherein the one or more speech recognition components includes one or more keyword spotters.

16. 15. The system of clause 14, wherein canceling the at least one execution of the first and second functions includes requesting the speech command service to cancel execution of the first function.

17. 15. The system of clause 14, wherein canceling the at least one execution of the first and second functions includes ignoring a command received from the speech command service.

18. 15. The system of clause 14, wherein the second function includes a media control function.

19. 15. The system of clause 14, wherein the operation further comprises stopping transmitting the audio in response to detecting the local command representation in the user speech.

20. 15. The system of clause 14, wherein canceling the at least one execution of the first and second functions includes notifying the speech command service that the second function has been initiated.

Claims (15)

A device having computer-executable instructions stored therein, wherein when the computer-executable instructions are executed, the one or more processors of the device include:
Receiving audio including user speech;
Detecting a trigger expression in the user speech;
In response to detecting the trigger expression in the user speech,
Streaming the received audio to a remote speech command service;
Analyzing the received audio to detect a local command expression associated with a device function following the trigger expression in the user speech;
In response to detecting the local command expression following the trigger expression in the user speech, initiating the device function;
Receiving a response from the remote speech command service, wherein the response indicates a command to be executed in response to speech recognized by the remote speech command service on the streamed audio;
Executing the command indicated by the response if the local command expression is not detected following the trigger expression in the user speech;
Suspending execution of the command indicated by the response if the local command expression is detected following the trigger expression in the user speech;
A device that performs operations including   The device of claim 1, wherein the streaming is associated with a communication identifier and the response indicates the communication identifier.   The device of claim 1, wherein the device function comprises a media control function.   The device of claim 1, wherein the operation further comprises stopping the streaming of the received audio in response to detecting the command representation. Receiving audio including user speech;
Detecting a trigger expression in the user speech;
In response to detecting the trigger expression in the user speech,
Transmitting the received audio to a speech command service to recognize the speech in the received audio and perform a first function corresponding to the recognized speech;
Analyzing the received audio and detecting a local command expression associated with a second function following the trigger expression in the received audio;
In response to detecting the local command expression following the trigger expression in the received audio,
Starting the second function,
Canceling execution of the first function.   The method of claim 5, wherein canceling execution of the first function comprises requesting the speech command service to cancel execution of the first function. Further comprising receiving a communication indicating pending execution of the first function from the speech command service;
The method of claim 5, wherein canceling execution of the first function comprises requesting the speech command service to cancel the pending execution of the first function.   The method further includes receiving a command corresponding to the first function from the speech command service, and canceling execution of the first function is to suspend execution of the received command from the speech command service. 6. The method of claim 5, comprising.   6. The method of claim 5, further comprising notifying the speech command service that the second function has been initiated. Associating an identifier with the received audio;
Further comprising receiving from the speech command service a response indicating the identifier and a command corresponding to the first function;
The method of claim 5, wherein canceling execution of the first function includes suspending execution of the command. One or more speech recognition components configured to recognize user speech in received audio, detect a trigger expression in the user speech, and detect a local command expression in the user speech;
Control logic configured to perform an action in response to detecting the trigger expression in the user speech by the one or more speech recognition components;
The operation is
Transmitting the audio to a speech command service to recognize speech in the audio and to perform a first function corresponding to the recognized speech;
Responsive to detection of the local command representation in the user speech by the one or more speech recognition components, (a) identifying a second function corresponding to the local command representation; and (b) the Canceling at least one execution of the first and second functions;
Including the system.   12. The system of claim 11, wherein canceling the at least one execution of the first and second functions includes requesting the speech command service to cancel execution of the first function. .   The system of claim 11, wherein canceling the at least one execution of the first and second functions comprises ignoring a command received from the speech command service.   The system of claim 11, wherein the operation further comprises stopping transmitting the audio in response to detecting the local command representation in the user speech.   12. The system of claim 11, wherein canceling the at least one execution of the first and second functions includes notifying the speech command service that the second function has been initiated.

Images