JP2010128015A - Device and program for determining erroneous recognition in speech recognition - Google Patents

Abstract

Disclosed is a speech recognition misrecognition determination apparatus that determines whether a speech recognition result is misrecognition.
A speech recognition misrecognition determination apparatus includes a speech recognition unit that recognizes a speech command based on speech data and a speech command dictionary, and a recognition result that executes a response process for the recognition result by the speech recognition unit. The response unit, the face image acquisition unit 15 that acquires the user's face image data within a certain time after the response process, the utterance data acquisition unit 16 that acquires the utterance data, and the face image data acquired by the face image acquisition unit 15 Unconscious utterance recognition for recognizing unconscious utterances based on utterance data acquired by the utterance data acquisition unit 16 and unconscious utterance dictionary 22 Recognized when a predetermined facial expression or head movement is recognized by the unit 18 and the face image recognition unit 17 or when an unconscious utterance is recognized by the unconscious utterance recognition unit 18 And a erroneous recognition determination unit 19 determines that the recognition result is erroneous.
[Selection] Figure 1

Description

  The present invention relates to a speech recognition misrecognition determination apparatus and a speech recognition misrecognition determination program for determining whether recognition of speech uttered by a user is erroneous recognition.

  2. Description of the Related Art Conventionally, in various fields, a voice recognition device that recognizes a voice uttered by a user and operates a device according to the recognition result is used. In such a speech recognition apparatus, if speech recognition is not performed correctly, device operation due to misrecognition that is not intended by the user is performed, and the user may feel uncomfortable.

  In such a case, it is determined that the voice recognition has not been performed correctly based on a user response such as a device operation for canceling the device operation due to the misrecognition, and at that time, a measure that does not cause discomfort to the user is taken. A speech recognition device (see, for example, Patent Document 1) has been proposed.

In addition, a control device based on unconscious utterance that recognizes an erroneous operation from an unconscious utterance when a user performs an erroneous operation such as a command mistake and returns an appropriate response (see, for example, Patent Document 2) has also been proposed.
JP 2001-228894 A JP-A-5-165600

  However, in the speech recognition device of Patent Document 1, an error in speech recognition is determined based on a reaction of a user's device operation. Accordingly, since the determination is performed after the device operation by the user, the canceling operation becomes troublesome. Furthermore, it is necessary to wait for the determination until the user performs a canceling device operation, and there is a problem that it takes time.

  In addition, the control device based on unconscious utterances in Patent Document 2 targets the user's own error, and the response to the error on the device side is insufficient. Moreover, only the voice of unconscious utterance is targeted, and there is a problem in the error detection accuracy.

As described above, in the speech recognition apparatus, erroneous recognition cannot be avoided. Usually, when a result different from the voice input by the user is output (for example, the output “Amaton” for the input “Avaton”), the user unconsciously shows some reaction. Specifically, “laughter” or “surprise” when the result is significantly different from the input, “angry”, “disappointment”, “drill” when the goal cannot be achieved due to misrecognition many times, Show expressions such as “sadness”. Furthermore, the head may bend, shake the head, or sway, or the voice of "Eh", "Lie", "Why", etc. may be unconscious. In order to solve the above-mentioned problems, focusing on the user's reaction when the speech recognition result of the user's utterance is incorrect, the voice recognition misrecognition determination that accurately determines whether or not the recognition is wrong An object of the present invention is to provide an apparatus and a program for determining misrecognition of speech recognition.

  In order to achieve the above object, an erroneous recognition determination apparatus for speech recognition according to claim 1 is based on speech data input by a user and a speech command dictionary in which speech commands corresponding to speech data are registered. A voice command recognition means for recognizing a voice command corresponding to the voice data that has been recorded, a recognition result response means for executing a response process for a recognition result by the voice command recognition means, and a response process by the recognition result response means A face image acquiring means for acquiring the user's face image data within a predetermined time; a utterance data acquiring means for acquiring the user's utterance data within the predetermined time; and the face image acquiring means. Image recognition means for recognizing a predetermined facial expression and a predetermined head movement based on the face image data acquired by Unconscious utterance recognition means for recognizing the unconscious utterance corresponding to the acquired utterance data based on the utterance data acquired by the utterance data acquisition means and the unconscious utterance dictionary in which the unconscious utterance corresponding to the utterance data is registered; When the predetermined facial expression or the predetermined head movement is recognized by the image recognition unit, or when the unconscious utterance is recognized by the unconscious utterance recognition unit, the recognition result by the voice command recognition unit Misrecognition judging means for judging that it is misrecognized.

  According to the first aspect of the present invention, it is possible to determine the misrecognition of the speech recognition result based on the facial expression, head movement, or unconscious utterance that the user shows in response processing corresponding to the recognition result of the user utterance. .

  3. The voice recognition misrecognition determination apparatus according to claim 2, wherein the recognition result response means outputs a recognition result by the voice command recognition means. And / or device operating means for operating the device in response to the recognition result by the voice command recognition means.

  According to the second aspect of the present invention, it is possible to determine the erroneous recognition of the speech recognition result based on the output of the recognition result corresponding to the recognition result of the user utterance or the reaction shown by the user to the device operation.

  The voice recognition misrecognition determination apparatus according to claim 3 is the voice recognition misrecognition determination apparatus according to claim 1 or 2, wherein the recognition result by the voice command recognition means is erroneously recognized by the misrecognition determination means. The apparatus further includes device operation control means for stopping the operation of the device by the device operation means when determined.

  According to the third aspect of the present invention, when the recognition result of the user utterance is determined to be misrecognition, the device operation based on the misrecognition can be stopped.

  The voice recognition misrecognition determination apparatus according to claim 4 is the voice recognition misrecognition determination apparatus according to any one of claims 1 to 3, wherein the predetermined facial expression is laughing, surprised, angry, discouraged. The voice command recognition means misidentified by the voice command recognition means, such as bruise, sadness, etc., and the voice command recognition means, such as the predetermined head movement is a neck movement, a head swing, a sledding, etc. This is the operation shown by the user when

  According to the fourth aspect of the present invention, with respect to the response process corresponding to the recognition result of the user utterance, the facial expression and the head movement that the user shows, such as laughter, surprise, anger, discouragement, fear, sadness, etc. The recognition result can be determined to be misrecognition when shown or when a head movement such as a neck sway, a head swing, or a sled is performed.

  The speech recognition misrecognition determination program according to claim 5 causes a computer to function as each unit constituting the speech recognition misrecognition determination apparatus according to any one of claims 1 to 4.

  According to the fifth aspect of the present invention, it is possible to determine the misrecognition of the voice recognition result based on the facial expression, the head movement, or the unconscious utterance that the user shows in response processing corresponding to the recognition result of the user utterance. .

  According to a sixth aspect of the present invention, there is provided a misrecognition determination program for voice recognition, wherein a computer is configured to input voice data based on voice data inputted by a user and a voice command dictionary in which voice commands corresponding to the voice data are registered. Voice command recognition means for recognizing the corresponding voice command, recognition result response means for executing a response process for the recognition result by the voice command recognition means, and within a predetermined time after executing the response process by the recognition result response means Based on the face image data acquired by the face image acquisition means for acquiring the face image data of the user, the utterance data acquisition means for acquiring the utterance data of the user within the predetermined time, and the face image data acquired by the face image acquisition means. Image recognition means for recognizing an image of a predetermined facial expression and a predetermined head movement; Unconscious utterance recognition means for recognizing unconscious utterances corresponding to the acquired utterance data based on the utterance data acquired by the data acquisition means and an unconscious utterance dictionary in which unconscious utterances corresponding to the utterance data are registered, and the image When the predetermined facial expression or the predetermined head movement is recognized by the recognition means, or when the unconscious utterance is recognized by the unconscious utterance recognition means, the recognition result by the voice command recognition means is erroneously recognized. Function as a misrecognition determination means.

  According to the sixth aspect of the present invention, it is possible to determine the misrecognition of the speech recognition result based on the facial expression, the head movement, or the unconscious utterance that the user shows with respect to the response process corresponding to the recognition result of the user utterance. .

  As described above, according to the present invention, it is possible to accurately determine whether or not the speech recognition result of the user utterance is an error.

  Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings. In the present embodiment, a description will be given of device operation when the erroneous recognition determination device for speech recognition according to the present invention is used in a vehicle car navigation system (hereinafter referred to as “navigation”) having a speech recognition function. In addition, this invention is not limited to the above-mentioned embodiment, It is applicable also to what changed the design within the range described in the claim.

  FIG. 1 is a block diagram showing a configuration of a speech recognition misrecognition determination apparatus according to an embodiment of the present invention. As shown in the figure, the voice recognition misrecognition determination apparatus includes a voice data input unit 11, a voice recognition unit 12, a command execution unit 13, a recognition result output unit 14, a face image acquisition unit 15, and a utterance. The data acquisition unit 16, the face image recognition unit 17, the unconscious utterance recognition unit 18, the voice command dictionary 21, and the unconscious utterance dictionary 22 are provided.

  The voice data input unit 11 includes a microphone, and accepts user voice data input to execute a voice command.

  The voice recognition unit 12 recognizes the voice data input by the voice data input unit 11 using the voice command dictionary 21.

  The command execution unit 13 executes the voice command recognized by the voice recognition unit 12 to perform device operation.

  The recognition result output unit 14 includes a speaker, and outputs a message based on the voice command recognized by the voice recognition unit 12 from the speaker. Further, the recognition result output unit 14 may display the message on the map display screen of the navigation instead of voice output, or both may be performed simultaneously.

  The face image acquisition unit 15 includes a CCD camera, and acquires the user's face image data for a predetermined time after the voice command is executed by the command execution unit 13 and the message is output by the recognition result output unit 14. To do.

  The utterance data acquisition unit 16 includes a microphone, and acquires voice data uttered by the user for a predetermined time after the execution of the voice command by the command execution unit 13 and the output of the message by the recognition result output unit 14. To do.

  The face image recognition unit 17 performs image recognition on the user's face image data acquired by the face image acquisition unit 15 and performs “laughter”, “surprise”, “anger”, “disappointment”, “clear”, “ If any of facial expressions such as “sadness” and head gestures such as “neck wagging”, “swinging”, and “no-sledding” are recognized, it is determined that the recognition is erroneous.

  The unconscious utterance recognition unit 18 recognizes the utterance data acquired by the utterance data acquisition unit 16 using the voice command dictionary 21 and the unconscious utterance dictionary 22, and one or more words registered in the unconscious utterance dictionary 22 are recognized. If it is detected, it is determined as a misrecognition.

  Based on the determination results of the face image recognition unit 17 and the unconscious utterance recognition unit 18, the misrecognition determination unit 19 determines whether or not the recognition result by the voice recognition unit 12 is a misrecognition. In the present embodiment, the misrecognition determination unit 19 determines misrecognition when any one of the face image recognition unit 17 and the unconscious utterance recognition unit 18 determines misrecognition.

  The voice command dictionary 21 is a dictionary in which navigation voice commands and their readings are registered in pairs. FIG. 2 shows an example of the voice command dictionary 21.

  The unconscious utterance dictionary 22 is a dictionary in which utterances that are considered to be uttered unconsciously and their readings are registered in pairs when it is understood that a voice command issued by the user has been misrecognized. The unconscious utterance dictionary 22 may be created in advance by intentionally causing misrecognition with respect to voice input in a voice recognition experiment or the like separately performed, and collecting user utterances immediately after the misrecognition. FIG. 3 shows an example of the unconscious utterance dictionary 22.

  The voice recognition misrecognition determination apparatus configured as described above determines whether or not the recognition result is a misrecognition based on the user's reaction to the recognition result of the voice command input by the user. FIG. 4 is a flowchart showing the flow of operation of the speech recognition misrecognition determination apparatus.

  In step 100, the voice data input unit 11 accepts a command utterance uttered by a user for navigation operation. In this embodiment, it is assumed that “convenience store display” is input in order to display a convenience store icon on the map screen of the navigation.

  In step 102, the voice recognition unit 12 recognizes the voice data received by the voice data input unit 11 using the voice command dictionary 21. In the present embodiment, it is assumed that the voice recognition unit 12 erroneously recognizes “two-screen display” from the voice commands registered in the voice command dictionary 21 for the input voice data.

  In step 104, the command execution unit 13 executes the voice command recognized by the voice recognition unit 12, and the recognition result output unit 14 outputs a message based on the recognized voice command. In this embodiment, the command execution unit 13 executes a device operation command for dividing the navigation map screen display into two screens based on the recognition result. In addition, the recognition result output unit 14 outputs a voice “Use two screen display” based on the recognition result using a speaker. The output by the recognition result output unit 14 is not limited to voice output, and may be output as characters on a navigation screen.

  In step 106, the face image acquisition unit 15 acquires the face image data of the user at a fixed time after the command execution unit 13 executes the device operation command and outputs a message based on the recognition result in the recognition result output unit 14. At the same time, the utterance data acquisition unit 16 acquires voice data uttered by the user. The time for acquiring the face image data and the voice data is set to 5 seconds in this embodiment as a time for capturing a reaction when the recognition result is recognized as misrecognition by command execution and message output. In the present embodiment, it is assumed that the user shows an expression of “surprise” and utters “what”.

  In step 108, the face image recognition unit 17 performs image recognition on the face image data acquired by the face image acquisition unit 15, and “laughs”, “surprise”, “anger”, “disappointment”, “clear”, “ When any one or more of facial expressions such as “sadness” and head gestures such as “neck wagging”, “swinging”, and “no sleigh” are recognized, it is determined that the recognition is erroneous. In this embodiment, a surprised facial expression is recognized, and the recognition result by the voice recognition unit 12 is determined to be erroneous recognition.

  Here, the facial expression recognition method may be any known method, for example, a method described in Document 1 (Japanese Unexamined Patent Application Publication No. 2008-146318 “Emotion Estimation Device”). Specifically, each facial expression to be recognized (laughter, surprise, anger, discouragement, fear, sadness, normal state) is learned in advance by using a neural network for the feature amount (expression map) of each facial expression. Next, the similarity between the facial image acquired by the facial image acquisition unit 15 as user reaction data and the facial expression map is calculated, and the highest similarity is adopted as the facial expression recognition result.

  The head gesture recognition method may be any known method. For example, Reference 2 (“Head gesture recognition robust to dialogue robot operation”, IEICE Transactions D Vol. J89-D No. 7 pp.1514-1522). Specifically, a large number of face image data of head gestures (neck wagging, swinging, sliding, and normal state) is collected in advance, and facial image feature points (eye corner position, nose position, etc.) for each gesture ) Is modeled using HMM (Hidden Marcov Model). Next, a head gesture is determined based on the degree of matching between the face image acquired by the face image acquisition unit 15 as user reaction data and the model by the HMM.

  In step 110, the unconscious utterance recognition unit 18 determines whether the utterance data acquisition unit 16 has acquired the user's utterance data. If the utterance data is acquired, the process proceeds to step 112, where the utterance data is acquired. If not, the process proceeds to step 114.

  In step 112, the unconscious utterance recognition unit 18 recognizes the user's utterance data acquired by the utterance data acquisition unit 16 using the voice command dictionary 21 and the unconscious utterance dictionary 22, and the words registered in the unconscious utterance dictionary 22. When one or more are recognized, it is determined as a misrecognition. In the present embodiment, it is assumed that the unconscious utterance recognition unit 18 recognizes the voice command “expanded” for the above-mentioned “what” utterance. In this case, since the unconscious utterance has not been recognized, it is determined that the recognition result by the voice recognition unit 12 is correct.

  In step 114, the misrecognition determination unit 19 determines whether the recognition result by the speech recognition unit 12 is a misrecognition based on the determination results of the face image recognition unit 17 and the unconscious utterance recognition unit 18. In this embodiment, since the face image recognition unit 17 determines that the recognition is incorrect and the unconscious utterance recognition unit 18 determines that the recognition is correct, the recognition result by the voice recognition unit 12 is determined to be erroneous recognition.

  As described above, the speech recognition misrecognition determination apparatus according to the present embodiment accurately determines whether or not the recognition result is a misrecognition from the user's reaction immediately after the speech recognition result is output to the speech input. Can do. Further, when it is determined that the recognition is erroneous, the subsequent dialogue processing can be smoothly proceeded.

  In addition, this invention is not limited to the above-mentioned embodiment, It is applicable also to what changed the design within the range described in the claim.

  For example, in the present embodiment, both the face image acquisition unit 15 and the face image recognition unit 17 and the utterance data acquisition unit 16 and the unconscious utterance recognition unit 18 are used, but the determination is made using only one of them. May be.

  In addition, when the erroneous recognition determination unit 19 determines that the user's command utterance has been erroneously recognized, the device operation may be controlled so as to stop the command operation based on the erroneous recognition.

It is a block diagram which shows the structure of the misrecognition determination apparatus of the speech recognition which concerns on embodiment of this invention. It is a figure which shows the structural example of a voice command dictionary. It is a figure which shows the structural example of an unconscious utterance dictionary. It is a flowchart which shows the flow of an effect | action of the misrecognition determination apparatus of the speech recognition which concerns on embodiment of this invention.

Explanation of symbols

DESCRIPTION OF SYMBOLS 11 Voice data input part 12 Voice recognition part 13 Command execution part 14 Recognition result output part 15 Face image acquisition part 16 Speech data acquisition part 17 Face image recognition part 18 Unconscious speech recognition part 19 False recognition determination part 21 Voice command dictionary 22 Unconscious speech dictionary

Claims (6)

Voice command recognition means for recognizing a voice command corresponding to the input voice data based on the voice data input by the user and a voice command dictionary in which voice commands corresponding to the voice data are registered;
Recognition result response means for executing a response process for the recognition result by the voice command recognition means;
Face image acquisition means for acquiring face image data of the user within a predetermined time after executing the response processing by the recognition result response means;
Utterance data acquisition means for acquiring the user's utterance data within the predetermined time;
Image recognition means for recognizing a predetermined facial expression and a predetermined head movement based on the face image data acquired by the face image acquisition means;
Unconscious utterance recognition means for recognizing an unconscious utterance corresponding to the acquired utterance data, based on the utterance data acquired by the utterance data acquisition means, and an unconscious utterance dictionary in which the unconscious utterance corresponding to the utterance data is registered,
When the predetermined facial expression or the predetermined head movement is recognized by the image recognition unit, or when an unconscious utterance is recognized by the unconscious utterance recognition unit, the recognition result by the voice command recognition unit is A misrecognition determination means for determining misrecognition;
An erroneous recognition determination device for speech recognition.   The recognition result response means is at least one of a recognition result output means for outputting a recognition result by the voice command recognition means and a device operation means for operating a device in response to the recognition result by the voice command recognition means. The misrecognition determination apparatus according to claim 1.   2. The apparatus operation control unit according to claim 1, further comprising: a device operation control unit that stops the operation of the device by the device operation unit when the recognition result by the voice command recognition unit is determined to be erroneous recognition by the misrecognition determination unit. 2. A speech recognition misrecognition determination apparatus according to 2.   The predetermined facial expression is a facial expression shown by the user when the voice command recognition means misrecognizes, such as laughter, surprise, anger, discouragement, anger, sadness, etc., and the predetermined head movement is a neck and neck The voice recognition misrecognition determination apparatus according to any one of claims 1 to 3, which is an operation indicated by the user when the voice command recognizing means such as swinging or sliding is erroneously recognized.   A speech recognition misrecognition determination program for causing a computer to function as each means constituting the speech recognition misrecognition determination apparatus according to any one of claims 1 to 4. Computer
A voice command recognition means for recognizing a voice command corresponding to the input voice data based on the voice data input by the user and a voice command dictionary in which a voice command corresponding to the voice data is registered;
A recognition result response means for executing a response process for the recognition result by the voice command recognition means;
Face image acquisition means for acquiring face image data of the user within a predetermined time after executing the response process by the recognition result response means;
Utterance data acquisition means for acquiring utterance data of the user within the predetermined time;
Image recognition means for recognizing a predetermined facial expression and a predetermined head movement based on the face image data acquired by the face image acquisition means;
Unconscious utterance recognition means for recognizing an unconscious utterance corresponding to the acquired utterance data, based on the utterance data acquired by the utterance data acquisition means, and an unconscious utterance dictionary in which unconscious utterances corresponding to the utterance data are registered, and When the predetermined facial expression or the predetermined head movement is recognized by the image recognition unit, or when an unconscious utterance is recognized by the unconscious utterance recognition unit, the recognition result by the voice command recognition unit is Misrecognition judging means for judging as misrecognition,
A misrecognition judgment program of voice recognition for functioning as

Images