JP2000075894A - Speech recognition method and apparatus, speech dialogue system, recording medium - Google Patents

Abstract

(57) [Problem] To provide a speech recognition device capable of obtaining a highly accurate recognition result even when acoustic features of recognition targets are very similar and word appearance frequencies are almost the same. SOLUTION: A first emotion information processing unit 12 selects an emotion value Qk corresponding to a prosody feature of input speech data from a prosodic sensitivity model 121. The speech recognition unit 13 performs speech recognition of the input speech data using the acoustic model 131 and the language model 132, derives a score Swn of the candidate word, and selects N as the word candidate Wn having the highest Swn. In the second sentiment information processing unit 14, the semantic sentiment model 1
41, the emotion value Rwn of each of the N word candidates Wn
Ask for. In the sentiment state integrating unit 15, N word candidates W
With respect to n, the recognition score Twn is calculated by weighting the word candidate score Swn using the emotion values Qk and Rwn.
The recognition result output unit 16 outputs a word candidate Wn having the highest recognition score Twn as a recognition result.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a voice recognition method and a voice recognition apparatus for correctly recognizing a voice of a speaker with sensibility.
And a voice interaction system to which a voice recognition device is applied.

[0002]

2. Description of the Related Art A conventional speech recognition apparatus utilizes an acoustic model held for each speech recognition unit, that is, a phoneme or a word, and a language model created based on a word appearance frequency such as a word string. Speech recognition is performed. The acoustic model is created in advance from a large amount of speech data and has acoustic characteristics of speech, and the language model is created in advance from a large amount of text data and holds the appearance frequency and chain probability of a word to be recognized. I have.

[0003] Speech recognition by such a speech recognition device is performed as follows. First, the input speech is divided for each phoneme and replaced with a feature vector. Then, each feature vector is checked against the acoustic model,
By selecting the word having the highest matching score, the input speech is converted into a phoneme / word string. At this time, the next likely word string is estimated by the language model. This allows
In addition to simply searching for phoneme strings, the recognition accuracy is increased by searching for the next occurrence. Through a series of these recognition processes, a collation score of the character string with respect to the input voice is calculated, and the character string having the maximum collation score is determined as a recognition result.

[0004]

In a conventional speech recognition apparatus, such as "upper grade" or "lower grade",
In the case where the acoustic features of the recognition targets are very similar and the word appearance frequencies are almost the same, there is a possibility that an incorrect recognition result may be obtained. In particular, when the sound is significantly deteriorated due to noise or the like, this phenomenon is remarkably observed. In addition, when speech recognition is used for dialogue processing, there are cases where recognition results inconsistent with the flow of dialogue are obtained.

Accordingly, an object of the present invention is to obtain a highly accurate recognition result even when acoustic features of recognition targets are very similar and word appearance frequencies are almost the same, and use speech recognition for interactive processing. In some cases, it is an object of the present invention to provide an improved speech recognition method capable of obtaining a recognition result consistent with the flow of a dialog. Another object of the present invention is to provide a speech recognition apparatus suitable for implementing the above speech recognition method and a recording medium for causing a computer device to execute the method. Another object of the present invention is to provide a voice interaction system to which the above voice recognition method is applied.

[0006]

The speech recognition method of the present invention for solving the above problems basically extracts prosody information from uttered speech data, estimates the kansei state of the speaker, and performs candidate recognition by speech recognition processing. By suppressing the recognition result inconsistent with the sentiment state estimated from the word candidates listed as "", highly accurate speech recognition is realized.

[0007] The emotional state referred to here is a state expressing human feelings such as pleasant, unpleasant, and emotional and emotional joy. In order to deal with very similar acoustic and linguistic features (frequency of appearance, etc.), a kansei state based on the prosodic features of the entire utterance voice and a kansei state based on the linguistic meaning of the recognition candidate word are obtained. Reduce the priority of conflicting.
This reduces recognition candidates that are inconsistent with the utterance content,
Speech recognition that matches the utterance content can be expected.

[0008] The speech recognition method of the present invention is embodied by executing the following processing on a computer device. (1) voice input processing for inputting the voice of the speaker and converting it into digital voice data; (2) prosody information extracted from the voice data obtained by the voice input processing and corresponding to the prosody information First emotion information processing for deriving emotion valence; (3) speech recognition processing for recognizing speech data obtained by the speech input processing and selecting a word candidate having a high recognition appearance rate; and (4) speech recognition processing. Second sentiment information processing for deriving the semantic emotional valence of each of the obtained word candidates; (5) for each of the word candidates obtained by the speech recognition processing, the prosody obtained by the first sentiment information processing; Weighting is performed using the emotional valence, the word candidate score obtained by the speech recognition processing, and the semantic emotional valence obtained by the second kansei information processing. (6) A recognition result output process for selecting a word candidate having the highest recognition score obtained by the above-described emotional state integration process and outputting it as a recognition result.

A recording medium according to the present invention for solving the above-mentioned other problems is a computer-readable recording medium in which a program for causing a computer device to execute the above processing is recorded.

According to another aspect of the present invention, there is provided a voice recognition apparatus for inputting voice of a speaker and converting the voice into digital voice data, and a prosody based on the input voice data obtained by the voice input means. A first sentiment information processing means for extracting information and obtaining a prosodic emotional valence corresponding to the prosody information; and recognizing input voice data obtained by the voice input means to select a word candidate having a high recognition appearance rate. Voice recognition means, second sensibility information processing means for obtaining the semantic emotional valence of each of the word candidates obtained by the voice recognition means, and the first sensibility for each of the word candidates obtained by the voice recognition means By weighting using the prosodic emotion value obtained by the information processing means, the word candidate score obtained by the voice recognition means, and the semantic emotion value obtained by the second sensitivity information processing means, Emotion state integration means for calculating a recognition score taking into account the speaker's emotion state, and a recognition result output means for selecting a word candidate having the highest recognition score obtained by the emotion state integration means and outputting it as a recognition result. It is characterized by having.

The sentiment state integrating means weights the word candidate score based on, for example, the difference between the prosodic sentiment value and the semantic sentiment value, so that the semantic sentiment contradictory to the prosodic sentiment state. The word candidate corresponding to the state is configured to be suppressed.

More specifically, the speech recognition apparatus extracts a prosodic feature vector XI from the input speech data obtained by the speech input means, and generates a prosodic feature vector corresponding to a vector Xj closest to the current feature vector Xi. The first sentiment information processing means for selecting the emotion value Qk and the input voice data obtained by the voice input means are recognized to derive a score Swn of a word as a candidate, and the N words having the highest score Swn are determined. Voice recognition means for selecting as word candidates Wn;
For the second sentiment information processing means for obtaining the semantic emotion value Rwn of each of the N word candidates Wn obtained by the voice recognition means, and for each of the N word candidates Wn obtained by the voice recognition means, The prosodic emotional value Qk obtained by the first emotional information processing means, the word candidate score Swn obtained by the voice recognition means, and the semantic emotional value Rwn obtained by the second emotional information processing means The recognition score Twn is obtained by the sentiment state integration means for calculating Twn = Swn × α (Qk−Rwn) ＾ (− β), where α and β are positive constants. Word candidate Wn with the highest recognized recognition score Twn
And a recognition result output means for outputting the selected result as a recognition result.

The first sentiment information processing means includes a prosodic feature vector Xk obtained in advance using a large amount of voice data.
A prosodic sensibility model is provided as a table showing the correspondence relationship between the prosodic sensation value Qk and the prosodic sensation value QK corresponding to the input voice data with reference to the prosody sensibility model. The second sentiment information processing means includes a semantic sentiment model as a table indicating the correspondence between the word Wj and the emotion value Rwj obtained in advance using a large amount of text data, and refers to this semantic sentiment model. To obtain the semantic emotional value Rwn of the word Wn.

In the speech dialogue system to which the speech recognition method of the present invention is applied, in the speech recognition function, "sound information (information based on acoustic features and linguistic features)" and "kansei information (information based on prosodic features)" By using the two types of information to narrow down the recognition result, recognition errors are reduced, and user-friendly voice recognition that conforms to the speaker's emotional state is enabled.

[0015]

Embodiments of the present invention will be described below in detail with reference to the drawings. FIG. 1 shows an embodiment of the speech recognition apparatus of the present invention. In this embodiment,
The concept of "kansei state" is used. "Sensitivity state"
This is a state in which human feelings such as pleasure, discomfort, and emotions are expressed in terms of the sensitivity of k categories.
The sensitivity degree of each category is represented by a continuous numerical value. For example, for the category of pleasant and unpleasant,
Discomfort is expressed as a numerical value on one axis, with “1” as pleasant and “5” as pleasant. The vector that integrates the continuous numerical values of each of these categories is collectively referred to as emotion valence (emotion valence for a word,
(There is an emotional value about the feature vector).

In FIG. 1, a voice input section 11 takes in an analog voice signal of a speaker through a microphone or the like, converts the analog voice signal into digital input voice data, and outputs the digital voice data. The input voice data obtained here is sent to the first sensitivity information processing unit 12 and the voice recognition unit 13.

As shown in FIG. 2, the first sentiment information processing section 12 is a feature vector (eg, a first formant, a second formant, etc.) representing prosody information (eg, pitch information) included in the uttered voice. Prosody model 1 showing the correspondence between the emotional value and the emotional value (emotional value of the feature vector)
21. The prosodic sensibility model 121 is a table showing the correspondence between prosodic feature vectors and emotion valences obtained in advance from a psychological experiment using a large amount of voice data. Then, a known FFT or lpc is input to the input audio data.
A prosody feature vector is extracted by performing processing such as analysis, a vector closest to the current feature vector is derived from the prosodic sensitivity model 121, and a corresponding emotion value is selected with reference to the model 121. The emotion value selected here is sent to the emotional state integrating unit 15.

The speech recognition unit 13 includes an acoustic model 131 held for each speech recognition unit (phonemes, words, etc.) and a language model 132 created based on the frequency of occurrence of words such as word strings.
It has. Then, speech recognition is performed on the input speech data using these models to derive a score of a word that is a candidate. For voice recognition, a conventional voice recognition technology can be used. At this time, one or more words having a high recognition score are selected as word candidates. The obtained word candidate is the second sentiment information processing unit 1
4 and the scores of the word candidates are sent to the affective state integrating unit 15.

As shown in FIG. 3, the second sentiment information processing unit 14 includes a semantic sentiment model 141 that indicates the correspondence between words and emotion values (emotion values of words). The semantic sensibility model 141 is a table showing the correspondence between words and emotional values obtained in advance from psychological experiments using a large amount of text data. Then, using the semantic sensitivity model 141, the emotion value of each of the word candidates passed from the speech recognition unit 13 is obtained. The emotion value obtained here is sent to the emotional state integrating unit 15.

The sentiment state integration unit 15 provides, for each of the word candidates from the speech recognition unit 13, the emotion value from the first sentiment information processing unit 12, the word candidate score from the speech recognition unit 13, and the second sentiment information. The recognition score is calculated using the emotion value from the processing unit 14. This recognition score is sent to the recognition result output unit 16. The recognition result output unit 16 selects a word candidate having the highest recognition score and outputs this as a recognition result.

The speech recognition apparatus according to the present embodiment having the above-described functions operates according to a computer program recorded on a magnetic disk, an optical or magneto-optical disk, a semiconductor memory, or the like, or a computer program transmitted through a communication medium. It can be implemented by a controlled computer device. Further, the present invention can be implemented by using a signal processor.

Next, a speech recognition method using the above speech recognition apparatus will be described. This method is specifically performed in the following procedure. First, the input voice data of the speaker is fetched from the voice input unit 11. This input voice data reflects the emotional state of the speaker.

The first sentiment information processing unit 12 extracts a prosodic feature vector XI from the input speech data, and extracts a vector Xj closest to the current feature vector Xi to the prosodic sentiment model 1.
21 and selects the corresponding emotional value Qk.
The voice recognition unit 13 performs voice recognition of the input voice data to derive a recognition score Swn of the word candidate.
Select N word candidates Wn with high wn. Then, these word candidates Wn are input to the second emotion information processing unit 14. The second sentiment information processing unit 14 obtains the emotion valence Rwn of each of the N word candidates Wn, and sends these to the sentiment state integrating unit 15.

The sentiment state integrating unit 15 determines the emotion value Qk obtained by the first sentiment information processing unit 12 and the sentiment value obtained by the speech recognition unit 13 for each of the N word candidates Wn passed from the speech recognition unit 13. Using the obtained word candidate score Swn and the emotion value Rwn obtained by the second sentiment information processing unit 14, a recognition score Twn is calculated by the following formula. At this time,
α and β are positive constants. Twn = Swn × α (Qk−Rwn) ＾ (− β)

The recognition score Twn of each of the N word candidates Wn is sent to the recognition result output unit 16. The recognition result output unit 16 selects a word candidate Wn having the highest score Twn from the following equation and outputs it as a recognition result. Wn = {Wn | argwn max (Twn)} where γ = α (Qk−Rwn) ＾ (− β), the recognition result can be expressed as follows. Wn = {Wn | argwn max (Swn × γ)}

As described above, in the present embodiment, the emotional value corresponding to the emotional state of the speaker is estimated from the input voice data,
The second kansei information processing unit 14 obtains the emotional valence of each word of the recognition result candidates obtained by the processing of the speech recognition unit 13, and the kansei state integrating unit 15 estimates the kansei estimated from the word candidates. Since the recognition result inconsistent with the state is suppressed, the recognition of the input voice data reflecting the emotional state can be performed with high accuracy.

In order to cope with very similar acoustic and linguistic features (frequency of appearance, etc.), a kansei state based on the prosodic features of the entire uttered speech and a kansei state based on the linguistic meaning of the recognition candidate word are obtained. However, the priority is reduced although the respective emotional states are inconsistent. Therefore, recognition candidates inconsistent with the utterance content can be suppressed, and speech recognition more consistent with the utterance content can be expected. Specifically, by multiplying the recognition score Swn used in the conventional speech recognition by the score γ based on the sensibility state as in the following expression, the recognition result inconsistent with the utterance content can be suppressed. Wn = {Wn | argwn max (Swn × γ)}

FIG. 4 is a diagram showing an example of a processing procedure when the above-described speech recognition method is realized by reading and executing a computer program in a computer device. In this case, the various models 121, 131, 13
2 and 141 are stored in an external recording medium.

In FIG. 4, when a speech to be recognized is inputted (S1), a feature vector Xk and an emotional value Qk corresponding to the prosodic information in the speech are obtained by referring to a prosodic kansei model (S2). Next, the recognized word candidates W1 to Wn are extracted with reference to the acoustic model and the language model, and respective recognition scores Sw1 to Swn are obtained (S3). Further, referring to the semantic sensitivity model, the word candidates W1 to Wn
The respective emotion values Rw1 to Rwn are obtained (S4). Next, for each of the word candidates W1 to Wn, the corresponding semantic emotion values Rw1 to Rwn, the word candidate scores Sw1 to Swn,
Recognition scores Tw1 to Twn are obtained from the prosodic emotional valence Qk (step S5), and the highest word candidate is selected from the recognition scores Tw1 to Twn and output as a discrimination result (step S5).
6).

Next, a speech dialogue system to which the speech recognition device of the present invention is applied will be described. FIG. 5 shows the configuration of this speech dialogue system.
1 shows a speech recognition apparatus according to the present embodiment, the configuration example of which is shown in FIG. The word sequence sequentially recognized by the speech recognition device 21 is
It is sent to the response processing device 22. This response processing device 22
Grasps the meaning from the input word string, extracts corresponding response voice data from the voice dictionary file 23, and outputs it to the voice synthesizer 24.

The voice synthesizer 24 converts the input response voice data into an analog voice signal, and outputs the voice through a speaker 25. When extracting the response voice data, the response processing device 22 selects the corresponding character information such as animation from the character file 26 and displays it on the display 27 as appropriate.

Hereinafter, as a specific example of speech recognition, a case where two words of “up” and “down” will be described. First, it is assumed that the user (speaker) has input a voice saying, "The test score has improved today." At this time, the speech recognition device 21 performs the following processing (see FIG. 1 for the codes of the functional blocks). The audio input unit 11 converts analog audio into digital input audio data. Further, as a result of the prosody feature extraction in the first sentiment information processing unit 12, a prosody feature vector Xk is obtained, and an emotion value Qk corresponding thereto is selected. This emotional value Qk is passed to the emotional state integrating unit 15.

Here, paying attention to the recognition of the "raised" part, the voice recognition unit 13 derives "raised (W1)" and "decreased (W2)" as word candidates of the recognition result. These word candidates W1 and W2 are passed to the second emotion information processing unit 14. The recognition scores Sw1 and Sw2 of each of the word candidates obtained by the voice recognition processing unit 13 are passed to the kansei information integrating unit 15.

In the second emotion information processing unit 14, the emotion value Rw1 of "raised (W1)" and the emotion value Rw2 of "decreased (W2)" are derived by comparison with the semantic sensitivity model 141. These emotion valences Rw1 and Rw2 are passed to the emotional state integrating unit 15. In the emotional state integration processing unit 15, the first
The recognition scores Tw1 and Tw2 are calculated from the processing results of the kansei information processing unit 12, the voice recognition unit 13, and the second kansei information processing unit 14, and the calculation results are passed to the recognition result output unit 16. The recognition result output unit 16 sets the larger of the recognition scores Tw1 and Tw2 as the recognition result. Here, Tw1 <Tw2, and the recognition result is “up”.

The response processing unit 22 pays attention to the recognition result “up”, extracts response voice data “good” from the voice dictionary file 23, and
4 through the speaker 25.

On the other hand, when the user voice-inputs "The test result has dropped today."
In 1, the recognition scores Tw1 and Tw2 satisfy Tw1> Tw2,
The recognition result is “reduced”. At this time, the response processing device 22 pays attention to “reduced” of the recognition result, extracts response voice data “sorry” from the voice dictionary file 23, and outputs the voice from the speaker 25 through the voice synthesis device 24. become.

Here, due to the influence of noise or the like, the portions "raised" and "decreased" become unclear, and "the test results have dropped x today" (the x portion is missing). Assume that voice is input. In this case, in the conventional method, since the appearance rates of “up” and “down” are the same, a correct response cannot be expected. On the other hand, according to the method of the present invention, the emotional state is estimated from the prosodic inflection to grasp whether the expression is a happy expression or a sad expression, and weights this as an emotional valence.

According to this method, when the expression is grasped as a happy expression, the recognition score becomes Tw1 <Tw2, the recognition result becomes "up", and a response voice of "good" is output. Become like When the expression is grasped as a sad expression, the recognition score becomes Tw1> Tw2, the recognition result becomes "reduced", and a response voice saying "Sorry." Therefore, even if the voice input becomes unclear and some basic words are lost as information, options that do not match the prosodic kansei state are excluded, and the probability of outputting a correct response is significantly improved. Become like

As described above, the voice recognition is used as the man-machine interface, and the computer recognizes the voice and executes some kind of reaction. As the reaction, an expression or the like is output as an animation. When aiming at construction, it is considered to be extremely effective to apply the present invention, which can expect a high-accuracy recognition rate even when the acoustic features of the recognition targets are very similar and the word appearance frequencies are almost the same.

[0040]

As is apparent from the above description, according to the present invention, the acoustic characteristics of the recognition target are very similar, and
There is an effect that a highly accurate recognition result can be obtained even when the word appearance frequencies are almost the same. Also, when speech recognition is used for dialog processing, there is an effect that a recognition result consistent with the flow of the dialog can be obtained.

[Brief description of the drawings]

FIG. 1 is a functional block diagram showing a configuration of an embodiment of a speech recognition device of the present invention.

FIG. 2 is a diagram showing a specific example of a prosodic kansei model used in the embodiment.

FIG. 3 is a view showing a specific example of a semantic affective model used in the embodiment.

FIG. 4 is a flowchart showing a processing flow of a voice recognition method according to the present invention.

FIG. 5 is a block diagram showing a configuration of a voice interaction system using the voice recognition device of the present invention.

[Explanation of symbols]

 DESCRIPTION OF SYMBOLS 11 Voice input part 12 1st kansei information processing part 121 Prosodic kansei model 13 Speech recognition part 131 Acoustic model 132 Language model 14 2nd kansei information processing part 141 Semantic kansei model 15 Kansei state integration part 16 Recognition result output part 21 Speech Recognition device 22 Response processing device 23 Voice dictionary file 24 Voice synthesis device 25 Speaker 26 Character file 27 Display

Claims (10)

[Claims] 1. A process of extracting prosodic information from input voice data to estimate a kansei state of a speaker, and performing recognition inconsistent with the estimated kansei state from word candidates nominated as candidates by voice recognition processing. A speech recognition method including a step of specifying a result and a step of suppressing the specified recognition result. 2. A voice input process for inputting a voice of a speaker and converting it into digital voice data; prosody information extracted from voice data obtained by the voice input process, and a prosodic emotion corresponding to the prosody information. First sensitivity information processing for deriving a value, voice recognition processing for recognizing voice data obtained by the voice input processing, and selecting a word candidate having a high recognition appearance rate, and word candidate obtained by the voice recognition processing. A second sentiment information processing for deriving a semantic emotion value of each of the following, and for each of the word candidates obtained by the speech recognition processing, a prosodic emotion value obtained by the first emotion information processing and the speech recognition processing Weighting using the word candidate score obtained by the above and the semantic emotion value obtained by the second sentiment information processing, to derive a recognition score taking into account the sentiment state of the speaker Speech recognition characterized by executing, in a computer device, a sentiment state integration process, and a recognition result output process of selecting a word candidate having the highest recognition score obtained by the sentiment state integration process and outputting the word candidate as a recognition result. Method. 3. A voice input means for inputting a voice of a speaker and converting the voice data into digital voice data, prosody information is extracted from input voice data obtained by the voice input means, and a prosody corresponding to the prosody information is obtained. A first sentiment information processing means for obtaining an emotional value; a speech recognition means for recognizing input speech data obtained by the speech input means and selecting a word candidate having a high recognition appearance rate; A second sentiment information processing means for obtaining a semantic emotion value of each of the word candidates; and a prosodic emotion value obtained by the first emotion information processing means for each of the word candidates obtained by the voice recognition means. By weighing using the word candidate score obtained by the voice recognition means and the semantic emotion value obtained by the second sentiment information processing means, a recognition score taking into account the sensibility state of the speaker is calculated. That the emotional state integration means, recognition scores obtained by the emotional state integration means elect a highest word candidates, the recognition result output as recognition result output means and the speech recognition apparatus characterized by comprising a. 4. The sensory state integrating means weights the word candidate score based on a difference between the prosodic emotional value and the semantic emotional value, so that the semantic emotional contradiction to the prosodic emotional state is obtained. The speech recognition device according to claim 3, wherein a word candidate corresponding to the state is suppressed. 5. Speech input means for inputting speech of a speaker and converting it into digital speech data; prosody feature vector XI extracted from input speech data obtained by said speech input means; A first sentiment information processing means for selecting a prosodic emotional value Qk corresponding to the closest vector Xj; and recognition of input voice data obtained by the voice input means to derive a score Swn of a word which is a candidate. The word N having the highest score Swn is determined as a word candidate Wn.
Voice recognition means, a second sentiment information processing means for obtaining a semantic emotional value Rwn of each of the N word candidates Wn obtained by the voice recognition means, and N voice information obtained by the voice recognition means. For each of the word candidates Wn, the prosodic emotional value Qk obtained by the first sentiment information processing means, the word candidate score Swn obtained by the voice recognition means, and the word sentence score Swn obtained by the second sentiment information processing means. Sentiment state integrating means for calculating a recognition score Twn using the semantic emotion valence Rwn according to Twn = Swn × α (Qk−Rwn) ＾ (− β) (where α and β are positive constants); A speech recognition apparatus comprising: a recognition result output unit that selects a word candidate Wn having the highest recognition score Twn obtained by the kansei state integrating unit and outputs the word candidate Wn as a recognition result. 6. The first sentiment information processing means includes a prosodic sentiment model as a table showing a correspondence between a prosodic feature vector Xk and an emotional value Qk obtained in advance using a large amount of voice data. The speech recognition apparatus according to claim 5, wherein the speech recognition apparatus is configured to derive a prosodic emotional value QK corresponding to the input speech data with reference to the prosodic sensitivity model. 7. The second sentiment information processing means includes a semantic sentiment model as a table indicating a correspondence between a word Wj and an emotional value Rwj obtained in advance using a large amount of text data. 6. The speech recognition apparatus according to claim 5, wherein the apparatus is configured to obtain a semantic emotion value Rwn of the word Wn with reference to the semantic sensitivity model. 8. A speech recognition apparatus according to claim 3, wherein the speech recognition apparatus outputs response speech data having a meaning corresponding to a recognition result of the speech recognition apparatus; A voice dialogue system comprising: voice output means for converting the voice into an analog voice signal and outputting the voice signal; and performing an automatic response to the input voice with a feeling. 9. A character information holding means for holding character information such as an animation with a facial expression, and a display means for displaying specific character information, wherein the response processing means includes a character corresponding to the response voice data. The information is selected from the character information holding means and is displayed on the display means.
A spoken dialogue system as described. 10. A voice input process for inputting a voice of a speaker and converting it into digital voice data, prosody information is extracted from voice data obtained by the voice input process, and a prosodic emotion value corresponding to the prosody information. A first sentiment information processing for deriving a voice recognition process; a voice recognition process for recognizing voice data obtained by the voice input process and selecting a word candidate having a high recognition appearance rate; and a word candidate obtained by the voice recognition process. A second sentiment information processing for deriving a semantic emotion value, for each of the word candidates obtained by the voice recognition processing, a prosodic emotion value obtained by the first emotion information processing and a prosodic emotion value obtained by the voice recognition processing. Weighting is performed using the word candidate score and the semantic emotional value obtained in the second sentiment information processing, thereby obtaining a recognition score that takes into account the sentiment state of the speaker. State integration processing, a computer readable program recorded with a program for causing a computer to execute a recognition result output processing of selecting a word candidate having the highest recognition score obtained by the emotional state integration processing and outputting the word candidate as a recognition result recoding media.