JP6580281B1 - Translation apparatus, translation method, and translation program - Google Patents

Abstract

The present invention provides a translation device capable of estimating a speaker's emotion and reflecting it in translation with a more compact configuration than the conventional one. A translation apparatus according to the present invention includes a speech recognition unit that recognizes speech of an input speaker, and an emotion estimation unit that estimates a speaker's emotion from speech information recognized by the speech recognition unit. And a translation unit that translates the text information recognized by the speech recognition unit into translated text information of a language system different from the language system of the text information while reflecting the emotion of the speaker estimated by the emotion estimation unit. ing. [Selection] Figure 1

Description

  The present invention relates to a translation apparatus, a translation method, and a translation program.

2. Description of the Related Art Conventionally, as a translation device that translates a speaker's speech, a device that estimates a speaker's emotion and uses it for translation is known.
For example, Patent Document 1 discloses a translation device that estimates a speaker's emotion by reading the speaker's facial expression.

Japanese Patent Laid-Open No. 2-183371

  However, the conventional translation device requires an image recognition device equipped with a camera in order to read the expression of the speaker, and there is room for improvement in making the translation device compact.

  Therefore, an object of the present invention is to provide a translation device that can estimate a speaker's emotion and reflect it in translation with a more compact configuration than the conventional one.

  A translation apparatus according to the present invention includes a speech recognition unit that recognizes speech of an input speaker, an emotion estimation unit that estimates a speaker's emotion from speech information that has been speech-recognized by the speech recognition unit, and a speech A translating unit that translates text information recognized by the recognizing unit into translated text information of a language system different from the language system of the text information while reflecting the emotion of the speaker estimated by the emotion estimating unit.

  The emotion estimation unit may estimate the speaker's emotion based on the speed characteristics of the speech.

  Further, the emotion estimation unit may estimate the speaker's emotion based on the feature of the inflection of the uttered speech.

  The emotion estimation unit estimates the speaker's emotion from the speech information recognized by the speech recognition unit, using a learning model that has learned the speed characteristics and inflection characteristics of the speech speech that has been speech-recognized. Also good.

  The learning model may be SVM (support vector machine).

  In addition, the emotion estimation unit uses a learning model that has learned the speed characteristics of speech utterances recognized by speech, the characteristics of inflection, and the relationship between at least one of these and the emotion of the speaker. The emotion of the speaker may be estimated from the voice information recognized by the recognition unit.

  Further, a learning model having a function of LSTM (Long short term memory) may be used as the learning model.

  In addition, the learning model includes a feedback unit that accepts emotion estimation information that is an estimation result of the speaker's emotion by the emotion estimation unit, and a recursive learning unit that performs recursive learning using the emotion estimation information input to the feedback unit. You may prepare.

  Further, the emotion estimation unit may estimate the speaker's emotion for the speech-recognized word.

  Moreover, the speech part which utters the sound based on translation text information may be provided, and the speech part may reflect the emotion of the speaker estimated by the emotion estimation part when speaking the speech based on the translation text information.

  Further, the translation method according to the present invention includes a speech recognition step in which a computer recognizes speech of an input speaker, and an emotion for estimating a speaker's emotion from speech information recognized by the speech recognition step. A translation step for translating the text information recognized in the speech recognition step into translated text information in a language system different from the language system of the text information, while reflecting the emotion of the speaker estimated in the emotion estimation step, Execute.

  The translation program according to the present invention also includes a speech recognition function for recognizing speech of a speaker input to a computer, and an emotion for estimating a speaker's emotion from speech information recognized by the speech recognition function. A translation function that translates text information recognized in the speech recognition function into text information recognized in the speech recognition function, while reflecting the emotion of the speaker estimated by the emotion estimation function, into translated text information in a language system different from the language system of the text information, Is realized.

  In the translation apparatus of the present invention, the emotion estimation unit estimates the speaker's emotion from the speech information that the speaker speaks and the speech recognition unit recognizes. Then, the translation unit translates the text information recognized by the speech recognition unit into translated text information of a different language system while reflecting the emotion of the speaker. For this reason, a speaker's emotion can be estimated and reflected in translation with a more compact configuration than before.

It is a figure which shows a mode that a user uses the translation apparatus of this invention. It is a block diagram which shows the structural example of the translation apparatus which concerns on 1st Embodiment. It is a figure explaining the process performed in the emotion estimation part of the translation apparatus shown in FIG. It is a figure which shows the dictionary data shown in FIG. It is a figure which shows the processing flow in a translation apparatus. It is a figure explaining the process performed in the emotion estimation part of the translation apparatus which concerns on 2nd Embodiment. It is a figure explaining the learning phase of the learning model in an emotion estimation part. It is a figure explaining the estimation phase of the learning model in an emotion estimation part. It is a figure which shows the modification of learning data. It is a block diagram which shows the structural example of the translation apparatus which uses the learning data shown in FIG. It is a figure which shows the processing flow in the translation apparatus shown in FIG.

(First embodiment)
A first embodiment of the present invention will be described with reference to the drawings.
FIG. 1 is a diagram showing a configuration of translation apparatus 100 and its surroundings. The translation device 100 is a device that translates the speech of the user 10 (speaker) while reflecting the emotion of the user 10. The translation apparatus 100 can utter the translation result toward the user 10.

FIG. 2 is a block diagram illustrating a configuration example of the translation apparatus 100. The translation apparatus 100 includes an input unit 110, a storage unit 120, a processing unit 130, an utterance unit 140, and a display unit 150.
The input unit 110 has a microphone function for inputting the speech voice of the user 10. When receiving the audio data, the input unit 110 transmits the audio data to the processing unit 130.

The storage unit 120 has a function of storing various programs and various data necessary for the translation apparatus 100 to operate. The storage unit 120 is realized by various storage media such as an HDD, an SSD, and a flash memory.
The translation apparatus 100 may store various programs in the storage unit 120, execute the programs, and the processing unit 130 may execute processing as each functional unit included in the processing unit 130. These various programs cause the translation apparatus 100 to realize each function executed by the processing unit 130. Each function includes a voice recognition function, an emotion estimation function, and a translation function, which will be described later.

The storage unit 120 translates various programs as a voice recognition program for recognizing text information from input voice data, an emotion estimation program for estimating the emotion of the user 10 from the uttered voice, and the recognized text information. For memorizing translation programs. In this description, text information is a character string that is recognized by being divided into meaningful units by a speech recognition unit, which will be described later, and is a concept that includes words, idioms, or short sentences that have meaning.
The storage unit 120 also includes dictionary data 121 used by the translation unit 133 (described later) during translation.

As shown in FIG. 4, the dictionary data 121 stores text information and translation text information in association with corresponding emotion information.
The dictionary data 121 stores text information and a plurality of translated text information into which the text information is to be translated in association with each attribute of the speaker's emotion.
The text information is information indicating the text recognized by the voice recognition unit 131. The translated text information is text information of a word obtained by translating text information specified by text information and emotion information. Further, text information and translated text information may be stored for each dialect. Here, a dialect is a colloquial expression used in a limited manner within a predetermined area.

Emotion information is information indicating a speaker's emotion when a word corresponding to text information is uttered. Emotion information is classified into five categories, for example, emotions and normality. That is, for each word, it indicates an attribute of what kind of emotion the user 10 is used.
In addition, emotion information may not be stored for words that are used regardless of the emotion of the user 10.

  The processing unit 130 illustrated in FIG. 2 controls each unit of the translation apparatus 100, and may be a central processing unit (CPU), a microprocessor, an ASIC, an FPGA, or the like, for example. In addition, the process part 130 is not restricted to these examples, What kind of thing may be sufficient.

The processing unit 130 includes a voice recognition unit 131, an emotion estimation unit 132, a translation unit 133, and a voice generation unit 134.
The voice recognition unit 131 has a function of analyzing voice data transmitted from the input unit 110. The speech recognition unit 131 performs speech recognition processing including speech feature extraction and morphological analysis, which will be described later, on the transmitted speech of the user 10 and converts the speech into speech information.

The emotion estimation unit 132 estimates the speaker's emotion from the speech information recognized by the speech recognition unit 131 using the learning model in which the features of the speed of the uttered speech recognized and the features of the inflection are learned. .
The emotion estimation unit 132 estimates the emotion of the user 10 based on at least one of the speed feature of the uttered voice and the feature of inflection. This point will be described in detail below.

The learning model is, for example, SVM (Support Vector Machine).
SVM is one of pattern recognition models, and by classifying data according to attributes, it is possible to estimate the emotion of the user 10 who uttered the uttered voice. Learning and classification using SVM learning data will be described with reference to FIG.

The SVM according to the present embodiment is a classifier that estimates the emotion of a speaker who has uttered input speech based on at least the speech utterance speed and speech inflection.
For example, the SVM prepares a plurality of voice data indicating emotions to be classified as learning data in advance, and calculates the threshold value by classifying the voice data. For example, a plurality of voices indicating joy and voices indicating sadness are prepared, and the thresholds of joy and sadness are calculated by classifying them. Here, a dialect may be used as voice data prepared in advance. As described above, when a learning model prepared for each dialect is prepared using speech data for each dialect, emotion estimation considering an utterance speed and intonation peculiar to the dialect becomes possible.

Then, at the estimation stage, the speech whose emotion is to be estimated is converted into a vector having at least the speech rate and intonation as elements.
For example, when the number of seconds required for speech is used as the speech rate of “thank you” and the change in amplitude of the speech is used for inflection, the speech vector of thank- (Spectrum value). In addition, a vector may be generated by further including other elements such as a volume (or a change thereof).

  Then, the emotion of the voice is estimated by specifying (classifying) the emotion of the voice belongs to where the generated vector is located in the space defined by the SVM. SVM may use what has been learned in advance, or may use what has been learned by the translation apparatus.

As shown in FIG. 3, for example, the SVM may grasp the feature amounts of a large number of learning data using a two-dimensional vector space. In this embodiment, the feature amount in this case is a speech feature amount including the speed feature and the inflection feature of the speech voice.
In the illustrated example, a large number of learning data constitutes two classes by attributes. For example, in the vector space shown in FIG. 3A, there are a class in which the emotion of the speaker of the uttered voice is “normal” and a class in which “joy” is present. And SVM is learning the boundary which isolate | separates a learning data group linearly. Further, after learning this boundary, it is not necessary to store a large number of learning data, but they may be stored.

  Then, when new utterance voice data is input to the emotion estimation unit 132, the SVM extracts the voice feature amount of the utterance voice data, and determines where the utterance voice data is located with respect to the boundary line. Thus, it is possible to classify whether the speaker's emotion in the speech data is “normal” or “joy”.

  Further, for example, as shown in FIG. 3B, the SVM simultaneously recognizes a vector space that can be classified into a class having “normal” as emotion information and a class having “sorrow”. It may be. In this case, the first vector space for classifying whether the speaker's emotion is “normal” or “joy” and the speaker's emotion is “normal” or “sad” Or the second vector space for classifying or. The emotion estimation in this case will be described.

For example, if the speaker's emotion is “joy” in the first vector space and the speaker's emotion is “normal” in the second vector space, the speaker's emotion is estimated to be “joy”. .
Next, if the speaker's emotion is “normal” in the first vector space and the speaker's emotion is “sorrow” in the second vector space, the speaker's emotion is estimated to be “sorrow”. To do.

If the speaker's emotion is “normal” in the first vector space and the speaker's emotion is “normal” in the second vector space, the speaker's emotion is estimated to be “normal”. To do.
If the speaker's emotion is “joy” in the first vector space and the speaker's emotion is “sorrow” in the second vector space, the emotion is estimated by the following judgment.

  That is, in this case, the position of the speech feature vector of the speech data is close to the median value of the data group corresponding to “joy” in the first vector space, or “ It is judged whether it is close to the median value of the data group corresponding to “sorrow”, and an attribute closer to the median value of the data group is adopted. Thereby, a plurality of emotions can be classified. Further, as described above, a plurality of vector spaces may not be provided, and a single vector space may have a class for classifying into a plurality of emotion information.

  Moreover, the emotion estimation part 132 estimates a speaker's emotion with respect to the speech-recognized word. At this time, the emotion estimation unit 132 may perform emotion estimation for all words. Moreover, the emotion estimation part 132 may extract the word in which especially emotion was expressed among uttered voices based on the text information converted by the voice recognition part 131, and may perform emotion estimation only for the word.

The translation unit 133 translates the text information recognized by the speech recognition unit 131 into translated text information of a language system different from the language system of the text information while reflecting the emotion of the user 10 estimated by the emotion estimation unit 132.
When the text information recognized by the speech recognition unit 131 is transmitted, the translation unit 133 searches the translated text information corresponding to the text information with reference to the text information of the dictionary data 121 shown in FIG. Then, the translated text information is specified with reference to emotion information corresponding to the emotion estimated by the emotion estimation unit.

For example, as shown in FIG. 4, when the voice text recognized by the voice recognition unit 131 is “oi” and the emotion estimated by the emotion estimation unit is “joy”, the translation unit , “Hi” is specified as the translated text of this text. As a result, the translation unit 133 reflects the emotion of the user 10 in the translation work.
The translation unit 133 transmits the translated text information and emotion information to the voice generation unit 134.

The voice generation unit 134 generates robot utterance voice data that utters toward the user 10 based on the translated text information. The robot utterance voice data generates robot utterance voice data based on the translated text information transmitted from the translation unit 133.
The utterance unit 140 has a function as a speaker that utters the robot utterance voice data generated by the voice generation unit 134.

  Here, the utterance unit 140 may utter the speech based on the translated text information, for example, monotonously, or may utter while reflecting the speaker's emotion estimated by the emotion estimation unit 132. In the latter case, the voice generation unit 134 refers to the emotion information estimated by the emotion estimation unit 132, and generates robot utterance voice data while reproducing the voice feature amount stored in association with the emotion information.

  The display unit 150 is a display that displays the translated text information translated by the translation unit 133. The display unit 150 displays text information indicating the speech information recognized by the speech recognition unit 131 together with the translated text information.

Next, a processing flow in the translation apparatus 100 will be described with reference to FIG.
As shown in FIG. 5, first, the speech of the user 10 is input to the input unit 110 of the translation apparatus 100 (S501). The input unit 110 transmits the received utterance voice to the voice recognition unit 131 and the emotion estimation unit 132 in the processing unit 130 (S502).
Next, the speech recognition unit 131 performs speech recognition by analyzing the speech of the user 10 from the detected sound as a speech recognition step. (S503). As a speech recognition process, first, an input speech waveform is cut out and a feature amount is extracted. Next, using the acoustic model and the language model, the phoneme sequence is converted into a word, and the converted content is output as text. Thereby, text information is specified as a voice recognition result. Note that the speech recognition process may be performed by other processes.
The voice recognition unit 131 transmits text information obtained by the analysis to the translation unit 133.

Next, the emotion estimation part 132 estimates a speaker's emotion from the feature-value of speech sound as an emotion estimation step. In the present embodiment, the SVM, which is a learning model included in the emotion estimation unit 132, estimates the speaker's emotion using the method described above (S504).
Next, the translation part 133 translates text information into a translation text as a translation step (S505). At this time, the translation unit 133 performs translation while reflecting the emotion of the user 10. The translation unit 133 refers to the dictionary data 121 and performs translation by specifying the translated text information corresponding to the text information transmitted from the speech recognition unit 131 and the emotion information transmitted from the emotion estimation unit 132.

Next, the display unit 150 displays the translated text information. As a result, the user 10 can confirm the translated contents reflecting the emotions by looking at the display contents on the display unit 150.
Next, the voice generation unit 134 generates robot utterance voice data from the translated text information (S506). Then, the utterance unit 140 utters the robot utterance voice data toward the user 10 as an utterance step (S507). As a result, the user 10 can confirm the translated contents reflecting the emotions by listening to the utterance contents of the utterance unit 140.

  As described above, according to the translation device 100 according to the present embodiment, the emotion estimation unit 132 estimates the emotion of the user 10 from the speech of the user 10. Then, the translation unit 133 translates the text information recognized by the voice recognition unit 131 into translated text information of a different language system while reflecting the emotion of the user 10. For this reason, it is possible to estimate the emotion of the user 10 and reflect it in the translation with a more compact configuration than before.

  In addition, since the emotion estimation unit 132 estimates the emotion of the user 10 from the speech voice utterance speed and the inflection of the user 10, compared to the configuration in which the emotion is estimated only from the word used by the user 10, it is more accurate. The emotion of the user 10 can be estimated.

  Further, since the speech recognition rate and the feature of inflection recognized by using the learning model called SVM are learned, it is possible to appropriately estimate emotions for various patterns of speech.

Further, when the utterance unit 140 utters a voice based on the translated text information while reflecting the emotion of the user 10 based on the emotion information transmitted from the translation unit 133, the utterance unit 140 is more accurately expressed. It can be reflected.
In addition, the emotion estimation unit 132 can accurately estimate the emotion of the user 10 by performing emotion information on the word to be translated.
Further, by using a learning model learned using learning data for each dialect, emotion estimation can be performed in consideration of dialect-specific speech speed, intonation, etc., and convenience is improved.

(Second Embodiment)
Next, the translation apparatus 100 according to the second embodiment of the present invention will be described with reference to FIGS. In the translation apparatus 100 according to the second embodiment, the structure of the learning model of the emotion estimation unit 132 is different from that of the first embodiment. In the following description, differences from the first embodiment will be mainly described, and description of the same configurations and effects as those of the first embodiment will be omitted.

  FIG. 6 is a diagram for explaining processing performed by the emotion estimation unit 132 of the translation apparatus 100 according to the second embodiment. FIG. 7 is a diagram for explaining the learning phase of the learning model in the emotion estimation unit 132. FIG. 8 is a diagram for explaining a learning model estimation phase in the emotion estimation unit 132.

  As shown in FIG. 6, the emotion estimation unit 132 according to the present embodiment includes the speed feature of the uttered speech that has been speech-recognized, the feature of inflection, and the relationship between at least one of these and the emotion of the speaker. The emotion of the speaker is estimated from the speech information recognized by the speech recognition unit 131 using the learning model that has learned

More specifically, the emotion estimation unit 132 has a learning model including an input layer, an intermediate layer, and an output layer.
The learning model in the present embodiment is a set of functions that output emotion estimation information obtained by estimating the emotion of the speaker of the uttered speech by inputting the uttered speech. First, a learning phase using learning data performed by a learning model will be described.

As shown in FIG. 7, in the learning phase, first, the user 10 inputs learning data including, for example, uttered voice and emotion information as a label to the learning model of the emotion estimation unit 132 (S301). That is, the learning model learns the difference between the speech speed feature and the intonation feature from the speech voice so that the emotion can be estimated. In addition, you may use a dialect as speech sound input into learning data.
The emotion information as the label is information indicating the emotion of the speaker associated with at least one of the speed feature of the uttered voice and the feature of the inflection.

  And since a learning model needs to learn various utterance voices and emotion information corresponding to it, a large amount of learning data is required. The emotion information obtained as an output in the learning model is set to four types so as to correspond to, for example, emotions. Note that the emotion information may be set by dividing it more than four types, or may be set by subdividing it.

The emotion estimation unit 132 extracts the speech feature amount including the speech speed of the speech speech and the feature of the speech speech inflection from the speech signal of the speech speech. The speech speed mainly depends on the time axis of the speech signal, and the inflection of the speech speech mainly depends on the amplitude axis of the speech signal.
Note that the audio feature amount may be a value obtained by converting the frequency spectrum into a mel band or logarithm, for example, as other values.

  Then, the learning model learns learning data (S302). In this case, the learning data is associated with at least one of the feature amount indicating the speech rate extracted from the speech and the feature amount indicating the inflection extracted from the speech, and emotion information (label) indicated by the speech. Information. The emotion information (label) associated with the learning data is given by the user 10 (operator), and here is information indicating one of emotions.

  By learning the learning data, the coefficient of each function constituting the learning model is changed. That is, the functions that make up the learning model change the respective coefficients, so that the extracted speech feature value and the emotion information value in the learning data are used to calculate the speech feature value. It is built into a learning model that leads emotion information.

The intermediate layer of the learning model has a LSTM (Long Short Term Memory) function. In the learning model having the LSTM function, the emotion of the user 10 is sequentially estimated, for example, in units of words, based on the voice feature amount of the speech voice that the user 10 has already spoken.
The intermediate layer functions as a feedback unit that accepts emotion estimation information, which is the estimation result of the speaker's emotion by the emotion estimation unit 132, and can perform backpropagation in the learning phase.

  Backpropagation refers to the case where the speech feature value in a certain learning data is input to the input layer, the calculation result calculated in the intermediate layer and output to the output layer differs from the emotion information indicated by the corresponding label. The calculation result is returned to the intermediate layer again and the calculation is performed again. At this time, the accuracy of emotion estimation by the intermediate layer can be improved by adjusting the coefficient of the intermediate layer so that the recalculation result matches the emotion information indicated by the corresponding label.

The intermediate layer may function as a recursive learning unit that performs recursive learning using the input emotion estimation information. Here, the recursive learning in the learning model is processing performed when the user 10 who actually uses the translation device 100 is not satisfied with the translated speech that is translated with respect to the input speech.
Specifically, the content to be originally translated with respect to the input speech, or the emotion information of the speech is re-input to the input layer, the input speech and the content to be translated designated by the user 10, or Using emotion information as a label, add new additional learning data to the original learning data group and create a learning model again. Thereby, the precision of emotion estimation by a learning model can be improved further.

  In this way, the learning model is in a learned state. By performing this operation on a large amount of learning data, the accuracy of the learning model is improved. Next, an estimation phase for actually estimating a speaker's emotion using a learning model will be described.

As shown in FIG. 8, in the estimation phase, first, utterance voice data is input to the learning model of the emotion estimation unit 132 (S401). The emotion estimation unit 132 extracts the speech feature amount including the speech speed in the speech speech and the feature of the speech speech inflection, and inputs the speech feature amount to the learning model.
The learning model may output, to the output layer, the accuracy of the calculation result calculated in the intermediate layer when a speech feature is input to the input layer. The accuracy is an index that quantitatively evaluates the certainty of which label the input data corresponds to, for example, “joy 0.7, anger 0.2, sorrow 0.4, comfort 0.1” Is displayed. In this case, “joy” having the largest numerical value is selected, and the emotion of the speaker of the uttered voice is estimated to be “joy” (S402). Further, instead of the accuracy, a tag indicating emotion may be output.

  In addition, when the user 10 uses the translation device 100, when the feeling estimation information estimated by the feeling estimation unit 132 is uncomfortable, the emotion estimation information can be input to the input layer as a feedback unit. . Then, using the emotion estimation information input to the input layer, the intermediate layer as the recursive learning unit can perform recursive learning so that emotion estimation information intended by the user 10 can be obtained.

(Modification of learning data)
Next, a modified example of the learning data will be described with reference to FIG.
As shown in FIG. 9, the learning data according to the modification has translation text information as a label instead of emotion information. In this case, the emotion of the speaker is included in the word indicated by the translated text information.

  And a learning model learns with the learning data which has such a label. In this case, the function constituting the learning model changes each coefficient, thereby using the extracted voice feature value and the translation text information in the learning data to obtain the voice feature. It is built into a learning model that derives translated text information from quantity.

A configuration example of the translation apparatus 100B using such learning data will be described with reference to FIG.
As shown in FIG. 10, the storage unit 120B of the translation apparatus 100B does not include dictionary data. In the processing unit 130B, the emotion estimation unit 132B is included in the translation unit 133B. A processing flow of translation apparatus 100B in this case will be described with reference to FIG. In this description, only parts different from the processing flow described above will be described.

  As shown in FIG. 11, the translation unit 133B translates the voice data into translated text information reflecting emotion information (S604). This point will be described in detail. When speech speech is input to the emotion estimation unit 132B of the translation unit 133B and a speech feature is input to the input layer of the learning model, it indicates an emotion that is a calculation result calculated in the intermediate layer. The tag is output to the output layer. The translation unit 133B compares the tag indicating the emotion and the translation text information, and the similar translation text information is selected, thereby confirming the translation text information. Thereby, the translation which estimated the speaker's emotion of the speech voice is performed.

  As described above, according to the translation apparatus 100 according to the present embodiment, the emotion estimation unit 132 has the characteristics of the speed of the utterance speech, the feature of the inflection, and the emotion of the speaker when the utterance speech is uttered. Since a speaker's emotion is estimated using a learning model in which information is learned, the speaker's emotion can be accurately estimated.

  In addition, by using a learning model having an LSTM function as a learning model, the emotion of the user 10 is sequentially estimated, for example, in units of words, based on the speech feature amount of the uttered speech that the user 10 has already uttered. be able to. Thereby, emotion estimation can be performed more accurately with respect to voice input.

  In addition, since the learning model includes an intermediate layer that functions as a feedback unit and a recursive learning unit, emotion estimation information that is an estimation result of the emotion of the speaker by the emotion estimation unit 132 can be input to the intermediate layer again. . Then, recursive learning can be performed using the emotion estimation information input again to the intermediate layer. Thereby, the precision of emotion estimation can be improved by continued use.

It goes without saying that the apparatus according to the above embodiment is not limited to the above embodiment, and may be realized by other methods. Hereinafter, various modifications will be described.
For example, the translation apparatus 100 is not limited to translation between Japanese and English, and can be used for translation between various languages. Further, for example, it may be used for translation between a standard language and a dialect in the same Japanese language, or may be used for translation between dialects between different languages. By using dialects for translation, the translated content can be made familiar.

Translation apparatus 100 may be realized in any manner. That is, as in the above-described embodiment, it may be realized by a single device, or for example, the input unit 110, the storage unit 120, the processing unit 130, and the speech unit 140 may be realized by a plurality of different devices. .
Moreover, the translation apparatus 100 does not need to include the utterance unit 140. In this case, for example, the translation apparatus 100 may output the translated text information only on the display unit 150 without speaking the voice based on the translated text information.

Moreover, the translation part 133 may produce | generate robot speech audio | voice data. In this case, by inputting the voice data to the translation apparatus 100, the robot utterance voice data translated from the translation unit 133 is output.
Further, the algorithm used as a learning model is not limited to SVM or LSTM as long as emotion can be estimated from speech, or translation can be executed after estimating the emotion, and other algorithms can be used. Good.

  Moreover, the program of the said embodiment may be provided in the state memorize | stored in the computer-readable storage medium. The storage medium can store the program in “a tangible medium that is not temporary”. The storage medium can include any suitable storage medium such as HDD or SDD, or a suitable combination of two or more thereof. The storage medium may be volatile, non-volatile, or a combination of volatile and non-volatile. The storage medium is not limited to these examples, and any device or medium may be used as long as it can store the program.

  Note that the translation apparatus 100 can realize the functions of the plurality of functional units shown in each embodiment, for example, by reading a program stored in a storage medium and executing the read program. Further, the program may be provided to the translation apparatus 100 via any transmission medium (communication network, broadcast wave, etc.). The translation apparatus 100 implements the functions of a plurality of functional units shown in each embodiment by executing a program downloaded via the Internet or the like, for example.

  The program can be implemented using, for example, a script language such as ActionScript or JavaScript (registered trademark), an object-oriented programming language such as Objective-C or Java (registered trademark), or a markup language such as HTML5.

  At least a part of the processing in the translation apparatus 100 may be realized by cloud computing including one or more computers. In addition, each functional unit of the translation apparatus 100 may be realized by one or a plurality of circuits that realize the functions shown in the above-described embodiment, or the functions of a plurality of functional units may be realized by one circuit. Good.

  Moreover, although the embodiments of the present disclosure have been described based on the drawings and examples, it should be noted that those skilled in the art can easily make various modifications and corrections based on the present disclosure. Therefore, it should be noted that these variations and modifications are included in the scope of the present disclosure. For example, the functions included in each means, each step, etc. can be rearranged so that there is no logical contradiction, and a plurality of means, steps, etc. can be combined or divided into one. . Moreover, it is good also as combining suitably the structure shown to each embodiment.

(Appendix)
The translation apparatus having the above-described configuration may be configured as follows.
A speech recognition unit that recognizes speech of the input speaker,
While estimating the speaker's emotion from the speech information speech-recognized by the speech recognition unit, the text information recognized by the speech recognition unit reflects the estimated speaker's emotion, and the language of the text information And a translation unit that translates into translated text information of a language system different from the system.

  The translation unit uses a learning model that learns translated text information that reflects the speed characteristics, inflection characteristics, and at least any one of these and the emotions of the speaker. In the preceding paragraph, the speech information recognized by the speech recognition unit is translated into translated text information of a language system different from the language system of the text information while estimating a speaker's emotion. Translation device.

100 translation device 121 dictionary data 131 speech recognition unit 132 emotion estimation unit 133 translation unit 140 utterance unit 150 display unit

Claims (11)

A speech recognition unit that recognizes speech of the input speaker,
An emotion estimation unit that estimates a speaker's emotion from the voice information voice-recognized by the voice recognition unit;
A translation unit that translates text information recognized by the speech recognition unit into translated text information of a language system different from the language system of the text information, while reflecting the emotion of the speaker estimated by the emotion estimation unit; Prepared ,
The text information and the translated text information include a dialect as a colloquial expression used limitedly within a predetermined region,
The emotion estimation unit estimates a speaker's emotion from the speech information recognized by the speech recognition unit using a learning model that learns the speed characteristics and inflection characteristics of the speech speech that has been speech-recognized. ,
The translation apparatus, wherein the learning model is learned for each dialect using speech data for each dialect. The translation apparatus according to claim 1 , wherein the learning model is an SVM (Support Vector Machine). A speech recognition unit that recognizes speech of the input speaker,
An emotion estimation unit that estimates a speaker's emotion from the voice information voice-recognized by the voice recognition unit;
A translation unit that translates text information recognized by the speech recognition unit into translated text information of a language system different from the language system of the text information, while reflecting the emotion of the speaker estimated by the emotion estimation unit; Prepared ,
The text information and the translated text information include a dialect as a colloquial expression used limitedly within a predetermined region,
The emotion estimation unit uses the learning model that learns the speed characteristics, the inflection characteristics, and the relationship between at least one of these and the emotion of the speaker, by using speech recognition. Estimate the speaker's emotion from the speech information recognized by the
The translation apparatus , wherein the learning model is learned for each dialect using speech data for each dialect . The translation apparatus according to claim 3 , wherein a learning model having a LSTM (Long Short Term Memory) function is used as the learning model. The learning model is
A feedback unit that accepts emotion estimation information that is an estimation result of a speaker's emotion by the emotion estimation unit;
Using said emotion estimation information input to the feedback section, the translation device according to any one of claims 1, characterized in that it comprises a and a recursive learning unit for recursive learning 4. The emotion estimation unit, to the words that are speech recognition, translation apparatus according to any one of claims 1, characterized in that for estimating the emotion of the speaker 5. An utterance unit that utters a voice based on the translated text information;
The said utterance part reflects the speaker's emotion estimated by the said emotion estimation part, when uttering the audio | voice based on the said translation text information, The any one of Claim 1 to 6 characterized by the above-mentioned. Translation device. Computer
A speech recognition step for recognizing speech of the input speaker,
An emotion estimation step for estimating a speaker's emotion from the voice information voice-recognized by the voice recognition step;
Translating the text information recognized in the speech recognition step into translated text information of a language system different from the language system of the text information, while reflecting the emotion of the speaker estimated in the emotion estimation step; Run ,
The text information and the translated text information include a dialect as a colloquial expression used limitedly within a predetermined region,
In the emotion estimation step, a speaker's emotion is estimated from the speech information recognized in the speech recognition step by using a learning model in which the speed feature and the inflection feature of the speech speech recognized by the speech recognition are learned. ,
The translation method, wherein the learning model is learned for each dialect using speech data for each dialect. Computer
A speech recognition step for recognizing speech of the input speaker,
An emotion estimation step for estimating a speaker's emotion from the voice information voice-recognized by the voice recognition step;
Translating the text information recognized in the speech recognition step into translated text information in a language system different from the language system of the text information, while reflecting the emotion of the speaker estimated in the emotion estimation step; Run,
The text information and the translated text information include a dialect as a colloquial expression used limitedly within a predetermined region,
In the emotion estimation step, the speech recognition is performed using a learning model that learns a speed feature, an inflection feature, and a relationship between at least one of them and a speaker's emotion. A translation method characterized in that a speaker's emotion is estimated from speech information that is speech-recognized in steps, and the learning model is learned for each dialect using speech data for each dialect. On the computer,
A voice recognition function that recognizes the voice of the input speaker,
An emotion estimation function for estimating a speaker's emotion from voice information recognized by the voice recognition function;
A translation function that translates text information recognized by the speech recognition function into translated text information of a language system different from the language system of the text information, while reflecting the emotion of the speaker estimated by the emotion estimation function. Realized ,
The text information and the translated text information include a dialect as a colloquial expression used limitedly within a predetermined region,
In the emotion estimation function, a speaker's emotion is estimated from the speech information recognized by the speech recognition function using a learning model in which the speed feature and the intonation feature of the speech speech recognized by the speech recognition are learned. ,
The translation program , wherein the learning model is learned for each dialect using speech data for each dialect . On the computer,
A voice recognition function that recognizes the voice of the input speaker,
An emotion estimation function for estimating a speaker's emotion from voice information recognized by the voice recognition function;
A translation function that translates text information recognized by the speech recognition function into translated text information of a language system different from the language system of the text information, while reflecting the emotion of the speaker estimated by the emotion estimation function. Realized,
The text information and the translated text information include a dialect as a colloquial expression used limitedly within a predetermined region,
The emotion estimation function uses the learning model that learns the speed characteristics, the inflection characteristics, and the relationship between at least one of these and the emotion of the speaker, by using speech recognition. Estimate the emotions of the speaker from the speech information recognized by the function,
The translation program, wherein the learning model is learned for each dialect using speech data for each dialect.

Images