JP2003288339A - Voice conversion device, voice conversion method, program, and medium - Google Patents

Abstract

(57) [Summary] [Problem] To realize a voice translator that is easy to carry and can be easily operated. SOLUTION: Audio input / output device 102, image output device 1
03, one or more buttons 106 and the image pointing device 105,
An arithmetic and control unit 101 for converting data on a source language input by a user into data on a target language in a speech linguistic manner and outputting the output data to the voice input / output device 102 and the image output device 103; An external large-capacity nonvolatile storage device 104 for holding a program and data for instructing the processing procedure to 101, an external data input / output terminal 107 for the arithmetic and control unit 101 to exchange data with the external device for the program, , A power supply device 108 for supplying necessary power.

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to, for example, a voice conversion device, a voice conversion method, a program, and a medium for converting a voice-input utterance of a source language into a target language and outputting the voice.

[0002]

2. Description of the Related Art Voice interpretation technology was developed as software intended for use by high-performance workstations and personal computers. If the range of conversation is limited to travel conversations, the performance will be at a practical level. Has reached. However, as a voice interpreter, in order for a general user to use it on a daily basis, hardware that is large enough to be easily carried on an overseas trip, etc., and a user interface that can be easily operated are designed and equivalent Functional software needs to be ported to that hardware.

Conventionally, the work of porting voice interpreting software to a notebook personal computer of B5 size has been underway.

On the other hand, recent advances in hardware technology have made it possible to realize a translation function by voice input, mainly for conversations used in overseas travels, using a small portable information device. Such other conventional translation function is bidirectional, and has both a conversion function from Japanese to English and a conversion function from English to Japanese, for example.

Other prior art inventions described above include a foreign language translation device (see Japanese Unexamined Patent Publication No. 8-77176) and a voice input translation device (see Japanese Unexamined Patent Publication No. 8-278972).
There is. In these inventions, the shape of the device, the arrangement of the display unit, and the contents thereof are determined so that two people of different languages can use one device in a face-to-face manner for conversation.

[0006]

However, the notebook personal computer of B5 size is not large enough for the user to carry around and use in various places. In addition, since it has to be operated with an ordinary keyboard or mouse, it is not easy to use as a user interface. Furthermore, the amount of computational resources such as CPU performance and working memory capacity required for speech recognition is
Generally, it is proportional to the size of the vocabulary to be recognized.

Since small hardware has a limited computational resource, it is difficult to implement necessary and sufficient words as a speech interpreter as a recognition target vocabulary, and there is a problem that the utility value of the speech interpreter becomes low. It was The above is the subject of the above-mentioned conventional technique (first subject).

Next, the problems of the above other prior art will be described (second problem).

That is, in the above-mentioned other prior art translation apparatus, when a small information device that fits in a pocket of clothes and its display area has a small resolution, all the information necessary for two users is obtained. It cannot be displayed. Therefore, there is a problem that usability as a translation device is reduced. Further, there is a problem that mounting a plurality of display units increases power consumption and shortens the operation time of the translation apparatus. In addition, since the translation device does not exclusively deal with the utterances of two users, there is a problem that if the utterances overlap each other, the recognition rate of the voice recognition is lowered and the performance as the translation device is lowered.

In consideration of the first problem of the above-mentioned conventional voice interpreter, the present invention can be further downsized as compared with the conventional voice interpreter, and the voice converter and voice converter can be easily operated. It is intended to provide a method, a program, and a medium.

Further, in consideration of the second problem of the above-mentioned conventional translation apparatus, the present invention makes it possible to improve the usability of the display contents as compared with the conventional speech conversion apparatus and speech conversion method. , Programs, and media are provided.

[0012]

[Means for Solving the Problems] The first invention (Claim 1)
(Corresponding to the present invention described) corresponds to a voice input unit for inputting a voice of a first language, a voice recognition unit for voice recognition of the input voice, and a recognition result of the voice. Any one of first extraction / display means for extracting and displaying one or a plurality of word strings in the first language, and a conversion target from the displayed word string to a second language And a conversion target selecting means for selecting the word string, and when all or part of the selected word string is specified, a term candidate corresponding to the specified content is extracted and displayed. Of the second candidate based on the extraction / display means, the candidate selection means for selecting one of the displayed candidates, the selected word string, and the selected candidate. The conversion target to the language is determined, and the determined conversion target is the first Conversion means for converting the audio language of the language, a speech conversion system having a.

According to a second aspect of the present invention (corresponding to the present invention according to claim 2), the first extracting / displaying means includes a plurality of word strings to be selected and the selected words. And a column having a display screen for displaying each of the columns in a predetermined area, and the second extracting / displaying unit includes the term candidates as a part of the display screen. Which is means for displaying in a window-like manner on the area of
Is a voice conversion device of the present invention.

According to a third aspect of the present invention (corresponding to the present invention according to claim 3), when the first extracting / displaying means displays the selected word string on the display screen, The voice conversion device according to the second aspect of the present invention, which is a means for displaying a part of a word string by adding information indicating that the corresponding word candidates can be displayed.

Further, a fourth aspect of the present invention (corresponding to the present invention according to claim 4) is for identifying a part of the word string in which the added information is displayed on the display screen. It is a voice conversion device according to the third aspect of the present invention, which is provided with screen display specifying means.

Further, in a fifth aspect of the present invention (corresponding to the present invention according to claim 5), the converting means defines the specified part of the word string as the selected candidate term. Is the speech conversion apparatus according to the first aspect of the present invention, in which the result of replacement is determined as the conversion target.

Further, a sixth aspect of the present invention (corresponding to the present invention according to claim 6) is a voice input step of inputting a voice of the first language, and voice recognition for recognizing the input voice. A first extracting / displaying step of extracting and displaying one or a plurality of word strings of the first language corresponding to the voice recognition result, and a second extracting step from the displayed word strings. Conversion target selection step for selecting any word string that is to be converted into the language, and when all or part of the selected word string is specified, the specified content is supported A second extraction / display step of extracting and displaying a candidate of a term to be displayed, and a candidate selection step of selecting one of the displayed candidates from the displayed candidates.
A conversion for determining a conversion target to the second language based on the selected word string and the selected candidate, and converting the determined conversion target to a speech language of the second language. And a voice conversion method including steps.

A seventh aspect of the present invention (corresponding to the present invention according to claim 7) is the voice recognition means and the first aspect of the voice conversion device according to any one of the first to fifth aspects of the present invention. Display / extraction means, the conversion target selection means, and the second display /
It is a program for causing a computer to function as all or a part of the extraction means, the candidate selection means, and the conversion means.

An eighth aspect of the present invention (corresponding to the present invention according to claim 8) includes the voice recognition step and the first display / extraction step of the voice conversion method according to the sixth aspect of the present invention. A program for causing a computer to execute all or part of the conversion target selection step, the second display / extraction step, the candidate selection step, and the conversion step.

A ninth aspect of the present invention (corresponding to the present invention according to claim 9) is a medium carrying the program according to the seventh aspect of the present invention, characterized in that it can be processed by a computer. Is.

A tenth aspect of the present invention (corresponding to the present invention according to claim 10) is a medium carrying the program according to the eighth aspect of the present invention, characterized in that it can be processed by a computer. Is.

With the above arrangement, the present invention can provide, for example, a small piece of hardware that a user can hold with one hand and easily operate with a button or a touch panel. Then, for example, it is possible to classify and hold the words included in the example sentences to be voice-interpreted, and install only a small number of words representing the class in the voice recognition unit as the recognition target vocabulary. When a sentence including a word representing a class is uttered, an example including the word can be retrieved and presented to the user. Usually, the user selects a desired example and outputs a translated voice. But if necessary,
The user may replace the word with another word in the class and output the translated voice. For example, if you want to enter "Do you have aspirin?" In Japanese, replace it with the word "medicine" that represents the class to which the word "aspirin" belongs, and say "Do you have any drug?" In Japanese. , And then replace "drug" with "aspirin". By such a stepwise operation, the utility value as a voice interpreter is maintained without implementing a large-scale recognition target vocabulary.

An eleventh aspect of the present invention (corresponding to the present invention according to claim 11) is an input section for inputting a voice of a first or second language, and (1) the first section from the input section. When a voice of a language is received, the voice is recognized, and (1-a) the voice-recognized notation data of the first language is output based on a predetermined control instruction, or (1- b)
A conversion target determined based on the recognition result obtained by the voice recognition is converted into the second language, at least notation data of the converted language is output, and (2) the second language is input from the input unit. When the voice is received, the voice is recognized, and (2-a) the voice-recognized notation data of the second language is output based on a predetermined control instruction, or (2-b) ) A translation unit that converts the conversion target determined based on the recognition result obtained by the voice recognition into the first language and outputs at least notation data of the converted language, and a translation unit of the translation unit. A support unit for supporting the determination, a display unit for displaying the notation data in the converted language output from the translation unit based on the predetermined control instruction, and the predetermined control instruction. , At least the translation department And a control unit which performs relative to the display section, a speech conversion system having a.

A twelfth aspect of the present invention (corresponding to the present invention according to claim 12) is a user who uses the first language,
Dialogue for sequentially holding the notation data based on the voice input by the input unit at the time of dialogue with another user who uses the second language, and outputting it as history information to the display unit. It is the speech conversion apparatus according to the eleventh aspect of the present invention, which includes a history management unit.

The thirteenth invention (corresponding to the invention of claim 13) is any translation from the first language to the second language or from the second language to the first language. Is provided with a language conversion direction detection unit that detects information for determining a translation direction that should be performed by the speech translation unit, and the control unit, based on the detection result, the speech translation unit to the speech translation unit. The speech conversion apparatus according to the eleventh aspect of the present invention, which specifies a translation direction and controls the input unit.

In the fourteenth aspect of the present invention (corresponding to the present invention according to claim 14), the control of the input section by the control section means that the input section is composed of a plurality of voice input sections. The voice conversion device according to the thirteenth aspect of the present invention, which is to select the voice input unit that most collects the voice of the user who speaks.

Further, in a fifteenth aspect of the present invention (corresponding to the present invention according to claim 5), the control section responds to the translation direction according to the translation direction.
The speech conversion apparatus according to the thirteenth aspect of the present invention, wherein the display content of the display unit is controlled so as to rotate substantially 180 degrees with respect to the display screen of the display unit.

According to a sixteenth aspect of the present invention (corresponding to the present invention according to claim 16), the language conversion direction detecting section is composed of a button switch, and a user who speaks presses the button switch to translate. It is the audio conversion device according to the thirteenth aspect of the present invention, which is configured to select a direction.

According to a seventeenth aspect of the present invention (corresponding to the present invention according to claim 17), the language conversion direction detecting section is an angle sensor for detecting a direction having the best acoustic directivity of the movable microphone. The speech input translation device according to the thirteenth aspect of the present invention, which is configured so that a speaking user changes the direction of a microphone to select the translation direction.

In the eighteenth aspect of the present invention (corresponding to the present invention according to claim 18), the language conversion direction detection section is constituted by a gyro sensor installed inside the voice conversion device, The voice conversion device according to the thirteenth aspect of the present invention, wherein the user selects the translation direction at a position where the user holds the voice conversion device.

In the nineteenth aspect of the present invention (corresponding to the present invention according to claim 19), the language conversion direction detecting section is constituted by a sound source direction detecting device of an input section constituted by a microphone array unit. The speech conversion apparatus according to the thirteenth aspect of the present invention, wherein the translation direction is selected according to the vocalization position of the user who speaks with respect to the microphone array unit.

The twentieth present invention (corresponding to the present invention according to claim 20) inputs the voice of the first or second language,
When the voice input step of outputting and (1) the voice of the first language output by the voice input step is received, the voice is recognized, and based on a predetermined control instruction, (1-a ) Outputting the voice-recognized notation data of the first language, or (1-b) converting a conversion target determined based on the recognition result of the voice recognition into the second language, and When at least notation data of the converted language is output, and (2) when the voice of the second language output by the voice input step is received, it is voice-recognized and based on a predetermined control instruction. , (2
-A) outputting the speech-recognized notation data of the second language, or (2-b) converting the conversion target determined based on the recognition result of the speech recognition into the first language. A translation step of outputting at least notation data of the converted language, a support step of supporting the determination of the conversion target in the translation step, and the conversion of the converted language output by the translation step. A voice conversion method comprising: a display step for displaying notation data based on the predetermined control instruction; and a control step of performing the predetermined control instruction on at least the translation step and the display step. is there.

The 21st aspect of the present invention (corresponding to the present invention according to claim 21) is that the translation section and the support section of the speech conversion apparatus according to any one of the 11th to 19th aspects of the present invention. A program for causing a computer to function as all or a part of the display unit and the dialogue history management unit.

The 22nd aspect of the present invention (corresponding to the invention of claim 22) is the translation step, the support step, and the display step of the speech conversion method according to the 20th aspect of the invention. It is a program for causing a computer to execute all or part of the dialogue history management step.

The twenty-third aspect of the present invention (corresponding to the present invention according to claim 23) is a medium carrying the program according to the twenty-first aspect of the present invention, characterized in that it can be processed by a computer. Is.

A twenty-fourth aspect of the present invention (corresponding to the present invention according to claim 24) is a medium carrying the program according to the twenty-second aspect of the present invention, characterized in that it can be processed by a computer. Is.

According to the above configuration of the present invention, for example, a voice conversion device that the user holds with one hand and can easily operate with a button or a touch panel is used. Then, two facing users (one user uses the first language and the other user uses the second language), or means for manually obtaining the operation authority, or By adding a method of manually giving the operation authority to the other party or a method of automatically acquiring it, it is explicitly shown which one has the operation authority,
Moreover, a display and an input means that the user can easily operate are provided. Thereby, for example, the usability of the display content can be improved as compared with the conventional case without increasing the display power.

[0038]

BEST MODE FOR CARRYING OUT THE INVENTION The configuration and operation of a speech interpreter according to an embodiment of the speech conversion apparatus of the present invention will be described below.
Along with the description with reference to the drawings, the operation of the voice conversion method of the present invention will also be described.

FIG. 1 is a block diagram showing the hardware configuration of the speech interpreting apparatus of this embodiment.

The voice input / output device 102 receives the user's utterance in the original language and outputs the voice translated into the target language. The image output device 103 displays an example of the source language to be translated by the interpreter. The image indicating device 105 and the button 106 are used to allow the user to select the example displayed on the image output device 103. Arithmetic control device 101
Converts the data related to the source language input from the voice input / output device 102, the image instruction device 105, and the button 106 into data regarding the target language in a voice linguistic manner, and outputs the data to the voice input / output device 102 and the image output device 103. The external large-capacity non-volatile storage device 104 holds a program and data for instructing the arithmetic and control unit 101 about a processing procedure. The external data input / output terminal 107 is connected to the arithmetic and control unit 10
1 is used to exchange programs and data with external devices. The power supply device 108 supplies power necessary for driving the arithmetic and control unit 101.

Here, the voice input means of the present invention corresponds to the voice input device 102, and the first extraction / display means and the second extraction / display means of the present invention are the image output device 103.
And the components including the arithmetic and control unit 101 and the like.
Further, the screen display specifying means of the present invention is the image indicating device 105.
And button 106. Further, the first language of the present invention corresponds to the source language of the present embodiment, and the second language of the present invention corresponds to the target language of the present embodiment.

FIG. 2 shows a concrete example of the configuration in which a PC / AT compatible mother board is used for the arithmetic and control unit 101. The voice input / output device 203 is connected using the USB terminal of the motherboard 201. The image output device 204 is the mother board 2
Connect using the 01 digital RGB interface terminal. A 2.5-inch hard disk drive 202 is used as the external large-capacity nonvolatile storage device 104, and is connected to the mother board 201 through an IDE interface. A flash memory disk may be used instead of this hard disk drive. In addition, the power supply device 108 has a Li-ion
A secondary battery 208 is used to supply + 5V and + 12V voltages to the motherboard 201. Among the input / output terminals of the motherboard 201, an analog display output terminal, a local area network terminal, and a keyboard connection terminal are drawn out to form an external data input / output terminal 207.

A detailed structure of the image output device 204 is shown in FIG. A backlight of a cold cathode ray tube is mounted on the back surface to connect to an LCD unit 301 having a display area of 4 inches and a resolution of VGA using 18 bits from a digital RGB interface of a motherboard 302. Also, a video synchronization signal and a backlight control signal are connected.

FIG. 4 shows the detailed structure of the image pointing device 205 and the button 206. The 3.8-inch pressure-sensitive touch panel 402 is connected to the touch panel controller 401, and the X and Y coordinates of the indicated position are converted into RS232C standard serial data and the serial terminal COM1 of the motherboard 405 is converted.
Connect to. The buttons 403 and 404 are connected to the touch panel controller 401, and
The information of ON or OFF is added to the information of the designated position. The serial data received by the device driver software of the touch panel controller 401 mounted on the motherboard 405 is decoded, and the button 403 corresponds to the left button when the mouse is connected to the motherboard 405 and the button 404 corresponds to the right button. A mouse click event occurs.

The detailed structure of the voice input / output device 203 is shown in FIG.
Shown in. The USB audio interface 504 converts the input analog voice into digital data and transmits the digital data to the motherboard 505.
(Corresponding to 1) is converted to analog voice. USB interface is used for sending and receiving digital data. Microphone 5 for collecting analog voice
Use 03. The output of the USB interface 504 is amplified by the audio amplifier 502 and output from the speaker 501. An audio interface mounted on the motherboard 505 may be used instead of the USB audio interface 504.

FIG. 6 shows a perspective view of an example in which the configuration of FIG. 2 is mounted in a housing that can be held by a user with one hand, and three views thereof are shown in FIGS. 7 (a) to 7 (c). . Main housing 601
An image instruction device 205, an image display device 204, and a button 205 are mounted on the. Reference numerals 603 and 604 correspond to the button 403 and the button 404, respectively. The voice input / output device 203 is mounted on the sub-housing 602. If the interpreter is not used, the image display device 20
The sub-casing 602 covers and protects the display surface of No. 4.

When this interpreter is used, the sub-housing 802 is connected to the voice input / output device 203 (microphone 80 as shown in FIG. 8).
It is used after the directivity direction of 3) is moved to a predetermined position facing the user's face. The three views are shown in FIGS. 9 (a) to 9 (c). That is, the speaker 804 mounted on the sub-housing 802 is lifted until it faces the user, and the microphone 803 is also lifted. In this state, the LCD 805 with a touch panel can be used.

The main housing 60 is shown in FIGS. 10 (a) to 10 (c).
The state of implementation for 1 is shown. The 4-inch VGA LCD unit 301 and the touch panel 402 are stacked and mounted as a touch panel LCD 1005. 11A to FIG.
1 (c) shows how the sub-housing 602 is mounted.

FIG. 12 shows a software configuration diagram as an embodiment of the program and data of the present invention. In FIG. 12, a control unit 1201 gives an instruction to each component and controls the flow of data from each component, and 1202 displays information from the control unit 1201 and sends an input from the user to the control unit 1201. GUI (Graphical User Interface)
Unit 1203, a voice input unit that records the user's voice according to an instruction from the control unit 1201, 1204 is a voice recognition unit that continuously recognizes the user's voice sent from the voice input unit, and 1205 is an example of a source language and a target language. The example database 1206 that holds the correspondence of
In 05, a word class dictionary holding the words that have been classified into classes 1207, an example selection unit 1208 that selects an example from the voice recognition result sent from the control unit 1201 by referring to the example database 1205. Part 1
In accordance with an instruction from 201, a word selection unit that selects a classified word from the examples selected by the example selection unit 1207, 1209 can be changed to the classified word specified by the control unit 1201. An alternative word selecting unit for selecting an alternative word by referring to the word class dictionary 1206 for the word
The example database 1205 specified by 01
A language conversion unit for converting the target language into a target language by referring to the word class dictionary 1206 and a speech synthesis unit 1211 for synthesizing and outputting an example sentence in the target language designated by the control unit. Here, the example database 1205 to the language conversion unit 1210 are collectively referred to as the translation unit 122.
Call it 0.

Here, the voice recognition means of the present invention corresponds to the voice recognition section 1204, and the conversion target selection means of the present invention is
It corresponds to the example selection unit 1207 and the like. The screen display specifying means of the present invention corresponds to the word selecting section 1208 and the like, and the candidate selecting means of the present invention corresponds to the alternative word selecting section 1209 and the like. Further, the conversion means of the present invention corresponds to a component including a language conversion unit 1210, a voice synthesis unit 1211 and the like.

FIG. 14 shows a specific example of the example database 1205. The example corresponds to one sentence of the dialogue, and holds the correspondence between the source language and the target language, together with the information (components of the source language and the dependency relationship of the components) that is predetermined in each example. Source language words enclosed in <> indicate that they are classified words. Classified words indicate that they can be replaced with words of the same class.

FIG. 15 shows a concrete example of the word class dictionary 1206. Here, a class is a word with a high degree of abstraction such as “fruit”, and a word that belongs to a class is a word that expresses a concrete entity of the class such as “apple” or “mandarin orange”. is there. Note that the classifying abstraction degree can be changed according to the performance of the voice recognition unit 1204, so that the example selection can be efficiently performed. Further, the word class dictionary 1206 may be constructed by classifying the classes into hierarchies.

FIG. 16 shows details of the GUI unit 1202 displayed on the LCD 805 with a touch panel. Reference numeral 1601 is a translation direction designating section for designating a translation direction, 1603 is a voice recognition result display section for displaying the voice recognition result recognized by the voice recognition section 1204, and 1604 is a case example selected by the example selecting section 1207. Example candidate display unit, 16
Reference numeral 05 is an example selection result display section for displaying an example specified by the user, 1606 is a translation result display section for displaying an example converted into the target language by the language conversion section, 1607, 1608.
Correspond to buttons 806 and 807, respectively, and are input by the user. Also, LCD with touch panel 805
On the other hand, the user can make a pointing input.

FIG. 13 is a flowchart of the software of the present invention. 1301 is a step of selecting a translation direction, 1302 is a step of inputting a voice with a microphone 803 to perform voice recognition, 1303 is a step of searching an example database 1205 based on the voice recognition result, 1
304 is a step in which the user selects an example from the searched examples, and 1305 is a step in which it is determined whether the example selected in step 1304 is modified or translated.
1306 is a step of selecting a word to be corrected in the example selected in step 1304, and 1307 is a step 1
The step of outputting a list of words that can be replaced with the word to be modified selected in 306, 1308 is step 1307.
The step of selecting a word desired by the user from the list of words output in step 1309, and the step 1309 of replacing the example with the word changed in step 1308. 1310 is a step of converting the example sentence determined in step 1305 into the target language, and 1311 is voice synthesis of the example sentence converted into the target language in step 1309, and the speaker 804.
This is the step to output from.

The operation of the software of the present invention will be described below with reference to FIG.
3 and the display contents of the GUI unit 1202 displayed on the LCD 805 with a touch panel shown in FIGS. 17 to 25. As an example, a case where the user wants to translate the sentence "Are you aspirin?" Specifically, the user inputs "Is there any medicine?", And then replaces "medicine" with "aspirin". In the present invention, two types of input operations can be performed using the touch panel and the buttons, so in the following, touch panel input and button input will be described in order.

Step 130 for touch panel input
FIG. 17 shows the display contents of the GUI unit 1202 from 1 to step 1303. In step 1301, the user clicks on the translation direction designating section 1701 by touch panel input and designates the translation in the Japanese-English direction. At this time, the GUI unit 120
2 transmits the translation direction to the control unit 1201, and the control unit 120
1 instructs the voice input unit 1203 to input voice. The user uses the microphone 803 to say "Is there any medicine?" The voice input unit 1203 transmits the input voice to the voice recognition unit 1204. In step 1302, it is assumed that the voice recognition unit 1204 performs voice recognition corresponding to the designated translation direction, and transmits the recognition result including erroneous recognition "Is there a 7-day drug?" To the control unit 1201. Control unit 120
1 transmits the voice recognition result to the GUI unit 1202 and the example selecting unit 1207. The GUI unit 1202 displays the transmitted voice recognition result on the recognition result display unit 1702. On the other hand, in step 1303, the example selection unit 1207 searches for an example by the following method based on the voice recognition result, and transmits the searched example to the control unit 1201.

The example selection unit 1207 selects the example database 12 from the voice recognition result of "Is there any drug for 7 days?"
As a set of important words defined in 05, "7 days",
"Medical" and "Yes" are extracted.

Here, "7 days" is a classified word <
Days> and “drug” belongs to the classified word <drug>. "Yes" does not belong to any classified word.

The example selection unit 1207 sequentially confirms the dependency relationships of the constituent elements in FIG. 14, and selects one from the examples having the largest number of relationships among the examples in which one or more dependency relationships are satisfied. For example, with regard to the example of the example number 1, since there is no "hang" in the above set of important words, the number of established dependencies is 0.
Is. As for the example of example number 2, since "something" does not exist in the above set of important words, (1 → 2) does not hold in the dependency relation of the component, but (2 → 3) It holds (see FIG. 14). Therefore, the number of established dependencies is one.

If the example selecting unit 1207 is designed to select from the example database 1205 in which the number of dependency relationships established is 1 or more, the example of example number 1 is not selected in FIG. The example is selected. Since "something" does not exist in the set of important words, for the selected example of example number 2, output the notation "Do you have medicine?".

In the following description, the example database 120
It is assumed that the other examples in 5, "Is this medicine?" And "It is medicine" are selected in the same manner as above.

The control unit 1201 sends the example sentence sent from the example selecting unit 1207 to the GUI unit 1202. The GUI unit 1202 displays the selected example sentence in the example candidate display unit 170.
Display in 3.

The display contents of the GUI unit 1202 in step 1304 are shown in FIG. In step 1304, the user selects the example “Is there any medicine?” With the same meaning from the sentence uttered by the user by clicking 1801 on the touch panel input from the example candidates displayed in the example candidate display portion 1703. To do. At this time, the GUI unit 1202 transmits the selected example sentence to the control unit 1201.

The display contents of the GUI unit 1202 in step 1305 are shown in FIG. In step 1305, the GUI unit 1
202 displays the selected example sentence on the example result display section 1901 and clears the example candidate display section 1902. Then, choose to determine and translate the example, or modify the example to replace the classified word with a substitutable word. At this time, the user can determine the example by clicking the example result display unit 1901 on the touch panel. The determined example is transmitted to the control unit 1201. In addition, the user can double-click the example result display portion 1901 on the touch panel to shift to the mode in which the words in the example are replaced.

When the example is determined in step 1305
The display contents of the GUI unit 1202 are shown in FIG. Step 1
In 310, the control unit 1201 transmits the example “Is there any medicine?” Decided by the user to the language conversion unit 1210.
The language conversion unit 1210 converts the target language “Any medicine” using the example database 1205 and transmits the conversion result to the control unit 1201. The control unit 1201 transmits the conversion result to the GUI unit 1202 and the voice synthesis unit 1211. In step 1311, the GUI unit 1202 displays the conversion result on the interpretation result display unit 2001. On the other hand, the voice synthesizing unit 1211 performs voice synthesis on the conversion result to produce the speaker 8
Output from 04.

FIG. 21 shows the display contents of the GUI unit 1202 in step 1306. In step 1306, when the user selects the word selection mode in step 1305, the word to be changed is selected. At this time, the control unit 1201
Instructs the word selection unit 1208 to select a word. The word selection unit 1208 extracts the word “medicine” that is classified into classes from the examples, and sends it to the control unit 1201. Control unit 1
201 transmits the word to the GUI unit 1202, and the GUI unit 120
2 is “medicine” displayed in the example result display portion 2101
Is underlined to indicate to the user that the word can be changed. The user clicks the word "medicine" to be corrected by touch panel input. The GUI unit 1202 transmits the selected word to the control unit 1201.

FIG. 22 shows the display contents of the GUI unit 1202 in step 1307. In step 1307, a list of alternative words for the word “medicine” designated by the user in step 1306 is displayed. The control unit 1201 transmits the word “medicine” designated by the user to the alternative word selection unit 1209. The alternative word selection unit 1209 refers to the word class dictionary 1206 shown in FIG. 15, and refers to a word in the same class as the word “medicine” specified by the user • “aspirin” • “cold drug” • “troche” • “gastrointestinal drug” Is extracted and transmitted to the control unit 1201. Control unit 1201
Sends a list of alternative words to the GUI unit 1202, and the GUI unit 1
202 displays a list of alternative words in a list window 2201.

FIG. 23 shows the display contents of the GUI unit 1202 in step 1308. In step 1308, the desired word is selected from the alternative word list shown in the list window 2201. At this time, the GUI unit 1202 uses the user's touch panel input to input the alternative word 23 desired by the user.
Alternate word "Aspirin" by clicking 01
Is acquired and transmitted to the control unit 1201.

The display contents of the GUI unit 1202 in step 1309 are shown in FIG. In step 1309, the example is changed to "Is there aspirin?" By the designated alternative word "Aspirin". After that, the GUI unit 1202 changes the example displayed in the example result display unit 2401 to “Are you aspirin?” And displays it. Then, the process returns to step 1305.

In FIG. 25, steps 1305 to 1308 are repeated, and when the user selects the example decision in step 1305, “Is there aspirin?” Is converted into the target language “Any aspirin” and the synthesized speech is output. G
This is the display content of the UI unit 1202.

Next, the case of button input will be described. In the following description, SW1 physically corresponds to the button 806 and SW2 physically corresponds to the button 807.

FIG. 17 shows the display contents of the GUI unit 1202 from step 1301 to step 1303. In step 1301, SW1 is clicked to specify the Japanese-English translation, and SW2 is clicked to specify the English-Japanese translation. In this case, click SW1 to specify Japanese-English translation. At this time, the GUI unit 120
2 transmits the translation direction to the control unit 1201, and the control unit 120
1 instructs the voice input unit 1203 to input voice. The user uses the microphone 803 to ask, "Is there any medicine?"
I say. The voice input unit 1203 transmits the input voice to the voice recognition unit 1204. In step 1302, it is assumed that the voice recognition unit 1204 performs voice recognition corresponding to the designated translation direction, and transmits the recognition result including erroneous recognition "Is there a 7-day drug?" To the control unit 1201. The control unit 1201 transmits the voice recognition result to the GUI unit 1202 and the example selection unit 1207. The GUI unit 1202 displays the transmitted voice recognition result on the recognition result display unit 1702. On the other hand, in step 1303, the example selection unit 1207
Transmits an example to the control unit 1201 based on the voice recognition result.

The example selection unit 1207 selects the example database 12 from the voice recognition result of "Is there any medicine for 7 days?"
As a set of important words defined in 05, "7 days",
"Medical" and "Yes" are extracted.

Here, "7 days" is a classified word <
Days> and “drug” belongs to the classified word <drug>. "Yes" does not belong to any classified word.

The example selecting unit 1207 sequentially confirms the dependency relationships of the constituent elements in FIG. 14, and selects one from the examples having the largest number of relationships among the examples in which one or more dependency relationships are satisfied. For example, with regard to the example of the example number 1, since there is no "hang" in the above set of important words, the number of established dependencies is 0.
Is. As for the example of example number 2, since "something" does not exist in the above set of important words, (1 → 2) does not hold in the dependency relation of the component, but (2 → 3) It holds (see FIG. 14). Therefore, the number of established dependencies is one.

When the example selecting unit 1207 is designed to select from the example database 1205 in which the number of dependency relationships established is 1 or more, the example of the example number 1 in FIG. The example is selected. Since "something" does not exist in the set of important words, the notation "Do you have medicine?" Is output for the selected example of Example No. 2.

In the following description, the example database 120
Other examples in 5, "Is this medicine?" And "It is medicine" are selected as described above.

The control unit 1201 sends the example sentence sent from the example selecting unit 1207 to the GUI unit 1202. The GUI unit 1202 displays the selected example sentence in the example candidate display unit 170.
Display in 3.

The display contents of the GUI unit 1202 in step 1304 are shown in FIG. In step 1304, the user selects the example “Is there any medicine?” Having the same sentence meaning from the sentence uttered by the user by button input from the example candidates displayed in the example candidate display portion 1604. The selection method is SW
Clicking 1 moves the specified line up one line, and clicking SW2 moves the specified line down one line. SW1 to select an example
Select by double-clicking on. At this time, GU
The I unit 1202 transmits the selected example sentence to the control unit 1201.

The display contents of the GUI unit 1202 in step 1305 are shown in FIG. In step 1305, the GUI unit 1
202 displays the selected example sentence on the example result display section 1901 and clears the example candidate display section 1902. Then, choose to determine and translate the example, or modify the example to replace the classified word with a substitutable word. At this time, the user can determine the example by clicking SW2 with a button input. The determined example is transmitted to the control unit 1201. It is also possible to switch to the word replacement mode in the example by clicking SW1 with a button input.
201 is transmitted.

When the example is determined in step 1305
The display contents of the GUI unit 1202 are shown in FIG. Step 1
In 310, the control unit 1201 transmits the example “Is there any medicine?” Decided by the user to the language conversion unit 1210.
The language conversion unit 1210 converts the target language “Any medicine” using the example database 1205 and transmits the conversion result to the control unit 1201. The control unit 1201 transmits the conversion result to the GUI unit 1202 and the voice synthesis unit 1211. In step 1311, the GUI unit 1202 displays the conversion result on the interpretation result display unit 2001. On the other hand, the voice synthesizing unit 1211 performs voice synthesis on the conversion result to produce the speaker 8
Output from 04.

FIG. 21 shows the display contents of the GUI unit 1202 at step 1306. In step 1306, when the user selects the word selection mode in step 1305, the word to be changed is selected. At this time, the control unit 1201
Instructs the word selection unit 1208 to select a word. The word selection unit 1208 extracts the word “medicine” that is classified into classes from the examples, and sends it to the control unit 1201. Control unit 1
201 transmits the word to the GUI unit 1202, and the GUI unit 120
2 is “medicine” displayed in the example result display portion 2101
Is underlined to indicate to the user that the word can be changed. The user selects the word "medicine" to be corrected by button input. That is, clicking SW1 moves left one word, and clicking SW2 moves right one word. In addition, it is possible to select the correction word by double-clicking SW1. GUI unit 1202
Transmits the selected word to the control unit 1201.

FIG. 22 shows the display contents of the GUI unit 1202 in step 1307. In step 1307, a list of alternative words for the word “medicine” designated by the user in step 1306 is displayed. The control unit 1201 transmits the word “medicine” designated by the user to the alternative word selection unit 1209. The alternative word selection unit 1209 refers to the word class dictionary 1206 shown in FIG. 15, and refers to the word “medicine” designated by the user, the same class of words as “aspirin”, “cold medicine”, “troche”, and “gastrointestinal medicine”. Is extracted and transmitted to the control unit 1201. Control unit 1201
Sends a list of alternative words to the GUI unit 1202, and the GUI unit 1
202 displays a list of alternative words in a list window 2201.

FIG. 23 shows the display contents of the GUI unit 1202 in step 1308. In step 1308, the desired word is selected from the alternative word list shown in the list window 2201. At this time, the GUI unit 1202 acquires the alternative word “aspirin” desired by the user by the button input by the user, and transmits it to the control unit 1201. As for the input method, clicking SW1 moves the cursor to the word one word above, and clicking SW2 moves the cursor to the word one word below. To select a word, double-click on SW1 to select it.

The display contents of the GUI unit 1202 at step 1309 are shown in FIG. In step 1309, the example is changed to "Is there aspirin?" By the designated alternative word "Aspirin". After that, the GUI unit 1202 changes the example displayed in the example result display unit 2401 to “Are you aspirin?” And displays it. Then, the process returns to step 1305.

In FIG. 25, when step 1305 to step 1308 are repeated, and the user selects example decision in step 1305 to convert “Are you aspirin?” Into the target language “Any aspirin” and output the synthesized voice. Is the display content of the GUI unit 1202.

In the above description, the user's input to the GUI unit 1202 is limited to touch panel input and button input, but it is also possible to select and determine words and examples by voice using voice recognition processing. It is possible. It is also possible to operate by combining input modalities of a touch panel, buttons, and voice. Moreover, although Japanese and English are taken as an example, the present invention can be implemented in other languages such as Chinese in a similar manner, and the present invention does not depend on the language.

[0088] Also, the word sequence of the present invention, in the above embodiment has been described with the case of a sentence including a plurality of words, for example not limited to this, one like the "Hello" It may consist of words.

Further, although the first extraction / display means and the second extraction / display means of the present invention are realized by the same display device in the above-mentioned embodiment, the present invention is not limited to this. Alternatively, the configuration may be realized by independent display devices.

As described above, the voice interpreter, which is an example of the present invention, is a voice interpreter that selects an example based on an input by voice and translates, and the hardware of the voice interpreter is: A voice input / output device is provided as a voice modality, an image output device is provided as an image modality, one or more buttons and an image pointing device are provided as a contact modality, and the voice input / output device, the image pointing device, and the An arithmetic and control unit for converting data on a source language input from a button into data on a target language in a linguistic manner, and outputting the output data to the voice input / output device and the image output device; An external large-capacity non-volatile storage device that holds a program and data that instruct a processing procedure, and the arithmetic and control unit are connected to an external device and And the external data input and output terminals for exchanging grams and data, a voice interpreting device characterized by being constituted by a power supply for supplying power necessary for driving the arithmetic and control unit.

Another example is the above arithmetic and control unit.
The above audio interpreter is characterized by using a PC / AT compatible motherboard.

Another example is the voice interpreter characterized in that a hard disk drive of 2.5 inches or less is used as the external large capacity nonvolatile memory device.

Another example is the voice interpreter characterized in that a flash memory disk is used as the external large capacity nonvolatile memory device.

Another example is a voice interpreter characterized in that a liquid crystal display device having a resolution of 240 dots or more in the vertical direction and 240 dots or more in the horizontal direction is used as the image output device. is there.

In another example, the button is 2
The above voice interpreter is characterized in that it uses a single mechanical button and is functionally equivalent to a mouse button when a mouse is connected to the motherboard.

In another example, as the image pointing device, a touch panel having the same size as the display surface of the liquid crystal display device or the size including the display screen of the previous period is used. It is a voice interpreter.

In another example, the external data input / output terminal uses a keyboard connection terminal, an analog display output terminal, or a local area network terminal among the input / output terminals of the motherboard. It is a voice interpreter.

In another example, the voice input / output device is
A USB audio interface that inputs and outputs analog audio data and digital audio data through the USB terminal of the motherboard, a microphone that collects user's utterances and gives the USB audio interface, and an audio amplifier that amplifies the output of the USB audio interface. The audio interpreting device is characterized in that it is configured by a speaker connected to the audio amplifier.

In another example, the voice input / output device is
The audio interface of the motherboard, a microphone that collects user's utterances and gives the audio interface to the audio interface, an audio amplifier that amplifies the output of the audio interface, and a speaker connected to the audio amplifier. It is the voice interpreter.

Another example is the voice interpreting device, wherein the power supply device is composed of a lithium ion secondary battery.

In another example, the voice interpreter can be held by the user in one hand, and the button can be easily operated by the thumb of the one hand.
The image pointing device can be operated with the other hand, and the direction of the normal line of the display surface of the image display device and the direction of the directivity of the audio input / output device can easily face the user's face. The voice interpreter is characterized in that it is designed.

In another example, the voice interpreting device is composed of a main casing in which buttons, an image indicating device and an image display device are mounted, and a sub casing in which a voice input / output device is mounted. When the voice interpreter is not used, the display surface of the image display device is covered and protected by the sub-housing, and when the voice interpreter is used, the sub-housing is directed to the voice input / output device. The voice interpreter is characterized in that the voice interpreter is used after being moved to a predetermined position where the direction of is facing the user's face.

Further, according to an example of the present invention, in a voice interpreter for selecting an example based on voice input and performing translation, the software of the voice interpreter has a GUI unit for inputting and outputting with a user, and a voice. , A source language input unit for performing voice recognition, a translation unit for translating the source language input from the source language input unit into a target language, and a target language translated by the translation unit for voice synthesis. A speech interpreting device comprising: a speech synthesizing unit for outputting the speech, a source language input unit, the GUI unit, the translation unit, and a control unit for controlling the speech synthesizing unit.

Another example is the above speech interpreter, characterized in that one sentence in a dialogue is used as a unit.

Another example is the above-mentioned voice interpreting device, which is characterized by holding a sentence pattern that is frequently used in travel conversation.

Further, another example is the above speech interpreting device, wherein the words included in the above example are classified together with related words that can replace the words.

In another example, the source language input section performs a continuous voice recognition on a voice input section that inputs a voice according to an instruction from the control section and a voice input from the voice input section. The above speech interpreter is characterized in that it is composed of a speech recognition unit for converting into a word string.

In another example, the translation unit includes an example database that holds correspondence between examples of the source language and the target language.
A word class dictionary that holds class information of words included in the example database; an example selection unit that selects the corresponding example from the example database based on an input from a source language input unit; and a selection made by the example selection unit. A word selecting unit that selects a word to be corrected from the selected examples, and an alternative word selecting unit that selects a word that can replace the word selected by the word selecting unit from the word class dictionary, The speech interpreting device is characterized in that it is configured by a language conversion unit that converts the target language into the target language based on the example database.

In another example, the GUI section includes a translation direction designating section for designating a translation direction on the display section, and a speech recognition result display section for displaying a speech recognition result output by the source language input section. , An example candidate display section for displaying an example selected from the example database by the example selection section, an example result display section for displaying an example selected by the user, and an example of the target language output by the language conversion section The above-mentioned voice interpreting device is characterized by comprising an interpreting result display unit.

As another example, when the user selects an example from the examples displayed in the example candidate display section, the GUI section selects the desired example by touch panel operation or button operation. It is the above-mentioned voice interpreting device characterized.

In another example, the word selection unit displays the GUI when one or more correctable words are presented to the user.
The above-mentioned voice interpreting device is characterized in that a mark is added to a modifiable word in the example result display section of the section.

Further, as another example, the mark of the correctable word is underlined, highlighted, highlighted, or made blinking. The above-mentioned voice interpreting device is characterized by displaying.

In another example, when the user selects a correction word, the word selection unit determines the GUI unit by touch panel operation, button operation, or voice operation by voice recognition. The above voice interpreter.

In another example, the alternative word selection unit is
When selecting an alternative word, the alternative word selecting unit acquires an alternative candidate list using a word class dictionary, and displays the alternative candidate list arranged in a list form by a GUI unit. Is.

In another example, when selecting an alternative candidate from the alternative candidate list, touch panel operation of the GUI unit,
Alternatively, the above-mentioned voice interpretation device is characterized in that the alternative candidate is selected by a button operation or a voice operation by voice recognition.

In another example, when the GUI section can be changed to an example desired by the user, the example is determined by touch panel operation or button operation, and the language conversion section sets the target language. The above speech interpreter is characterized by performing translation and outputting the synthesized speech of the example by the speech synthesis unit.

As is clear from the above description, the small-sized hardware can be carried as a voice interpreter without difficulty when the user goes on an overseas trip.
Moreover, since the user interface can be easily operated with one hand, it can be easily used in various scenes such as shopping and restaurants. Furthermore, since a voice can be input using a word representative of a class and the example can be determined and then replaced with a related word of the same class, the utility value of the voice interpreter does not decrease even with a small recognition target vocabulary.

Next, a speech input translation apparatus according to an embodiment of the speech conversion apparatus of the present invention for solving the above-mentioned second problem in the prior art will be described with reference to the drawings.

The structure of this embodiment is shown in FIG.

As shown in the figure, a voice input unit 4101, a translation support unit 4108, a voice translation unit 4102, a display unit 410.
3. The voice output unit 4107 realizes a basic voice translation function.

Here, the internal structure of the apparatus of this embodiment has already been described in the above embodiment, and therefore detailed description is omitted here.

The configuration of this embodiment (see FIG. 26)
The following is the correspondence relationship between the above and the configuration shown in FIG. 12, for example. The voice input unit 4101 in FIG. 26 corresponds to the voice input unit 1203 in FIG. 12, and the voice translation unit 4102 corresponds to the translation unit 1220, the voice recognition unit 1204, and the like. The translation support unit 4108 and the display unit 4103 are the GUI unit 120.
2 and the like, the voice output unit 4107 has a voice synthesis unit 121.
It corresponds to 1 etc.

Next, the peculiar components of this embodiment will be described.

In FIG. 26, the language conversion direction control unit 41
Reference numeral 05 determines which of the two users has the operation authority of the translation device, controls the input form of the voice input unit 4101, specifies the translation direction for the voice translation unit 4102, and displays the display unit 4103. Specify the display content of. Here, one of the two users is Japanese (corresponding to the first language of the present invention)
, And the other uses English (corresponding to the second language of the present invention).

The language conversion direction detection unit 4104 collects information necessary for the language conversion direction control unit 4105 to determine the user who has the operation authority. The dialogue management unit 4106
The translation pairs displayed on the display unit 4103 are sequentially held, and by using the translation pairs, as a history of dialogues between users,
It is displayed on the display unit 4103 in either one of the languages (FIG. 29).
reference).

In the following example, referring to FIG.
The operation of the translation device for Japanese and English will be described, and at the same time, an embodiment of the speech conversion method of the present invention will be described.

Here, as shown in FIG. 27, for example, it is assumed that a Japanese user faces the lower side of the figure and an English user faces the upper side of the translator with the translation device sandwiched therebetween.

The speech input translation device of FIG. 27 is a speech input translation device in which the language conversion direction detecting unit 4104 is a button. As an initial state, it is assumed that the language conversion direction control unit 4105 is instructing the conversion direction from Japanese to English. The input unit 4101 includes a microphone 4202 and a microphone 4206, but the input of the microphone 4206 is blocked in order to input Japanese. The user presses the voice input button 4201 and then speaks into the microphone 4202 of the input unit 4101 (for example, "Are there any medicines?"). The uttered voice is translated by the voice translation unit 4102.

The translation operation in the translation unit 4102 has been described in the above embodiment with reference to FIG. 14 and the like, so a detailed description will be omitted here, but the outline is as follows. .

The voice translation unit 4102 has a voice input unit 410.
When a Japanese voice is received from 1, the voice recognition is performed, and one or a plurality of Japanese word strings corresponding to the recognition result are extracted and displayed on the display unit 4103 as an example candidate.

That is, the example candidates are displayed in the example candidate selection window 4203 of the display unit 4103 (for example, "Is this medicine?", "Is there any medicine?", "Is the medicine" three examples), and translation support is provided. When the user selects one of them using the section 4108 (for example, touch panel selection), the selected example is displayed in the example result window 4204 (for example, "Do you have medicine?"), And translated the example. English text is uttered from the voice output unit 4107 (for example, "Do you have medicine?"). The translation pair is sent from the display unit 4103 to the dialogue management unit 4106 (for example, the translation pair "(Do you have medicine?", "Do you have medicine?")). To prompt the other party to operate the translation device in order to obtain the other party's answer, the button 4205 which is the language conversion direction detecting unit 4104 is pressed.

The language conversion direction control unit 4105, based on the information from the language conversion direction detection unit 4104, the speech translation unit 41.
02 and the display unit 4103 are instructed in the direction of conversion from English to Japanese. The display content of the display unit 4103 is 180 as shown in FIG. 28 so that it can be easily used by facing English users.
Rotate to display in English. The input unit 4101 blocks the input of the microphone 4307 in order to input English, and the microphone 4303 is enabled. In the dialogue history window 4301, the translation manager 4106 displays the English translation pair. Specifically, as a translation pair, for example,
If the translation pair consisting of "Do you have medicine?" And "Do you have medicine?" And the translation pair consisting of "Yes" and "Yes, I do." 29
In the dialogue window 4401 shown in (4), (Sun): “Is there any medicine?” And (English): “Yes” are displayed. The user presses the voice input button 4302 and then the input unit 410
Speak into the first microphone 4303 (for example, "Yes,
certainly "). The uttered voice is translated by the voice translation unit 4102.

The translation operation in the translation unit 4102 has been described in the above embodiment with reference to FIG. 14 and the like, so a detailed description is omitted here, but the outline is as follows. .

The voice translation unit 4102 has a voice input unit 410.
When an English voice is received from 1, the voice is recognized, one or a plurality of English word strings corresponding to the recognition result are extracted, and displayed on the display unit 4103 as an example candidate.

That is, the candidates are displayed in the example candidate selection window 4304 of the display unit 4103 (for example, "Yes, I
do. "," Surely. ", and" Certainly. "), and when the user selects one of them using the translation support unit 4108 (for example, touch panel selection), the example result window 4305 is displayed. The selected example is displayed (eg "Y
es, I do. "), Japanese text translated from the example is uttered from the voice output unit 4107 (for example," Yes. ").
A translation pair is sent from the display unit 4103 to the dialogue management unit 4106 (for example, a translation pair of (“Yes”, “Yes, I do.”)). To prompt the other party to operate the translation device in order to obtain the other party's answer, the button 4306, which is the language conversion direction detection unit 4104, is pressed.

The language conversion direction control unit 4105, based on the information from the language conversion direction detection unit 4104, the speech translation unit 41.
02 and the display unit 4103 are instructed in the direction of conversion from Japanese to English. The display content of the display unit 4103 is 18 as shown in FIG. 29 so that it can be easily used by facing Japanese users.
The display will be in Japanese by rotating it by 0 °. Input unit 4101
In order to input Japanese, the input of the microphone 4405 is cut off and the microphone 4403 is enabled. In the dialogue history window 4401, the translation manager 4106 displays the Japanese translation pair. The user uses the voice input button 44
After pressing 02, speak into the microphone 4403 of the input unit 4101 (for example, "Thank you").
The uttered voice is translated by the voice translation unit 4102.

The translation operation in the translation unit 4102 has been described in the above embodiment with reference to FIG. 14 and the like, so a detailed description is omitted here, but the outline is as follows. .

That is, candidates are displayed in the example candidate selection window 4413 of the display unit 4103 (for example, one example of "Thank you."), And the user uses the translation support unit 4108 to select one of them. When selected (for example, selected with the touch panel), the selected example is displayed in the example result window 4414 (for example, "Thank you."), And the Japanese text translated from the example is uttered from the voice output unit 4107 (for example, " Thank you. "). The translation pair is sent from the display unit 4103 to the dialogue management unit 4106 (for example,
(The translation pair "Thank you", "Thank you."). Further, in order to prompt the partner to operate the translation device in order to obtain the answer from the partner, the button 4404, which is the language conversion direction detection unit 4104, is pressed.

The speech input translation device of FIG. 30 is a speech input translation device in which the language conversion direction detection unit 4104 is the tilt angle sensor 4502 of the microphone 4501. That is, whether the microphone 4501 is tilted toward the Japanese user,
An angle sensor 4502 is used to determine if the English user is leaning. In the state of FIG. 30, the language conversion direction control unit 4105 indicates the conversion direction from Japanese to English. After pressing the voice input button 4503, the user speaks into the microphone 4501 of the input unit 4101 (for example, "Are there any medicines?"). The uttered voice is translated by the voice translation unit 4102.

The translation operation in the translation unit 4102 has been described in the above embodiment with reference to FIG. 14 and the like, so a detailed description is omitted here, but the outline is as follows. .

That is, the candidates are displayed in the example candidate selection window 4504 of the display unit 4103 (for example, "Is this a drug?""Is there a drug?""It is a drug"), and the translation support unit 4108 is displayed. When the user selects one of them (using the touch panel, for example), the selected example is displayed in the example result window 4505 (for example, "Do you have any medicine?"), And the translated English text of the example. Is uttered from the voice output unit 4107 (for example, "Do you
have medicine? "). From the display unit 4103 to the dialogue management unit 4
The translation pair is sent to 106 (eg, the translation pair "(Do you have medicine?", "Do you have medicine?")). In order to ask the partner to operate the translation device in order to obtain the answer from the partner, the microphone 4501 is turned toward the English user.

The language conversion direction control unit 4105, based on the information from the language conversion direction detection unit 4104, the speech translation unit 41.
02 and the display unit 4103 are instructed in the direction of conversion from English to Japanese. The display content of the display unit 4103 is 180 as shown in FIG. 31 so that it can be easily used by facing English users.
Rotate to display in English. Dialog history window 4
In 601 the translation manager 4106 displays the English translation pair. After pressing the voice input button 4602, the user speaks into the microphone 4603 of the input unit 4101 (for example, “Yes, certainly”). The uttered voice is translated by the voice translation unit 4102.

The translation operation in the translation unit 4102 has been described in the above embodiment with reference to FIG. 14 and the like, so a detailed description is omitted here, but the outline is as follows. .

That is, the candidates are displayed in the example candidate selection window 4604 of the display unit 4103 (for example, "Yes, I
do. "," Surely. ", and" Certainly. "), when the user selects one of them using the translation support unit 4108 (for example, touch panel selection), an example result window 4605 appears. The selected example is displayed (eg "Y
es, I do. "), Japanese text translated from the example is uttered from the voice output unit 4107 (for example," Yes. ").
A translation pair is sent from the display unit 4103 to the dialogue management unit 4106 (for example, a translation pair of (“Yes”, “Yes, I do.”)). Aim the microphone 4603 toward the Japanese user to seek the answer of the other party.

The language conversion direction control unit 4105, based on the information from the language conversion direction detection unit 4104, the speech translation unit 41.
02 and the display unit 4103 are instructed in the direction of conversion from Japanese to English. The display content of the display unit 4103 is 18 as shown in FIG. 32 so that it can be easily used by facing Japanese users.
The display will be in Japanese by rotating it by 0 °. In the dialogue history window 4701, the translation manager 4106 displays the Japanese translation pair. The user inputs a voice input button 4702
After pressing, speak into the microphone 4703 of the input unit 4101 (for example, "Thank you"). The uttered voice is translated by the voice translation unit 4102.

Since the translation operation in the translation unit 4102 has been described in the above embodiment with reference to FIG. 14 and the like, a detailed description thereof will be omitted here, but the outline is as follows. .

That is, the candidates are displayed in the example candidate selection window 4704 of the display unit 4103 (for example, one example of "Thank you."), And the user uses the translation support unit 4108 to select one of them. When selected (for example, selected on the touch panel), the selected example is displayed in the example result window 4705 (for example, "Thank you."), And Japanese text translated from the example is uttered from the voice output unit 4107 (for example, " Thank you. "). The translation pair is sent from the display unit 4103 to the dialogue management unit 4106 (for example,
(The translation pair "Thank you", "Thank you."). Furthermore, in order to ask the other party to operate the translation device in order to obtain the other party's answer, the microphone 4703 is turned to the English user.

The speech input translation device of FIG. 33 is a speech input translation device in which the language conversion direction detection unit 4104 is a gyro sensor 4801 for detecting the inclination of the main body. It is assumed that the language conversion direction control unit 4105 is instructing the conversion direction from Japanese to English in the state of the gyro sensor. The input unit 4101 includes a microphone 4802 and a microphone 4803, but the input of the microphone 4803 is blocked in order to input Japanese. After pressing the voice input button 4804, the user speaks into the microphone 4802 of the input unit 4101 (for example, "Are there any medicines?"). The uttered voice is translated by the voice translation unit 4102.

The translation operation in the translation unit 4102 has been described in the above embodiment with reference to FIG. 14 and the like, so a detailed description will be omitted here, but the outline is as follows. .

That is, the candidates are displayed in the example candidate selection window 4805 of the display unit 4103 (for example, the three examples of “Is this a drug?”, “Is there a drug?” And “Is a drug”). When the user selects one of them (using the touch panel, for example), the selected example will be displayed in the example result window 4806 (eg, "Is there any medicine?"), And the translated English text of the example. Is uttered from the voice output unit 4107 (for example, "Do you
have medicine? "). From the display unit 4103 to the dialogue management unit 4
The translation pair is sent to 106 (eg, the translation pair "(Do you have medicine?", "Do you have medicine?")). When urging the other party to operate the translation device in order to obtain the answer from the other party, the other person holds the voice translation device in an upside-down manner by holding it over.

The language conversion direction control unit 4105, based on the information from the gyro sensor 4901, the speech translation unit 410.
2 and the display unit 4103 are instructed in the conversion direction from English to Japanese. The display content of the display unit 4103 is 180 ° as shown in FIG. 34 so that it can be easily used by facing English users.
It turns to display in English. The input unit 4101 blocks the input of the microphone 4902 in order to input English,
The microphone 4903 is activated. Dialog history window 49
In 04, the translation manager 4106 displays the English translation pair. After pressing the voice input button 4905, the user speaks into the microphone 4903 of the input unit 4101 (for example, "Yes, certainly"). The uttered voice is translated by the voice translation unit 4102.

The translation operation in the translation unit 4102 has been described in the above embodiment with reference to FIG. 14 and the like, so a detailed description is omitted here, but the outline is as follows. .

That is, the candidates are displayed in the example candidate selection window 4906 of the display unit 4103 (for example, "Yes, I
do. "," Surely. ", and" Certainly. "), and when the user selects one of them using the translation support unit 4108 (for example, touch panel selection), the example result window 4907 is displayed. The selected example is displayed (eg "Y
es, I do. "), Japanese text translated from the example is uttered from the voice output unit 4107 (for example," Yes. ").
A translation pair is sent from the display unit 4103 to the dialogue management unit 4106 (for example, a translation pair of (“Yes”, “Yes, I do.”)). When urging the other party to operate the translation device in order to obtain the answer from the other party, the other person holds the voice translation device in an upside-down manner by holding it over.

The language conversion direction control unit 4105, based on the information from the gyro sensor 5001, the speech translation unit 410.
2 and the display unit 4103 are instructed in the direction of conversion from Japanese to English. The display content of the display unit 4103 is 180 as shown in FIG. 35 so that it can be easily used by facing Japanese users.
Rotate to display in Japanese. The input unit 4101 is
In order to input Japanese, the input of the microphone 5002 is blocked and the microphone 5003 is activated. In the dialogue history window 5004, the translation manager 4106 displays the Japanese translation pair. The user clicks the voice input button 5005
After pressing, speak into the microphone 5003 of the input unit 4101 (for example, "Thank you"). The uttered voice is translated by the voice translation unit 4102.

The translation operation in translation section 4102 has been described in the above embodiment with reference to FIG. 14 and the like, so a detailed description is omitted here, but the outline is as follows. .

That is, the candidates are displayed in the example candidate selection window 5006 of the display unit 4103 (for example, one example of "Thank you."), And the user uses the translation support unit 4108 to select one of them. When selected (for example, selected on the touch panel), the selected example is displayed in the example result window 5007 (for example, "Thank you."), And Japanese text translated from the example is uttered from the voice output unit 4107 (for example, " Thank you. "). The translation pair is sent from the display unit 4103 to the dialogue management unit 4106 (for example,
(The translation pair "Thank you", "Thank you."). Further, in order to ask the partner to operate the translation device in order to obtain the answer, the voice translation device is presented to the partner and the partner holds it upside down.

The speech input translation device of FIG. 36 has an input unit 41.
01 and the language conversion direction detection unit 4104 are a microphone input unit 5101 capable of detecting the direction of a sound source. The microphone array unit 5101 has a function of performing sound collection with sharp directivity after specifying the direction of the sound source, and generally includes a plurality of microphone units geometrically arranged and a plurality of microphone units from each microphone unit. It is composed of an arithmetic unit which processes an output by digital signal processing and converts it into one output.

When a Japanese user starts uttering (eg, "Ano,"), the microphone array unit 5101 detects the direction of the utterer's voice, and becomes utterable. The background color of the display unit 4103 is a color that calls the user's attention while not in the utterable state (for example, red), and becomes a color for which permission has been given when the utterable state is (for example, green). Language conversion direction control unit 4 based on the information of the microphone array unit 5101
Reference numeral 105 indicates the direction of conversion from Japanese to English. When the user utters in a state in which it can be generated (for example, “Are there any medicines?”), The uttered voice is translated by the voice translation unit 4102.

The translation operation in the translation unit 4102 has been described in the above embodiment with reference to FIG. 14 and the like, so a detailed description is omitted here, but the outline is as follows. .

That is, the candidates are displayed in the example candidate selection window 5102 of the display unit 4103 (for example, three examples of “Is this a drug?”, “Is there a drug?” And “Is a drug”). When the user selects one of them (using the touch panel, for example), the selected example is displayed in the example result window 5103 (eg, "Do you have any medicine?"), And the English text that translates the example. Is uttered from the voice output unit 4107 (for example, "Do you
have medicine? "). From the display unit 4103 to the dialogue management unit 4
The translation pair is sent to 106 (eg, the translation pair "(Do you have medicine?", "Do you have medicine?")).

When the other party starts speaking to answer (for example, "Hmm,"), the microphone array unit 5201 detects the direction of the voice of the speaker and becomes ready to speak. The language conversion direction control unit 4105, based on the information from the microphone array unit 5201, the voice translation unit 4102 and the display unit 410.
Instruct 3 to change the direction from English to Japanese. The display content of the display unit 4103 is rotated by 180 ° as shown in FIG. 37 so that it is easy for a facing English user to use, and the display is in English. In the dialogue history window 5202, the translation manager 4106 displays the English translation pair. When an English user speaks while speaking is possible (for example, "Yes,
certainly "), the uttered voice is translated by the voice translation unit 4102.

The translation operation in the translation unit 4102 has been described in the above embodiment with reference to FIG. 14 and the like, so a detailed description is omitted here, but the outline is as follows. .

That is, the candidates are displayed in the example candidate selection window 5203 of the display unit 4103 (for example, "Yes, I
do. "," Surely. ", and" Certainly. "), and when the user selects one of them using the translation support unit 4108 (for example, touch panel selection), the example result window 5204 is displayed. The selected example is displayed (eg "Y
es, I do. "), Japanese text translated from the example is uttered from the voice output unit 4107 (for example," Yes. ").
A translation pair is sent from the display unit 4103 to the dialogue management unit 4106 (for example, a translation pair of (“Yes”, “Yes, I do.”)).

When the other party starts speaking to answer (for example, "A"), the microphone array unit 5301 detects the direction of the voice of the speaker and becomes ready to speak. The language conversion direction control unit 4105 is a microphone array unit 5301.
Based on the information from the voice translation unit 4102 and the display unit 41.
The direction of conversion from Japanese to English is instructed to 03.
The display content of the display unit 4103 is rotated by 180 ° as shown in FIG. 38 so that it is easy for the facing Japanese user to use, and the display is in Japanese. In the dialogue history window 5302,
The dialogue management unit 4106 displays the Japanese translation pair. When the user utters in a state where the utterance is possible (for example, “thank you”), the uttered voice is translated by the voice translation unit 4102.

The translation operation in the translation unit 4102 has been described in the above embodiment with reference to FIG. 14 and the like, so a detailed description is omitted here, but the outline is as follows. .

That is, the candidates are displayed in the example candidate selection window 5303 of the display unit 4103 (for example, one example of "Thank you."), And the user uses the translation support unit 4108 to select one of them. When selected (for example, selected on the touch panel), the selected example is displayed in the example result window 5304 (for example, "Thank you."), And Japanese text translated from the example is uttered from the voice output unit 4107 (for example, " Thank you. "). The translation pair is sent from the display unit 4103 to the dialogue management unit 4106 (for example,
(The translation pair "Thank you", "Thank you.").

It is also possible to combine the input modalities of the touch panel, the button, and the voice, or replace the button with the touch panel for operation. Moreover, although Japanese and English are taken as an example, the present invention can be implemented in other languages such as Chinese in a similar manner, and the present invention does not depend on the language.

As is clear from the above description, according to the above configuration, since the operation screen of one language is displayed on the entire surface of the display unit, the usability of the translation apparatus is maintained even if the display unit is small. It Further, it is easy to understand which of the operation authorities is given from the contents of the screen, and the utterances of the two people do not overlap. Therefore, the recognition rate of voice recognition does not decrease, and the performance of the translation device does not decrease.

In the above embodiment, the case where the translation result is displayed and the voice is output in the second language has been described. However, the present invention is not limited to this. For example, the translation result may be displayed.

Further, in the above embodiment, the case where two users use the same device so as to face each other with the translation device interposed between them is described, but the present invention is not limited to this and, for example, two users are arranged side by side. It may be configured to use the device.

Specifically, the configuration shown in FIGS. 39 and 40 is obtained. As shown in these figures, the user of the first language (for example, Japanese) uses the voice input button 4201a, and the user of the second language (for example, English) uses the voice input button 4201b. To do. In this configuration, the microphone 5
One 501 is provided in the upper center of the device. Also in this case, the same effect as that of the above-described configuration is exhibited.

In the above embodiment, for example, as shown in FIG.
7, the case where the example result is to be translated has been described, but the present invention is not limited to this, and for example, from the word string displayed in the example result window 4204, the alternative word of the word specified by the user. May be displayed, a desired word may be selected from the alternative words, and the one reflecting the selection result may be the translation target. That is,
The configuration in this case is obtained by applying the configuration described in FIGS. 21 to 24 to the configuration shown in FIG. 27 and the like.

Specifically, it has the configuration shown in FIGS. That is, a list of alternative words of the word designated by the user (FIG. 41) is displayed from the word string displayed in the example result window 4204a (FIG. 42), and the desired word is selected from the alternative words. For example, the word 2301 of aspirin is selected (FIG. 43), the example result reflecting the selection result is displayed in the example result window 4204a (FIG. 44), and it is set as the translation target. The subsequent translation operation and the like are the same as those shown in FIG. As a result, the range of objects to be translated becomes wider and usability is improved.

The present invention is a program for causing a computer to execute the functions of all or part of the means (or elements, circuits, units, etc.) of the above-described voice conversion device of the present invention, and cooperates with the computer. It is a program that operates by doing.

Further, the present invention is a program for causing a computer to execute all or some of the steps (or steps, operations, actions, etc.) of the above-described voice conversion method of the present invention. It is a program that works together.

Further, the present invention is a medium carrying a program for causing a computer to execute all or some of the operations of all or some of the steps of the voice conversion method of the present invention described above, which is readable by the computer. The read program is a medium that executes the operation in cooperation with the computer.

Further, the present invention is a medium carrying a program for causing a computer to execute all or some of the functions of all or some of the means of the above-described voice conversion apparatus of the present invention, which is readable by the computer. The read program is a medium that executes the function in cooperation with the computer.

Further, some means (or devices, elements, circuits, sections, etc.) of the present invention, and some steps (or steps, operations, actions, etc.) of the present invention are those plural means. Also, it means some means or steps in a step, or means some functions or some operations in one means or steps.

Further, a part of the device (or element, circuit, part, etc.) of the present invention means some of the plurality of devices, or one of the devices. Means a part of means (or an element, a circuit, a part, etc.) or means a part of the function of one means.

Further, one usage form of the program of the present invention is as follows:
It may be recorded in a computer-readable recording medium and operate in cooperation with the computer.

Further, one usage form of the program of the present invention is:
Transmitted through a transmission medium, read by a computer,
It may be a mode of operating in cooperation with a computer.

Further, the data structure of the present invention includes a database, a data format, a data table, a data list, a data type and the like.

Further, the recording medium includes a ROM or the like, and the transmission medium is a transmission medium such as the Internet,
Light, radio waves, sound waves, etc. are included.

The computer of the present invention described above is C
The hardware is not limited to pure hardware such as PU, but may include firmware, OS, and peripheral devices.

As described above, the structure of the present invention is
It may be realized by software or hardware.

[0186]

As is clear from the above description, the present invention has the advantages that it can be made smaller than conventional ones and can be easily operated.

Further, the present invention has an advantage that the usability of display contents can be improved as compared with the prior art.

[Brief description of drawings]

FIG. 1 is a block diagram showing a hardware configuration of a voice interpreter according to an embodiment of the present invention.

[Figure 2] Detailed block diagram of Figure 1 when using a PC / AT compatible motherboard.

FIG. 3 is a detailed block diagram of an image output device 204.

FIG. 4 is a detailed block diagram of an image pointing device 205 and a button 206.

FIG. 5 is a detailed block diagram of a voice input / output device 203.

FIG. 6 is an overall view of a casing when a voice interpreter is not used

7 (a) is a front view showing the detailed structure of the voice interpreter shown in FIG. 6; FIG. 7 (b) is a side view showing the detailed structure of the voice interpreter shown in FIG. 6; Plan view showing the detailed structure of the voice interpreter

FIG. 8 is an overall view of a casing when using a voice interpreter.

9 (a) is a front view showing the detailed structure of the voice interpreter shown in FIG. 8; FIG. 9 (b) is a side view showing the detailed structure of the voice interpreter shown in FIG. 8; Plan view showing the detailed structure of the voice interpreter

10A is a front view for showing a method of mounting each component of FIG. 2 in a main housing 801, and FIG. 10B is a front view showing a method of mounting each component of FIG. 2 in a main housing 801. 2C is a plan view for showing a method of mounting each component of FIG. 2 in the main housing 801.

11A is a front view for showing a method of mounting each constituent element of FIG. 2 in a sub-housing 802, and FIG. 11B is a front view showing a method of mounting each constituent element of FIG. 2 in a sub-housing 802. 2C is a plan view for showing a method of mounting each component of FIG. 2 in the sub-housing 802.

FIG. 12 is a block diagram showing a software configuration of a voice interpreter according to an embodiment of the present invention.

FIG. 13 is a flowchart showing the flow of software processing.

FIG. 14 is a diagram showing an example of the contents of an example database 1205.

FIG. 15 is a diagram showing an example of contents of a word class dictionary 1206.

FIG. 16 is a diagram showing the display contents of the GUI unit 1202.

FIG. 17 is a diagram showing the display contents of the GUI unit 1202 from steps 1301 to 1303.

FIG. 18 is a GUI unit 12 in the process of step 1304.
Figure showing the display contents of 02

FIG. 19 is a GUI unit 12 in the process of step 1305.
Figure showing the display contents of 02

FIG. 20 is a diagram showing the display contents of the GUI unit 1202 in the processing of steps 1310 to 1311.

FIG. 21 is a GUI unit 12 in the process of step 1306.
Figure showing the display contents of 02

FIG. 22 is a GUI unit 12 in the process of step 1307.
Figure showing the display contents of 02

FIG. 23 is a GUI unit 12 in the process of step 1308.
Figure showing the display contents of 02

FIG. 24 is a GUI unit 12 in the process of step 1309.
Figure showing the display contents of 02

FIG. 25 is a diagram showing the display contents of the GUI unit 1202 in the processing from steps 1310 to 1311.

FIG. 26 is a block diagram showing a configuration of a speech input translation device according to an embodiment of the present invention.

FIG. 27 is a diagram showing use of Japanese in a speech translation device in which the language conversion direction detection unit 4104 is a button.

FIG. 28 is a diagram showing the use of English in a speech translation device in which the language conversion direction detection unit 4104 is a button.

FIG. 29 is a diagram showing use of Japanese in a speech translation device in which the language conversion direction detection unit 4104 is a button.

FIG. 30 is a diagram showing the use of Japanese in a speech translation device in which the language conversion direction detection unit 4104 is a microphone axis angle sensor.

FIG. 31 is a diagram showing the use of English in a speech translation device in which the language conversion direction detection unit 4104 is an angle sensor for a microphone axis.

FIG. 32 is a diagram showing the use of Japanese in a speech translation device in which the language conversion direction detection unit 4104 is an angle sensor of a microphone axis.

FIG. 33 is a diagram showing the use of Japanese in a speech translation device in which the language conversion direction detection unit 4104 is a gyro sensor.

FIG. 34 is a diagram showing the use of English in a speech translation device in which the language conversion direction detection unit 4104 is a gyro sensor.

FIG. 35 is a diagram showing the use of Japanese in a speech translation device in which the language conversion direction detection unit 4104 is a gyro sensor.

FIG. 36 is an input unit 4101 and a language conversion direction detection unit 410.
FIG. 4 is a diagram showing the use of Japanese in a voice translation device in which 4 is a microphone array unit.

FIG. 37 is an input unit 4101 and a language conversion direction detection unit 410.
FIG. 4 is a diagram showing the use of a speech translation device in which English is a microphone array unit in English.

FIG. 38 is an input unit 4101 and a language conversion direction detection unit 410.
FIG. 4 is a diagram showing the use of Japanese in a voice translation device in which 4 is a microphone array unit.

FIG. 39 is a diagram for explaining the use of Japanese in a speech translation device according to another embodiment of the present invention.

FIG. 40 is a diagram for explaining the use of English in the speech translation apparatus according to another embodiment of the present invention.

FIG. 41 is a diagram for explaining the function of alternative words in the use of Japanese in the speech translation apparatus according to yet another embodiment of the present invention.

FIG. 42 is a diagram for explaining a function of an alternative word in using Japanese in a speech translation apparatus according to still another embodiment of the present invention.

FIG. 43 is a diagram for explaining the function of an alternative word in the use of Japanese in the speech translation apparatus according to yet another embodiment of the present invention.

FIG. 44 is a diagram for explaining the function of an alternative word in the use of Japanese in the speech translation apparatus according to yet another embodiment of the present invention.

[Explanation of symbols]

101 arithmetic control device 102 audio input / output device 103 image output device 104 external large-capacity non-volatile storage device 105 image instruction device 106 button 107 external data input / output terminal 108 power supply device 201 motherboard 202 2.5 inch hard disk drive 203 audio input / output device 204 image output device 205 image instruction device 206 button 207 external data output terminal 208 Li-ion secondary battery 301 4 inch VGA LCD unit with backlight 302 motherboard 401 touch panel controller 402 3.8 inch pressure sensitive touch panel 403 button 404 button 405 motherboard 501 speaker 502 Audio amplifier 503 Microphone 504 USB audio device 505 Motherboard 601 Main casing 602 Sub casing 603 Button 604 Button 701 Front view 02 Right side view 703 Top view 801 Main case 802 Sub case 803 Microphone 804 Speaker 805 Touch panel LCD 901 Front view 902 Right side view 903 Top view 1001 Front view 1002 Right side view 1003 Top view 1004 Motherboard 1005 Touch panel LCD 1006 2.5 inch hard disk drive 1007 button 1008 button 1101 front view 1102 right side view 1103 top view 1104 microphone 1105 speaker 1106 USB audio device 1107 audio amplifier 1201 control unit 1202 GUI unit 1203 voice input unit 1204 voice recognition unit 1205 example database 1206 words Class dictionary 1207 Example selection unit 1208 Word selection unit 1209 Alternative word selection unit 1210 Language conversion unit 1211 Speech synthesis unit 1301 Translation direction Step 1302 to make a decision Step 1303 to perform voice recognition Step 1304 to retrieve an example from an example database Step 1305 Step 1305 to decide an example or decide whether to correct it Step 1306 Determine a word to be corrected Step 1307 Alternative word list Step 1308 to acquire Step 1308 to determine alternative words 1309 To modify example 1313 To perform language conversion Step 1311 To perform speech synthesis section 1601 Translation direction designation section 1602 Translation direction designation section 1603 Recognition result display section 1604 Example candidate display section 1605 Example result display section 1606 Interpretation result display section 1607 Button SW1 1608 Button SW2 1701 Translation direction designation section 1702 Recognition result display section 1703 Example candidate display section 1801 Selected example 1901 Example result Radical 113 1902 example candidate display unit 2001 interpretation result display unit 2101 example result display unit 2201 list window 2301 selected alternative words 2401 example result display unit 4105 language conversion direction controller 4106 dialogue history manager 4104 language conversion direction detector

Continued front page    (72) Inventor Kenji Matsui             1006 Kadoma, Kadoma-shi, Osaka Matsushita Electric             Sangyo Co., Ltd. (72) Inventor ▲ Hiro ▼ Yoshifumi Se             1006 Kadoma, Kadoma-shi, Osaka Matsushita Electric             Sangyo Co., Ltd. (72) Inventor Eiji Maekawa             1006 Kadoma, Kadoma-shi, Osaka Matsushita Electric             Sangyo Co., Ltd. (72) Inventor Shinichi Yoshizawa             1006 Kadoma, Kadoma-shi, Osaka Matsushita Electric             Sangyo Co., Ltd. F term (reference) 5B091 CA22 CB12 CB24 CB32 DA01                       DA11                 5D015 AA05 KK02 KK04 LL05 LL11                 5D045 AB03

Claims (24)

[Claims] 1. A voice input unit for inputting a voice of a first language, a voice recognition unit for voice recognition of the input voice, and a first voice recognition unit corresponding to a recognition result of the voice. First extraction / display means for extracting and displaying one or a plurality of word strings of a language, and any word string which is to be converted from the displayed word string into a second language. Conversion target selection means for selecting, and when all or part of the selected word string is specified,
Second extraction / display means for extracting and displaying a term candidate corresponding to the specified content; candidate selecting means for selecting one of the displayed candidates; and A conversion means for determining a conversion target to the second language based on the selected word string and the selected candidate, and converting the determined conversion target to a spoken language of the second language. Voice conversion device equipped with. 2. The first extraction / display means has a display screen for displaying a plurality of word strings to be selected and the selected word strings in predetermined areas. 2. The display unit according to claim 1, further comprising: a display unit provided, wherein the second extraction / display unit is a unit that displays the term candidates by overlapping them in a window shape in a partial area of the display screen. Voice converter. 3. The first extracting / displaying unit, when displaying the selected word string on the display screen, displays a candidate of the corresponding term for a part of the word string. 3. The voice conversion device according to claim 2, which is a means for adding and displaying information indicating that it is possible. 4. The voice conversion device according to claim 3, further comprising screen display specifying means for specifying, on the display screen, a part of the word string in which the added information is displayed. 5. The conversion means determines a result of replacing the specified part of the word string with the selected candidate term as the conversion target.
The voice conversion device described. 6. A voice input step of inputting a voice of a first language, a voice recognition step of voice-recognizing the input voice, and a voice recognition step of the first language corresponding to a recognition result of the voice. A first extraction / display step of extracting and displaying one or a plurality of word strings, and selecting one of the word strings that is to be converted into the second language from the displayed word strings. A conversion target selection step for, when all or part of the selected word string is specified,
A second extraction / display step of extracting and displaying a candidate for a term corresponding to the specified content; a candidate selecting step for selecting any candidate from the displayed candidates; A conversion step of determining a conversion target to the second language based on the selected word string and the selected candidate, and converting the determined conversion target to a spoken language of the second language. Voice conversion method with. 7. The speech recognition apparatus according to claim 1, wherein the speech recognition unit, the first display / extraction unit, the conversion target selection unit, and the second unit. display·
A program for causing a computer to function as all or a part of the extraction means, the candidate selection means, and the conversion means. 8. A voice conversion method according to claim 6, wherein the voice recognition step, the first display / extraction step,
A program for causing a computer to execute all or part of the conversion target selection step, the second display / extraction step, the candidate selection step, and the conversion step. 9. A medium carrying the program according to claim 7, which can be processed by a computer. 10. A medium carrying the program according to claim 8, which can be processed by a computer. 11. An input unit for inputting a voice of a first or a second language, and (1) when the voice of the first language is received from the input unit, the voice is recognized, and Based on a predetermined control instruction, (1-a) output the voice-recognized notation data of the first language, or (1-b) conversion determined based on the voice-recognized recognition result. The target is converted into the second language, at least the written data in the converted language is output, and (2) when the voice of the second language is received from the input unit, the voice is recognized, and , (2-a) outputting the voice-recognized notation data of the second language based on a predetermined control instruction, or (2-b) determining based on the voice-recognized recognition result. The conversion target is converted into the first language, and at least notation data in the converted language A translation unit to output, a support unit for assisting the translation unit in determining the conversion target, the notation data in the translated language output from the translation unit, based on the predetermined control instruction. A voice conversion device comprising: a display unit for displaying; and a control unit for performing the predetermined control instruction to at least the translation unit and the display unit. 12. A user who uses the first language,
Dialogue for sequentially holding the notation data based on the voice input by the input unit at the time of dialogue with another user who uses the second language, and outputting it as history information to the display unit. The voice conversion device according to claim 11, further comprising a history management unit. 13. A translation direction for determining which translation from the first language to the second language or from the second language to the first language should be performed by the speech translation unit. The language conversion direction detection part which detects information is provided, The said control part designates the said translation direction with respect to the said speech translation part based on the said detection result, and controls the said input part. The voice conversion device described. 14. The control of the input unit of the control unit means, when the input unit is composed of a plurality of voice input units, the voice input unit that most collects the voice of the user who speaks. The voice conversion device according to claim 13, wherein the voice conversion device is selected. 15. The voice according to claim 13, wherein the control unit controls the display content of the display unit to rotate substantially 180 degrees based on the display screen of the display unit according to the translation direction. Converter. 16. The voice conversion device according to claim 13, wherein the language conversion direction detection unit is configured by a button switch, and a user who speaks presses the button switch to select a translation direction. 17. The language conversion direction detection unit is composed of an angle sensor for detecting the direction of the movable microphone having the best acoustic directivity, and the user speaking changes the direction of the microphone to perform the translation. 14. The voice input translation device according to claim 13, wherein a direction is selected. 18. The language conversion direction detection unit is composed of a gyro sensor installed inside the voice conversion device, and the translation direction is set at a position where a uttering user holds the voice conversion device in his hand. The voice conversion device according to claim 13, which is selected. 19. The language conversion direction detection unit is configured by a sound source direction detection device of an input unit configured by a microphone array unit, and the translation direction is determined by a utterance position of a user who speaks with respect to the microphone array unit. The voice conversion device according to claim 13, which is selected. 20. Input voice of the first or second language,
A voice input step for outputting; (1) the first voice output by the voice input step;
When the voice of the language is received, the voice is recognized, and based on a predetermined control instruction, (1-a) the voice-recognized notation data of the first language is output, or (1 -B) converting a conversion target determined based on the recognition result obtained by the voice recognition into the second language, outputting at least notation data of the converted language, and (2) outputting the voice by the voice input step. When the received voice of the second language is received, it is voice-recognized, and (2-a) the notation data of the voice-recognized second language is output based on a predetermined control instruction. Or (2-b) a translation step of converting the conversion target determined based on the recognition result of the voice recognition into the first language and outputting at least notation data in the converted language, The above in the translation step A support step for supporting the determination of the conversion target, a display step for displaying the notation data in the converted language output by the translation step based on the predetermined control instruction, the predetermined A voice conversion method comprising: a control step of giving a control instruction to at least the translation step and the display step. 21. As the whole or a part of the translation unit, the support unit, the display unit, and the dialogue history management unit of the voice conversion device according to any one of claims 11 to 19. A program for operating a computer. 22. A program for causing a computer to execute all or part of the translation step, the support step, the display step, and the dialogue history management step of the voice conversion method according to claim 20. 23. A medium carrying the program according to claim 21, which can be processed by a computer. 24. A medium carrying the program according to claim 22, which can be processed by a computer.