JP2007256297A - Speech processing method and communication system, and communication terminal and server and program - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a system in which a program and data which are incorporated in a communication terminal beforehand are interlocked with speech processing and operated. <P>SOLUTION: The communication terminal 100 includes: a first program and data storage means 101; an speech processing means 102; an speech processing language information creating means 103 for creating language information for the speech processing means; a transmitting and receiving means 104 for acquiring the program/data from a server 200; a second program and data storage means 105 for storing the acquired program/data; and a control means 106 for interlocking and controlling the speech processing means 102 and the program/data of the first program and data storage means 101, based on the second program and the program/data stored in and data storage means. The server 200 includes: a transmitting and receiving means 201; and an speech processing language information creating means 202 for creating language information, based on the data transmitted from the communication terminal. <P>COPYRIGHT: (C)2008,JPO&INPIT

Description

  The present invention relates to a communication system, and more particularly to a system and method for realizing cooperation of processing between a terminal and a server connected to the terminal, and a computer program.

  Examples of conventional communication terminals and systems are described in Patent Document 1, Patent Document 2, and the like described later. Among these, the communication terminal described in Patent Document 1 below includes a speech recognition unit, a speech synthesis unit, a control unit, a transmission / reception unit, and a data conversion unit, converts input speech into text by speech recognition, transmits the text, and transmits and receives The data received from the unit is read out by speech synthesis.

  The communication terminal described in Patent Document 2 below includes a voice recognition unit, a voice synthesis unit, and a control unit (mail processing unit), creates a mail by voice input, and reads out the mail by voice synthesis. .

Japanese Unexamined Patent Publication No. 2003-188948 (page 4, FIG. 1) Japanese Patent Application Laid-Open No. 2002-077315 (Pages 2, 3 and 1)

  In the above-described conventional terminal, the program received from the outside of the terminal is a program built in the terminal, data managed by the program, in particular, data that changes dynamically according to the user's own data or the state of the system. The voice processing function cannot be operated in cooperation in a desired manner unique to the terminal, for example.

  Accordingly, an object of the present invention is to provide a communication system, a server, and a communication terminal capable of realizing a desired function by combining a program and / or data incorporated in a communication terminal and processing such as voice processing. is there.

  The invention disclosed in the present application is generally configured as follows in order to achieve the above object.

  A communication terminal according to one aspect of the present invention includes a speech processing unit that performs speech recognition and / or speech synthesis processing, a program for realizing a predetermined function on the communication terminal, and / or Alternatively, a first storage unit that stores at least data, a program and / or data that is input to the communication terminal from the outside of the communication terminal and stored in the first storage unit, and audio processing by the audio processing unit Is stored in the first storage unit, the second storage unit that stores at least a program and / or data that defines how to cooperate with the computer, the program and / or data input from the outside of the communication terminal Using the program and / or data, the voice processing by the voice processing unit and the program and / or data stored in the first storage unit That includes a control unit which performs control to work together with, and the function.

  In the present invention, the control unit activates a program stored in the second storage unit, and the activated program calls a program stored in the first storage unit or stores data. The voice processing by the voice processing unit and the program and / or data stored in the first storage unit are operated in cooperation.

  A communication terminal according to one aspect of the present invention includes a speech processing unit that performs speech recognition and / or speech synthesis processing, a first storage unit that stores at least information held by the communication terminal, and the communication A second storage unit that stores at least a program that is input from the outside of the terminal to the communication terminal and that defines a procedure for creating language information for voice processing; and a voice processing stored in the second storage unit A program that defines a procedure for creating language information for speech processing that is stored in the second storage unit. Uses at least the information stored in the first storage unit to create language information used for voice processing in the voice processing unit, and the voice processing unit uses the created language information. It performs the voice processing Te.

  A system according to another aspect (side surface) of the present invention, and a server for communication connection with the communication terminal, the communication terminal based on a program and / or data stored in advance in the communication terminal, Based on means for creating language information used in the speech processing means, programs and / or data stored in advance in the communication terminal using the language information, and programs and / or data downloaded from the server And a means for operating the voice processing in a coordinated manner.

  A system according to another aspect of the present invention includes a communication terminal and a server that is communicatively connected to the communication terminal, and the communication terminal stores a program and / or data stored and held in advance by the communication terminal. In accordance with the program and / or the data stored in the first storage unit, the voice processing unit that performs voice processing of at least one of voice recognition and voice synthesis, and the program and / or data stored in the first storage unit, Speech processing language information creation means for creating language information to be used (for example, dictionary, grammar, language model, etc.), means for obtaining a program and / or data from the server, and the program and / or data obtained from the server Based on a second storage unit that stores the program and / or data stored in the second storage unit, Serial includes first control means for cooperation control programs and / or data storage unit. Further, the server receives information transmitted from the communication terminal, and based on the data transmitted from the communication terminal and means for transmitting the program and / or data generated by the server to the communication terminal. Voice processing language information creating means for creating language information used by the voice processing means from data stored on the server side.

  In a system according to still another aspect of the present invention, the speech processing language information creating unit is configured to perform the above-described program and / or data stored in the first storage unit and the second storage unit. Language information used in the voice processing unit is created, and the control unit uses the program and / or data stored in the second storage unit to program the voice processing unit and the first storage unit. And / or control for linking data.

  A system according to still another aspect of the present invention includes a communication terminal and one or more servers, and the server receives information transmitted from the communication terminal and generates the information on the server side. Means for transmitting a program and / or data to a communication terminal; voice processing language information creating means for creating a dictionary for voice processing from data transmitted from the communication terminal and data stored on the server side; ,including. The communication terminal includes a first storage unit that stores a program and / or data stored and held in advance in the communication terminal, a voice processing unit that performs voice processing of at least one of voice recognition and voice synthesis, Means for acquiring a program and / or data from a server, a second storage unit for storing the program and / or data acquired from the server, and storing both in the first storage unit and the second storage unit Using speech processing language information creating means for creating language information used in the speech processing means in accordance with the program and / or data being stored, and the program and / or data stored in the second storage unit, The voice processing means; a control means for linking programs and / or data stored in the first storage unit; In including a spoken language information generated, and the audio processing language information integration unit for synthesizing the audio language information created by the speech processing language information generating means of said server, a.

A method according to one aspect of the present invention includes:
(A) Language information used in speech processing based on a program and / or data downloaded to the communication terminal from the outside by the communication terminal and a program and / or data stored in advance in the communication terminal Generating
(B) The communication terminal includes a step of executing a process in which a program and / or data stored in advance in the communication terminal and a voice process are operated in cooperation using the spoken language information.

A method according to another aspect of the present invention includes:
(A) a communication terminal acquiring a program and / or data from outside the communication terminal;
(B) The communication terminal generates speech language information used in speech processing based on the acquired program and / or data and the program and / or data stored in advance in the communication terminal; ,
(C) Using the speech language information, the communication terminal causes a program stored in advance in the communication terminal, a program and / or data downloaded from the outside of the communication terminal, and the voice processing to cooperate with each other. Process,
including.

A method according to yet another aspect of the present invention includes:
(A) The communication terminal is configured to store the program in the communication terminal and / or the plurality of servers based on the program and / or data downloaded from one or more servers and the program and / or data stored in advance in the communication terminal. Integrating speech language information for speech processing generated by at least one;
(B) The communication terminal uses the speech language information to link a program and / or data stored in advance in the communication terminal, a program and / or data downloaded from a server, and the voice processing. A process of
including.

A computer program according to one aspect of the present invention is provided in a computer constituting a communication terminal.
(A) a process of storing a program and / or data stored in advance in the communication terminal;
(B) a process of receiving a program and / or data generated outside the communication terminal;
(C) storing the received program and / or data;
(D) a process of executing at least one of speech recognition and speech synthesis;
(E) generating speech language information for performing speech processing based on a program and / or data stored in advance in the communication terminal;
(F) A process of linking a program and / or data stored in advance in the communication terminal and voice processing by the received program and / or data;
It consists of a program to execute.

A computer program according to another aspect of the present invention provides a computer constituting a communication terminal,
(A) processing for acquiring a program and / or data from outside the communication terminal;
(B) a process of executing at least one of speech recognition and speech synthesis;
(C) processing for generating spoken language information used in speech processing based on the downloaded program and / or data and the program and / or data stored in advance in the communication terminal;
(D) Using the spoken language information, executing a program stored in advance in the communication terminal, a program and / or data downloaded from the outside of the communication terminal, and a process for causing the voice processing to operate in cooperation with each other It consists of a program to let you.

A computer program according to still another aspect of the present invention provides a computer constituting a communication terminal,
(A) processing for storing a program and / or data stored in advance in the communication terminal in the first storage unit;
(B) a process of receiving a program and / or data from one or more servers outside the communication terminal;
(C) a process of storing the received program and / or data in a second storage unit;
(D) processing for performing at least one of speech recognition and speech synthesis;
(E) Audio for performing audio processing based on the program and / or data stored in the second storage unit and the program and / or data stored in advance in the first storage unit Processing to generate language information in the communication terminal;
(F) a process for integrating spoken language information generated in the communication terminal or in the server;
(G) A program and / or data stored in the first storage unit and a process for linking audio processing with the program and / or data stored in the first storage unit. It consists of a program.

  According to the present invention, even if a program built in a communication terminal such as a portable terminal does not support voice processing, it can be operated in cooperation with the voice processing function by downloading the program from the server. it can.

  Further, according to the present invention, programs having different cooperation methods can be freely exchanged and executed according to user preferences or the like.

  Next, the best mode for carrying out the present invention will be described in detail with reference to the drawings.

  Referring to FIG. 1, the first embodiment of the present invention includes a communication terminal 100 and a server 200. The communication terminal 100 includes a first program and data storage unit 101, a voice processing unit 102, a voice processing language information creation unit 103, a transmission / reception unit 104, a second program and data storage unit 105, and a control unit 106. I have. The server 200 includes transmission / reception means 201 and voice processing language information creation means 202. Each of these means generally operates as follows.

  The first program and data storage means 101 stores a program built in the communication terminal 100 in advance and data managed by the program. The data stored in the first program and data storage means 101 includes data that dynamically changes according to the state of the communication terminal 100 and personal data of the user of the communication terminal 100.

  The voice processing unit 102 performs at least one of voice recognition and voice synthesis.

  The voice processing language information creation unit 103 creates a dictionary, a grammar, a language model, and the like for the voice processing unit 102 based on the first program and data stored in the first program and data storage unit 101.

  The transmission / reception means 104 transmits information on the communication terminal 100 side to the server 200 and receives programs and data from the outside of the communication terminal 100.

  Second program and data storage means 105 stores the program and data received from the outside of communication terminal 100 by transmission / reception means 104.

  The control unit 106 calls the program and data (subroutine call or the like) from the second program and data acquired by the transmission / reception unit 104 to link the audio processing unit 102 with the first program and data.

  The transmission / reception means 201 of the server 200 receives information from the communication terminal 100 side, and transmits a program and data from the server 200 to the communication terminal 100 side.

  The voice processing language information creation unit 202 creates a voice processing dictionary from data stored on the server 200 side based on the data transmitted from the communication terminal 100.

  FIG. 2 is a flowchart for explaining the operation of the embodiment of the present invention. With reference to FIG.1 and FIG.2, the whole operation | movement of this Embodiment is demonstrated in detail.

  The speech language information used by the speech processing means 102 of the communication terminal 100 is classified into two types: a case where it is generated on the communication terminal 100 side and a case where it is generated on the server 200 side.

  When generating on the communication terminal 100 side, the transmission / reception means 104 receives the program and data from the outside of the communication terminal 100 (step Sa1) and stores them in the second program and data storage means 105 (step Sa2).

  Next, the control means 106 calls and activates the program and data stored in the second program and data storage means 105 (step Sa3).

  Language information is generated by using the first program and the data stored in the data storage means 101 according to the voice information language information creation procedure described in the activated program (step Sa4).

  The control means 106 reads out the generated speech processing language information and activates the speech processing means 102 (step Sa5).

  In accordance with the program started in step Sa3, the first program and the program of the data storage means 101 are called (step Sa6), and the voice processing means 102 is operated in cooperation (step Sa7).

  When generating on the server 200 side, the transmission / reception means 104 receives the program and data from the outside of the communication terminal 100 (step Sb1) and stores them in the second program and data storage means 105 (step Sb2).

  Next, the control means 106 starts the above-mentioned program and data from the 2nd program and data storage means 105 (step Sb3).

  Language information creation procedure for speech processing described in the activated program and data necessary for language information creation are transmitted to the server 200 (step Sb4). On the server 200 side, the speech processing language information creation means 202 generates speech language information using the language information creation procedure and data transmitted from the communication terminal 100 and the data stored on the server 200 side (step Sb5) and transmitted to the communication terminal 100 (step Sb6).

  In response to this, the communication terminal 100 reads the generated speech processing language information and activates the speech processing means 102 (step Sb7).

  Further, the first program and the program of the data storage means 101 are called in accordance with the program activated in step Sb3 (step Sb8), and the voice processing means 102 is operated in cooperation (step Sb9).

  Next, the effect of this Embodiment is demonstrated.

  In this embodiment, the communication terminal 100 and the server 200 are configured to execute the voice processing language information creation means 103 and 202, and therefore, the program built in the communication terminal 100 and the program manage it. Even if the data is not compatible with voice processing such as voice recognition and voice synthesis, the communication terminal 100 can have a voice processing function by downloading a program linked with the voice processing from outside the communication terminal 100. Can be used.

  In this embodiment, since the transmission / reception means 104, the means for storing the program acquired by reception, and the control means 106 for performing control for calling and executing the program are provided, the cooperation method Different programs can be replaced according to user preference.

  Next, the second embodiment of the present invention will be described in detail with reference to the drawings.

  Referring to FIG. 20, the second embodiment of the present invention includes a communication terminal 1000 and a server 200. The communication terminal 1000 includes a first program and data storage unit 1101, a voice processing unit 1102, a voice processing language information creation unit 1103, a transmission / reception unit 1104, a second program and data storage unit 1105, and a control unit 1106. I have. The server 200 includes transmission / reception means 201 and voice processing language information creation means 202. Each of these means generally operates as follows.

  The first program and data storage means 1101 stores a program built in the communication terminal 1000 in advance and data managed by the program. The data stored in the first program and data storage means 1101 includes data that dynamically changes according to the state of the communication terminal 1000, personal data of the terminal user, and the like. The voice processing unit 1102 performs voice recognition and / or voice synthesis.

  The voice processing language information creation unit 1103 creates a dictionary, a grammar, a language model, and the like for the voice processing unit 1102 according to the contents stored in the first program and data and the second program and data.

  The transmission / reception means 1104 transmits information on the communication terminal 1000 side, and receives programs and data from the outside of the terminal.

  The second program and data storage unit 1105 stores the program and data received from the outside of the communication terminal 1000 by the transmission / reception unit 1104.

  The control unit 1106 calls the program and data from the second program and data acquired by the transmission / reception unit 1104 to link the audio processing unit 1102 and the first program and data.

  The transmission / reception means 201 receives information from the communication terminal 1000 side, and transmits a program and data generated by the server 200 to the terminal side.

  The voice processing language information creation unit 202 creates a voice processing dictionary from data stored on the server 200 side based on the data transmitted from the communication terminal 1000.

  Next, the overall operation of the present embodiment will be described in detail with reference to the flowcharts of FIGS.

  First, the transmission / reception means 1104 receives the program and data from the outside of the communication terminal 1000 to the second program and data storage means 1105 (step S2101) and stores them in the second program and data storage means 1105 (step S2102).

  Next, the control unit 1106 activates the above-described program and data from the second program and data storage unit 1105 (step S2103), and the procedure for creating language information for voice processing described in the program is executed by a subroutine call or the like. Call (step S2104).

  In step S2104, the program and data to be created are called from both the first program and data storage means 1101 and the second program and data storage means 1105 in the called creation procedure (step S2105).

  When the speech processing language information creation unit 1103 of the communication terminal 1000 generates the speech information in the language information generation procedure read out in step S2104 (step S2106), the speech processing language information creation unit 1103 is based on the program and data called in step S2105. The language information is generated at (step S2107).

  If the voice processing language information creation unit 1103 of the communication terminal 1000 does not create in step S2107, the process proceeds to step S2108.

  When the voice processing language information creation unit 202 of the server 200 generates the data by the language information generation procedure read in step S2104 (step S2108), the data read in step S2105 is transmitted to the server 200 by the transmission / reception unit 1104. This data is received (step S2109). The speech processing language information creating unit 202 generates speech processing language information from the above-described data and the data stored in the server (step S2110).

  The speech processing dictionary generated by the transmission / reception means 201 of the server 200 is transmitted to the communication terminal 1000, and the transmission / reception means 1104 receives this (step S2111).

  If the speech processing language information creating unit 202 of the server 200 does not create speech processing language information in step S2108, the process advances to step S2112.

  In response to this, the control unit 1106 reads the generated speech processing language information and activates the speech processing unit 1102 (step S2112).

  Further, in accordance with the program started in step S2103, the first program and the program of the data storage means 1102 are called (step S2113), and the voice processing means 1102 is operated in cooperation (step S2114).

  Next, the effect of this Embodiment is demonstrated.

  In the present embodiment, the speech processing language information creating unit 1103 calls the program and data from both the first program and data storage unit 1101 and the second program and data storage unit 1105 to create the speech language information. It is possible to perform speech processing such as speech recognition and speech synthesis by linking a program built in the communication terminal 1000 in advance with the received program.

  Next, a third embodiment of the present invention will be described in detail with reference to the drawings.

  Referring to FIG. 23, the third embodiment of the present invention includes a communication terminal 2000, a plurality of servers 200, and a server n00. The communication terminal 2000 includes a first program and data storage unit 2101, a voice processing unit 2102, a voice processing language information creation unit 2103, a transmission / reception unit 2104, a second program and data storage unit 2105, a control unit 2106, A voice processing language information integration unit 2107 is provided.

  The server 200 includes transmission / reception means 201 and voice processing language information creation means 202. The server n00 includes transmission / reception means n01 and speech processing language information creation means n02. Each of these means generally operates as follows.

  The first program and data storage unit 2101 stores a program built in the communication terminal 2000 and data managed by the program. The data stored in the first program and data storage unit 2101 includes data that dynamically changes according to the state of the communication terminal 2000 and personal data of the terminal user. The voice processing unit 2102 performs voice recognition and / or voice synthesis.

  The voice processing language information creation unit 2103 creates a dictionary, a grammar, a language model, and the like for the voice processing unit 2102 according to the contents stored in the first program and data and the second program and data. The transmission / reception means 2104 transmits information on the communication terminal 2000 side, and receives programs and data from the outside of the terminal. The second program and data storage unit 2105 stores the program and data received from the outside of the terminal by the transmission / reception unit 2104.

  The control unit 2106 calls the program and data from the second program and data acquired by the transmission / reception unit 2104 to link the audio processing unit 2102 and the first program and data.

  The speech processing language information integration unit 2107 includes the speech language information generated by the speech processing language information creation unit 2103 of the communication terminal 2000, the speech processing language information generation unit 202 of the server 200, and the speech processing language information generation unit n02 of the server n00. Spoken language information is generated by synthesizing with the spoken language information generated in step (1).

  The transmission / reception means 201 of the server 200 receives information from the communication terminal 2000 side and transmits a program and data generated by the server 200 to the communication terminal 2000 side. The speech processing language information creation unit 202 creates speech language information for speech processing from the data stored on the server 200 side based on the data transmitted from the communication terminal 2000.

  Similarly, in the server n00, the transmission / reception means n01 receives information from the communication terminal 2000 and transmits the program and data generated by the server n00 to the communication terminal 2000. The speech processing language information creation unit n02 creates speech language information for speech processing from the data stored on the server n00 side based on the data transmitted from the communication terminal 2000.

  Next, the overall operation of the present embodiment will be described in detail with reference to the flowcharts of FIGS.

  First, the transmission / reception means 2104 receives the program and data from the outside of the communication terminal 2000 to the second program and data storage means 2105 (step S2401) and stores them in the second program and data storage means 2105 (step S2402).

  Next, the control means 2106 activates the above-mentioned program and data from the second program and data storage means 2105 (step S2403), and calls the language information creation procedure for speech processing described in the program. In the audio information creation procedure called in step S2404, the program and data to be created are called from both the first program and data storage means 2101 and the second program and data storage means 2105 (step S2405).

  When the speech processing language information creation unit 2103 of the communication terminal 2000 generates the speech information in the language information generation procedure read out in step S2404 (step S2406), the speech processing language information creation unit 2103 is based on the program and data called in step S2405. The language information is generated at (step S2407).

  If the voice processing language information creation unit 2103 of the communication terminal 2000 does not create in step S2406, the process proceeds to step S2408.

  In the case where the speech processing language information creation unit 202 of the server 200 generates the data by the language information generation procedure read in step S2404 (step S2408), the data read in step S2405 is transmitted to the server 200 by the transmission / reception unit 2104, and the transmission / reception unit In 201, this data is received (step S2409).

  The speech processing language information creating unit 202 generates speech processing language information from the above data and the data stored in the server (step S2410).

  The voice processing dictionary generated by the transmission / reception means 201 of the server 200 is transmitted to the communication terminal 2000, and the transmission / reception means 2104 receives this (step S2411).

  In step S2408, if the speech processing language information creation unit 202 of the server 200 does not create speech processing language information, the process advances to step S2412.

  In the speech processing language information integration unit 2107, the speech language information generated by the speech processing language information creation unit 2103 of the communication terminal 2000, the speech language creation unit 202 of the server 200, and the speech processing language information creation unit n02 of the server n00, It is synthesized into one speech language information (step S2412).

  In response to step S2412, the control unit 2106 reads the generated speech processing language information and activates the speech processing unit 2102 (step S2413).

  Further, in accordance with the program activated in step S2403, the first program and the program of the data storage means 2101 are called (step S2414), and the voice processing means 2102 is operated in cooperation (step S2415).

  Next, the effect of this Embodiment is demonstrated. In this embodiment, the speech processing language information integration unit 2107 is provided, and the speech language information generated on the communication terminal 2000 side and the speech language information generated on the plurality of servers 200 to n00 side are combined. It is possible to perform voice processing by linking these programs and data.

  Next, a first embodiment of the present invention will be described with reference to the drawings. The configuration of the first embodiment of the present invention is the configuration shown in FIG. 3 to 8 illustrate an example of the program and data stored in the first program and data storage means 101 of this embodiment shown in FIG.

  FIG. 3 is a diagram for explaining the correspondence between the first program and the program stored in the data storage means 101 and the data managed by the program. In FIG. 3, five programs of terminal management, address book, transmission incoming / outgoing (outgoing / receiving) history, GPS (Global Positioning System), and infrared are stored as programs. Further, as data corresponding to each, data is stored in terminal management data, address book data, incoming call history data, GPS data, and infrared rays.

  4 to 8 are diagrams showing examples of data managed by the first program shown in FIG. 3 and the program stored in the data storage means 101. The user-specific information of the communication terminal 100 and the operation of the communication terminal are shown in FIG. Data that changes continuously is assumed.

  FIG. 4 is an example of terminal management data managed by the terminal management program of FIG. As shown in FIG. 4, the terminal management data is composed of items and their values (parameters).

  Items include power supply, remaining battery level, radio wave status, lid open / closed. The values are: power = ON, remaining battery level = 45%, radio wave status = good, lid open / closed = closed. Yes.

  FIG. 5 is an example of address book data managed by the address book program of FIG. The address book data is composed of an ID, a name, a name reading, a mail address, and a telephone number for numbering the data. In the example shown in FIG. 5, ID = 001, name = Ichiro Tanaka, reading = Icchan, mail address=ichiro@xxx.com, and telephone number = 090-2222-3333.

  FIG. 6 is an example of outgoing / incoming (transmission / reception) history data managed by the outgoing / incoming (transmission / reception) history program shown in FIG. The outgoing / incoming history data includes an ID for numbering the data, the type of incoming / outgoing call, the date / time of outgoing / incoming call, and the telephone number of the destination of outgoing / incoming call.

In the example shown in FIG. 6, there are three data,
In the data of ID = 001, it is as follows: Outgoing and incoming calls = outgoing, outgoing date and time = 01:02:34 on December 18, 2003, outgoing address = 090-2222-3333,
In the data of ID = 002, it is as follows: Outgoing incoming / outgoing = incoming, incoming date and time = 1: 4: 34 on December 18, 2003, incoming call = 090-2222-3333,
In the data of ID = 003, the type of incoming / outgoing call = incoming, incoming date / time = December 18, 2003 2: 4: 34, and incoming destination = 090-2222-3333.

  FIG. 7 is an example of GPS data managed by the GPS program of FIG. In the GPS data, data acquired when the GPS program is operated immediately before is held. In the example shown in FIG. 7, GPS data is composed of item names and their values. The items are composed of the measurement date and time, latitude and longitude, and the address of the current location. Measurement date and time = December 18, 2003 1:00:34, Latitude = N35 ° 51.475, Longitude = E139 ° 51.475, Current location address = It is 1-1-1, Shiba, Minato-ku, Tokyo.

FIG. 8 is an example of infrared data managed by the infrared program of FIG. In infrared data, a communication history with an infrared port outside the terminal is held. In the example shown in FIG. 8, the infrared data is composed of an ID for numbering data, a communication date and time, a communication destination ID, a communication command, and parameters associated with the command. There is data
In the data of ID = 001, the communication date and time = December 18, 2003 1: 0: 34 10ms, communication destination = IDxxxxxx, communication command = connection request,
In the data of ID = 002, the communication date and time = December 18, 2003 1: 0: 34: 30ms, communication destination = IDxxxxxx, communication command = connection request response,
For data with ID = 003, the communication date and time = December 18, 2003 1: 0: 34, 50 ms, communication destination = IDxxxxxx, communication command = data read request, parameter = "data name = file 1.txt file type = txt "
For data with ID = 004, communication date and time = December 18, 2003 1: 0: 34, 0 ms, communication destination = Idxxxxxx, communication command = data read response, parameter = "data name = file 1.txt file type = txt It has become.

In the example shown in FIG.
At the time indicated by the data of ID = 001, a connection request is transmitted from the communication terminal to the communication destination indicated by the data of ID = 001,
A connection with the communication destination is established at the time indicated by the data of ID = 002,
Send a request to read the file 1.txt, which is a text format file, at the time indicated by the data of ID = 003,
The file of 1.txt file was sent to the communication terminal at the time indicated by the data of ID = 004,
It means that.

  Next, the operation of the speech processing language information creation unit 103 of the communication terminal 100 according to an embodiment of the present invention will be described with reference to FIGS.

  The speech processing language information creation unit 103 creates speech language information used by the speech processing unit 102. The voice processing unit 102 performs voice recognition and voice synthesis.

  As speech language information for speech recognition, a word dictionary composed of a word string and its reading, a grammar by a finite language network, and a language model based on a probability statistical model are used.

  As speech language information for speech synthesis, a dictionary composed of word strings and their readings is used.

  As an example of creating a word dictionary for speech recognition, morphological analysis is performed using a target program or data format, a word with a specific part of speech is read, and registered in the word dictionary.

  At this time, a reading rule is determined for each part of speech, and reading is performed according to the rule.

  For those in which the structure of the stored data is clear, the structural analysis is performed in advance, the data type is classified using this result, and reading is performed for each classified result.

  Examples of data for which the former morphological analysis is effective include Web page data managed by a Web browser and mail data managed by a mailer.

  Examples of the latter effective data structure analysis include address book data and terminal status data.

  When the analysis result is a person name, if there is corresponding reading information in the same data, the information is given priority, and if there is no reading, a reading is generated by the kana-kanji conversion function.

  The phone number and mail address of the analysis result are registered including numbers and alphabetical readings, as well as those including names such as “Tanaka's phone” and “Tanaka's mail address”.

  The nouns that indicate the functions and parts of the terminal such as “battery” and “power”, and the nouns, adjectives, and verbs that indicate the states of the results of the analysis are stored in advance as synonyms, pronunciation variants, and abbreviations as a database. Then, the corresponding one is registered in the dictionary from the database.

  As an example of a method for creating a grammar for speech recognition, a grammar template is prepared in advance for each target program. This grammar template defines a network of word strings and a method for generating word strings, which are grammatical structures.

  The voice processing language information creation means 103 inserts data and creates a grammar according to the grammar template.

  As an example of a dictionary generation method for speech synthesis, a morphological analysis is performed using a target program or data format in the same manner as a word dictionary for speech recognition, and a reading rule is defined for each part of speech. Register the reading.

  In addition, if the structure of the stored data is clear, structural analysis is performed in advance, the data type is classified using this result, and a reading rule is prepared for each classified result. Use to read.

  Examples of reading rules include the following methods.

  When the analysis result is a person name, as a reading rule, if there is corresponding reading information in the same data, the information is prioritized. If there is no reading, a reading is generated by the kana-kanji conversion function.

  If the analysis result is an e-mail address, if the owner can determine from the relationship between the e-mail addresses using the morphological analysis result as a reading rule, e-mail address like "Tanaka's e-mail address" Is read including the name of the owner, and if the owner cannot be determined, it is read as “this mail address”.

  It is defined in the program whether to use normal alphabets and numbers that do not need to be registered in the dictionary, or to use the reading rules.

  If the result of the analysis is a phone number, as a reading rule, if the owner can determine from the relationship of the phone number before and after using the result of morphological analysis, for example, “Mr. Tanaka's phone number” , Read with the name of the owner. On the other hand, when the owner cannot be determined, reading is performed like “this phone number”. Further, the type of the telephone such as a fixed telephone, a mobile phone, an IP telephone, a free dial, etc. is determined from the station number.

  In addition, there are a method using reading of numbers and symbols defined by speech synthesis, a method of replacing hyphens, minuses, and parentheses as delimiters between station numbers with '' '. For example, in the method of replacing hyphens, minuses, and parentheses with 'no', "044-999-1234" is read as "Zeroyonyon no Kyukyukyu no Ichi Nisansan". It is ending. Which reading rule is adopted is defined in the program.

  For nouns that indicate the function or part of the terminal such as “battery” or “power source”, the reading is registered in the database in advance, and the corresponding one is registered in the dictionary from the database.

  FIG. 9 is a diagram for explaining generation of a word dictionary for speech recognition. FIG. 9 shows an example where a dictionary is generated for the terminal information data shown in FIG. 4 as an example. Since the terminal information data is a noun indicating the function or part of the terminal or a noun indicating the state thereof, the corresponding information is registered in a dictionary from a previously registered database.

  An example is shown in which a dictionary is generated with “power source” and “remaining battery power”, which are item names of terminal information data, as notations.

  In the notation “power”, two words, “Dengen”, which is a word reading pseudonym, and “Pawa”, which is a synonym for power, are registered as speech recognition readings.

  In addition, in the expression “remaining battery power”, in addition to the word reading pseudonym “Denchi no Zanri”, the phonetic variants of “battery” and “battery” are synonyms for “battery”. Combined with “Nokori”, which is a synonym for “Batari” and “Remaining amount”, “Denchinokori”, “Batari no Zanri”, “Batari no Zanri”, “Batari” Register "Nawori" and "Nori no Nori".

  In addition, the abbreviations “battery remaining”, “denchi”, “zanryo”, and synonyms and pronunciation variations “battery”, “battery”, “smile” sign up.

  FIG. 10 is a diagram for explaining generation of a dictionary for speech synthesis. FIG. 10 shows an example in which a dictionary is generated for the address book data shown in FIG. 5 as an example, and “Tanaka Ichiro”, “ichiro@xxx.com”, “090-2222-” An example of generating speech synthesis readings for three types of notation “3333” is shown.

  For the notation “Ichiro Tanaka”, since it is a personal name, the reading “Icchan” given to the address book data is registered.

  Also, for the notation “ichiro@xxx.com”, since it is an e-mail address, register “Icchan's Mail Address”, which includes the reading of the e-mail owner.

  Furthermore, since the notation “090-2222-3333” is a telephone number, “Ichichan no Keitai Denwa” which is a format including the reading of the owner of the telephone number and the analysis result of the station number is registered.

  FIG. 11 is a diagram for explaining generation of a speech recognition grammar. FIG. 11 shows a grammar for operating the address book using the address book data shown in FIG. 5 as an example.

  FIG. 11A shows an example of a grammar template for generating a grammar. The grammar template defines a grammar structure and a method for defining a word string to be registered. In this example, a wording that utters each item of [Reading], [Participant] in the address book, [Item], [Participant], and [Operation] in the address book in order is defined. Of these, the word strings registered in [reading] and [item] are referred to from the address book. For other [particles] and [operation], word strings to be registered are defined in advance.

  FIG. 11B shows an example of a word string registered according to the grammar template of FIG. Each word string is composed of notation and reading. It defines a word string of four items: [reading] and [item name] referenced from the address book, [participant] and [operation] registered in advance. [Reading] has one word “Ichichan”, [Item name] has four words “name”, “reading”, “mail address”, “phone number”, and [Participant] has Two words “No” and “O” are registered in [Operation], and five words “Erase”, “Erase”, “Edit”, “Edit” and “I want to edit” are registered. . Each word is given a reading and a notation.

  FIG. 11C shows a correspondence between an example of a utterance that can be recognized as a result of inserting a word string into a template and the template in FIGS. 11A and 11B.

  By defining as described above, a grammar for recognizing an utterance “I want to edit my phone number” is generated.

  Next, the operation of the speech processing language information creating unit 202 of the server 200 according to the embodiment shown in FIG. 1 will be described with reference to FIGS.

  The voice processing language information creation unit 103 transmits a data generation method and data managed by the communication terminal 100 to the server 200 from the communication terminal 100 side. Using these, the speech language information is generated from the data stored in the server 200.

  The speech language information generated by the speech processing language information creation unit 202 of the server 200 is composed of a word string and its reading for speech recognition, similar to that generated by the speech synthesis information creation unit 103 of the communication terminal 100. A language model based on a word dictionary, a grammar by a finite language network, and a probability statistical model is assumed.

  The speech language information for speech synthesis is also assumed to be a dictionary composed of word strings and their readings. The method for creating a word dictionary and grammar and language model for speech recognition is the same except that data stored in the server 200 is used. The same applies to the speech synthesis dictionary.

  Hereinafter, the generation of the speech language information will be described with reference to FIGS.

  FIGS. 12 and 13 show an example of generating a speech recognition word dictionary using data stored in the server 200 as an example of generating the speech language information in the server 200.

  Here, a method of transmitting a telephone number of the communication terminal 100 from the server 200 side and creating a word dictionary for using the service based on the usage history managed by the server 200 will be described.

  FIG. 12 is a diagram illustrating usage history data that is an example of data managed by the server 200. This data is composed of a campaign indicating the ID for numbering the data, the telephone number of the user, the service name, the number of times of use, the date of last use, the total amount of use, and the degree of use desired by the service subject. In FIG. 12, when the telephone number of the communication terminal sent from the communication terminal 100 to the server 200 is 090-XXXXYYYY, it indicates that there are three cases of ID = 001 to 003.

In addition, about each use,
With ID = 001, the AA Museum ticket purchase is used 5 times, the last use date and time is 19:30:25 on March 12, 2003, the usage amount is 5500 yen, and the degree of hope of the service subject is low And
At ID = 002, Chinese restaurant B has been used 8 times, the last use date and time is 12:30 on March 12, 2003, the usage amount is 25800 yen, and the degree of hope of the service subject is medium Yes,
In ID = 003, the beauty salon C has no usage history and the service subject's degree of use is high.
It is shown that.

  FIG. 13 is a word dictionary for speech recognition created using FIG. In the example shown in FIG. 13, services that can be used are listed from the usage history, and registered service names and words for using the services are registered in the dictionary. A word string for using the service is defined in the server 200 in advance.

  When there are restrictions on the number of words that can be registered and the order of registration in the word string to be registered, restrictions are made according to the number of uses, the last use date, the total amount, and the degree of use desired by the service subject.

For example, in the case of the AA Museum ticket purchase service registered with ID = 001 in FIG.
Register AA Museum, Art Museum, ticket purchase, etc. in the word dictionary for voice recognition.

  As the speech language information generated by the speech processing language information creation unit 202 of the server 200, it is assumed that speech language information for operating a plurality of programs in cooperation is created.

  With reference to FIG. 14, generation of spoken language information for operating a plurality of programs in cooperation with each other will be described. In the example shown in FIG. 14, generation of a speech recognition grammar for operating a complex facility information guide program and a store service reservation program in the complex facility in cooperation with each other will be described.

  FIG. 14A and FIG. 14B show already generated speech recognition grammars.

  FIG. 14A is a grammar for controlling the information guidance program of the complex facility by voice. This grammar consists of four word strings: [Category] for specifying the category of the store in the complex, [Store name], [Item] for specifying the content of the guidance, and [Command]. Is done.

In the word string [Category], there are three words “restaurant”, “esthetic” and “museum”
In [Store Facility Name], the four words “Store A”, “Museum A”, and “Aesthetic Salon A”
In [Item], the four words "location", "business hours", "regular holiday", and "budget"
“Command” includes “where”, “how much”, “when”
These three words are registered.

  Each word registered in each word string has a notation and a reading.

  With this grammar, it is possible to recognize “when is a regular holiday at esthetic salon A”.

FIG. 14B is a service reservation program for a store in a complex facility.
[Service name] to identify the service,
[Time] to specify the time you want to receive the service,
[Reservation command] for specifying operations related to reservation
It is made up of.

In the word string [service name], two words “service A” and “service B”
In [Time], the two words “at 10:00” and “at 11:00”
[Reservation command] contains three words “reservation”, “cancellation”, and “confirmation”. Each word registered in each word string has a notation and a reading.

  In this grammar, “reservation of service A at 10:00” can be recognized.

  FIG. 14 (c) shows a grammar template for generating a new grammar by merging the grammars of FIG. 14 (a) and FIG. 14 (b).

In this template,
A grammar that specifies [Category], [Store Facility Name], [Item], and [Command] in sequence,
Two grammars are defined that specify [category], [store facility name], [service name], [time], and [reservation command] in sequence.

  The word strings defined in the items [category], [store facility name], [item], and [command] are inserted from the grammar of FIG.

  Also, the word strings defined in the items of [service name], [time], and [reservation command] are inserted from FIG.

FIG. 14D is a diagram showing a grammar created using the grammar template of FIG. This
“When is esthetic salon salon A regular holiday?”
"Esthetic esthetic salon A service A reserved at 10:00"
Both grammars can be recognized.

  Next, the program stored in the second program and data storage means 105 in FIG. 1 will be described with reference to FIGS.

  In the program stored in the second program and data storage means 105, the program and data stored in the first program and data storage means 101, the voice processing means 102, and the method of cooperative operation with the server 200 are defined. Yes.

  In this case, as the first program and the program stored in the data storage means 101 (program stored in advance in the communication terminal 100), an outgoing / incoming program for managing outgoing and incoming calls, a GPS program for determining the current location, an infrared ray, There are infrared communication programs for communication.

  Hereinafter, a specific program will be described with reference to FIGS. 15 to 19.

  Referring to FIGS. 15 and 16, as an example of a program stored in second program and data storage means 105, an outgoing / incoming program stored in first program and data storage means 101, and voice processing means 102, The operation of the program that operates in cooperation will be described. Specifically, in the above-described program, when a call is received, a call reception history and a message corresponding to the terminal state are output with synthesized sound.

  FIG. 15 is a flowchart for explaining the processing procedure of the program stored in the second program and data storage means 105. As shown in FIG. 15, when a call is received, the control means 106 first searches for data corresponding to the received telephone number from the address book (step S1301). If the incoming call number is registered ("Yes" branch in step S1302), the name is temporarily saved (step S1303). Next, the outgoing call / incoming history is searched, and if there is a corresponding telephone number ("Yes" branch in step S1304), the number of outgoing calls and the number of received times are temporarily stored (step S1305). Next, the terminal state is searched, and the battery state is temporarily saved (step S1306).

  Further, in response to the storage state in step S1303, step S1305, and step S1306, a sentence to be output is created (step S1307) and output by speech synthesis (step S1308).

  FIG. 16 is a diagram for explaining an example of a rule for creating an utterance sentence in step S1307. First, in step S1302, if there is no corresponding data in the address book, “No phone” is set. If there is corresponding data in the address book, create it like “Ichiro Tanaka's phone call” using the readings registered in the address book.

  Next, in step S1304, if the number of outgoing calls and the number of incoming calls are both 0 to 4, no corresponding sentence is created.

  When the number of outgoing calls is 5 or more and the number of incoming calls is 0 to 4, it is said that "I'm glad you've finally received it". Furthermore, when the number of incoming calls is 5 or more, it is assumed that “Today often comes”.

  In step S1306, if the remaining battery level is 40% or more, there is no corresponding sentence. If it's less than 40%, say "I don't care, but please connect it to the power source. The battery will run out."

  For example, if there is applicable data in step S1302, the number of incoming calls is 5 times or more, and the remaining battery level is less than 40%, “You are calling from Mr. Tanaka. But please connect it to the power supply. The remaining battery power is exhausted. ”In step S1302, there is no corresponding data, there are no incoming and outgoing calls, and the remaining battery capacity is 40% or more. “It ’s a phone.”

  Next, FIG. 17 shows an example of a program stored in the second program and data storage means 105. With reference to FIG. 17, the operation of the first program and the program that causes the GPS program stored in the data storage means 101 and the voice processing to operate in cooperation will be described. FIG. 17 is a diagram illustrating an example in which a GPS program, a server, and voice processing are operated in cooperation. Specifically, a program for determining a complex facility from the current position of the communication terminal and performing a voice search for the complex facility service It is a flowchart of.

  First, the first program in the communication terminal 100 and the GSP program stored in the data storage means 101 are activated (step S1501), and the current location is measured (step S1502). The current location data is transmitted from the transmission / reception means 104 to the server, and the transmission / reception means 201 of the server 200 receives this (step S1503).

  The voice processing language information creation unit 202 of the server 200 creates a dictionary from the current location data received in step S1503 and the service list of the complex facility managed in the server (step S1504).

  The dictionary is transmitted to the communication terminal 100 by the transmission / reception means 201 of the server 200, and the transmission / reception means 104 of the communication terminal 100 receives the dictionary (step S1505).

  Next, using the speech processing dictionary received in step S1505, speech recognition of the speech processing means 102 is activated (step S1506).

  Further, the browser stored in the first program and data storage means 101 in the communication terminal 100 is activated (step S1507), and the complex facility service page is displayed from the address of the current location acquired in step S1502 (step S1508).

  When a user utters during the display of the browser (step S1509), the voice processing means 102 recognizes the voice (step S1510), and command processing such as jumping to a link in the page and inputting characters is performed (step S1511). ). If the command is an end command, exit.

  Next, referring to FIG. 18 and FIG. 19, as an example of the program stored in the second program and data storage means 105 in FIG. 1, the infrared program stored in the first program and data storage means 101 and the sound The process and the operation of the program that causes the server 200 to cooperate with each other will be described.

  FIG. 18 and FIG. 19 are diagrams showing an example of a program that uses the infrared program and the voice processing means 102 in conjunction with each other. Specifically, the service at a complex facility such as a movie theater or a shopping mall is voiced. It is an example of the program for receiving by recognition or composition.

  In this program, it is assumed that infrared ports will be embedded in the fixtures such as walls, pillars, and furniture of the complex, and infrastructure such as terminals dedicated to infrared ports will be provided.

  FIG. 18 is a diagram for explaining a form of service, and represents a service list for each infrared port in the complex. In this example, it is represented by a port ID unique to each infrared ray, fixed position information such as a store name and an installation floor of each port, information for specifying a fixture or a terminal, and a service received from the port.

  For example, the infrared port with port ID = 001 is installed in the exhibition A installed on the A2 pillar on the first floor of the museum. From this port, it is possible to call an exhibit information providing service. Specifically, information related to exhibition A is read out by voice synthesis.

  The infrared port with port ID = 100 is installed on the T100 table in the Chinese restaurant B on the 6th floor of the West Building. From this port, menu description ordering services are movable. Specifically, the menu introduction is read out by speech synthesis or an order is placed through speech recognition.

  FIG. 19 is a diagram illustrating an operation procedure of a program that links an infrared program and audio processing. In this example, a program necessary for operation and a dictionary necessary for voice processing are generated in advance and stored in the first program and data storage means 101.

  First, the infrared program stored in the first program and data storage means 101 is activated (step S1701), and the infrared port ID of the current location is acquired (step S1702). The service program is switched according to the acquired port ID (step S1703).

  In the service program, since speech recognition or speech synthesis is set, the speech processing function is activated (step S1704).

  If there is a user input such as button input or utterance for the activated program (step S1705), voice processing is executed (step S1706). Specifically, in this process, an output by speech synthesis or a speech recognition result is converted into a command and executed. Further, when the result of the voice processing is finished (step S1707), the process is finished.

  Next, a second embodiment of the present invention will be described with reference to the drawings. FIG. 20 is a diagram showing the configuration of this embodiment. In the present embodiment, the speech processing language information creating unit 1103 refers to both the first program and data storage unit 1101 and the second program and data storage unit 1105 to generate speech language information. This embodiment is different from the first embodiment.

  FIG. 22 is a diagram for explaining the operation of the voice processing language information creation unit 1103.

  FIG. 22A is a diagram for explaining the program and data read from the first program and data storage unit 1103 by the speech processing language information creation unit 1103. In this example, the mailer and data managed by the mailer are used. It is assumed that certain address book data and mail data are used.

  FIG. 22B is a diagram for explaining the program and data read by the voice processing language information creation unit 1103 from the second program and data storage unit 1105. In this example, a product search program and product data are used.

  FIG. 22C is a diagram for explaining an example of the product data, and includes a product ID, a product name, and a product data file.

  FIG. 22D is a grammar template generated by the speech processing language information creation unit 1103. When this template is used, a grammar composed of a reading, a particle, a product name, and a mail command is generated. The mail command is composed of a mail and a command (mail). Refer to the address book for reading. The product name is referenced from the product data.

  Word strings to be registered in particles, mails, and commands (mails) that cannot be referenced from data managed by each program are given in advance.

  FIG. 22 (e) is a diagram for explaining that a grammar is generated by referring to data in a grammar template. After the reference, the utterance of “send product 001 by e-mail” is recognized by voice. This means that the grammar has been generated.

  As described above, the voice processing language information creation unit 1103 performs the voice processing by linking the program and data read from the first program and data storage unit 1101 with the program and data read from the second program and data storage unit 1101. By generating language information, a program specific to the terminal, a model specific to the user, information specific to the user, information that changes dynamically depending on the terminal status, and a general-purpose program or data created without depending on the model It is possible to control by voice processing such as voice recognition and voice synthesis.

  Next, a third embodiment of the present invention will be described with reference to the drawings. FIG. 23 is a diagram showing the configuration of the third exemplary embodiment of the present invention.

  In this embodiment, the communication terminal 2000 is integrated with the processing terminal 2000 and the communication terminal 2000 for integrating the speech language information generated by the communication terminal 2000 and the plurality of servers 200 to n00. The point that the means 2107 is provided is different from the first embodiment. .

  FIG. 24 is a diagram for explaining the operation of the speech processing language information integration unit 2107. The speech processing language information integration unit 2107 reads speech language information generated by each of the server 200, the server n00, and the communication terminal 2000, and generates speech language information.

  FIG. 25A is a diagram illustrating an example of speech language information generated by the server 200, and illustrates a configuration diagram of a speech recognition grammar for searching for information about a movie, and an example of a grammar corresponding to the configuration diagram. ing.

  This grammar is composed of [movie] which is an item related to the movie name, [item (200)] which is an item related to the movie, and [command (200)] which is an item for inquiry. A word string to be registered in is defined. By defining in this way, “Tell me the start time of movie A” can be recognized by this grammar.

  FIG. 25B is an example of speech language information generated by the server n00, and shows an example of a grammar corresponding to the structure of the speech recognition grammar for performing information search regarding stores and facilities.

  This grammar is an item for specifying a store or facility [store and facility name], an item related to the store and facility [item (n00)], and an item for inquiries [command (n00). ], And a word string to be registered in each item of [Category] and [Name] of [Store and facility name] is defined. With this definition, “how much is the budget for restaurant A” can be recognized with this grammar.

  FIG. 25C is an example of speech language information generated by the communication terminal 2000, and shows an example of the grammar corresponding to the configuration of the speech recognition grammar for performing the mail operation.

  This grammar consists of [Read], which is an item for specifying the mail address, [Mail], which is an item related to mail, and [Command (Mail)], which is an item for inquiries. A word string to be registered in the item is defined. By defining in this way, it is possible to recognize "send an email to Icchan" with this grammar.

  FIG. 25D is an example for synthesizing the speech language information by the speech processing language information integration unit 2107, and a grammar template for synthesizing the speech language information in FIGS. 25A to 25C. The structure of is shown. This grammar template assumes a grammar for sending movie and facility information by email. The grammar consists of [Reading] to specify the destination of the email, [Participant], [Movie and facility information] to specify the name and item of the movie or facility, and [Mail command to send the email. ]. The item [reading] is referred to from the grammar generated by the communication terminal 2000. The item [movie and facility information] includes [movie information], [particles], [facility information], and [particles].

  Furthermore, [movie information] includes [movie] and [item (200)], and [movie] and [item (200)] are referred to by the grammar generated by the server 200.

  [Facility information] is composed of [Store and facility name] and [Item (n00)], and [Store and facility name] and [Item (n00)] are referenced from the grammar generated by the server n00.

  FIG. 25E is a diagram for describing generation of a grammar with reference to data in a grammar template. By referring to the data, it means that a grammar for voice recognition is generated, which says, “I will send you the start time of movie A and the budget of restaurant A by email”.

  As described above, by synthesizing the speech language information generated in the plurality of servers by the speech processing language information creation unit and the speech language information generated in the communication terminal, the speech language information is generated. By linking a program, model, user-specific information, information that changes dynamically depending on the state of the terminal, and a general-purpose program created without depending on the model and its data, voice processing such as voice recognition and voice synthesis It becomes possible to control.

  Although the present invention has been described with reference to the above embodiment, the present invention is not limited to the configuration of the above embodiment, and various modifications that can be made by those skilled in the art within the scope of the present invention. Of course, modifications are included.

  INDUSTRIAL APPLICABILITY According to the present invention, it can be applied to a use in which voice processing such as voice recognition and voice synthesis is linked with a program built in the terminal and data managed by the program in a mobile phone or a portable terminal. In particular, the present invention can be applied to use of data and personal information that dynamically change depending on the state of a system managed by a terminal in cooperation with voice processing.

  Also, the present invention can be applied to a use in which a voice server and a program built in a terminal, data managed by the program, and a server are linked. Specifically, it can be applied to information retrieval guidance service use of a complex facility using a mobile phone. For example, in the above-described embodiments, the portable communication terminal has been described as an example, but it is needless to say that the present invention can be applied to any electronic device having a voice processing function.

It is a block diagram which shows the structure of the 1st Embodiment of this invention. It is a flowchart which shows the operation | movement of the 1st Embodiment of this invention. It is a figure which shows the specific example of the program memorize | stored in the communication terminal of the 1st Embodiment of this invention, and its data. It is a figure which shows the specific example of the program previously stored in the communication terminal of the 1st Embodiment of this invention, and its data. It is a figure which shows the specific example of the program previously stored in the communication terminal of the 1st Embodiment of this invention, and its data. It is a figure which shows the specific example of the program previously stored in the communication terminal of the 1st Embodiment of this invention, and its data. It is a figure which shows the specific example of the program previously stored in the communication terminal of the 1st Embodiment of this invention, and its data. It is a figure which shows the specific example of the program previously stored in the communication terminal of the 1st Embodiment of this invention, and its data. It is a figure which shows the specific example of the speech language information preparation for speech processing in the communication terminal of the 1st Embodiment of this invention. It is a figure which shows the specific example of the speech language information preparation for speech processing in the communication terminal of the 1st Embodiment of this invention. It is a figure which shows the specific example of the speech language information preparation for speech processing in the communication terminal of the 1st Embodiment of this invention. It is a figure which shows the specific example of the data stored in the server of the 1st Embodiment of this invention. It is a figure which shows the specific example of the speech language information preparation for speech processing in the server of the 1st Embodiment of this invention. It is a figure which shows the specific example of the speech language information preparation for speech processing in the server of the 1st Embodiment of this invention. It is a figure for showing an example of operation of a program downloaded from the outside of a terminal of a 1st embodiment of the present invention. It is a figure for showing an example of operation of a program downloaded from the outside of a terminal of a 1st embodiment of the present invention. It is a figure for showing an example of operation of a program downloaded from the outside of a terminal of a 1st embodiment of the present invention. It is a figure for showing an example of operation of a program downloaded from the outside of a terminal of a 1st embodiment of the present invention. It is a figure for showing an example of operation of a program downloaded from the outside of a terminal of a 1st embodiment of the present invention. It is a figure which shows the structure of the 2nd Embodiment of this invention. It is a flowchart which shows the operation | movement of the 2nd Embodiment of this invention. It is a figure which shows the specific example of the speech language information preparation for speech processing in the communication terminal of the 2nd Embodiment of this invention. It is a figure which shows the structure of the 3rd Embodiment of this invention. It is a flowchart which shows operation | movement of the 3rd Embodiment of this invention. It is a figure which shows the specific example of speech processing language information integration in the communication terminal of the 3rd Embodiment of this invention.

Explanation of symbols

100, 1000, 2000 Communication terminal 101, 1101, 2101 First program and data storage means 102, 1102, 2102 Voice processing means 103, 1103, 2103 Voice processing language information creation means 104, 1104, 2104 Transmission / reception means 105, 1105, 2105 Second program and data storage means 106, 1106, 2106 Control means 2107 Voice processing language information integration means 200, n00 server 201, n01 transmission / reception means 202, n02 Voice processing language information creation means

Claims (43)

A speech processing unit that performs speech recognition and / or speech synthesis processing;
A first storage unit that stores at least a program and / or data for realizing a predetermined function on a communication terminal;
Program and / or data that defines how to link the program and / or data input to the communication terminal from the outside of the communication terminal and stored in the first storage unit and the voice processing by the voice processing unit A second storage unit that stores at least
Using the program and / or data input from the outside of the communication terminal and the program and / or data stored in the first storage unit, voice processing by the voice processing unit, and the first A control unit that performs a control for cooperative operation of the function based on the program and / or data stored in the storage unit;
A communication terminal characterized by comprising:   The control unit starts the program stored in the second storage unit, and the started program calls the program stored in the first storage unit or uses data, The communication terminal according to claim 1, wherein voice processing by a voice processing unit and a program and / or data stored in the first storage unit are operated in a coordinated manner. A speech processing unit that performs speech recognition and / or speech synthesis processing;
A first storage unit that stores at least information held by the communication terminal;
A second storage unit that stores at least a program that is input to the communication terminal from the outside of the communication terminal and that defines a procedure for creating language information for speech processing;
A control unit that controls activation of a program that defines a procedure for creating language information for voice processing, which is stored in the second storage unit;
With
When activated, the program that defines the procedure for creating language information for voice processing is used for voice processing in the voice processing unit using at least the information stored in the first storage unit. Create language information,
The communication terminal, wherein the voice processing unit performs the voice processing using the created language information. The second storage unit is connected to a program and / or data input to the communication terminal from the outside of the communication terminal and stored in the first storage unit, and voice processing by the voice processing unit. Memorize the program that defines how,
The first storage unit is executed on the communication terminal and stores a program that realizes a predetermined function that is determined in advance or the program and data,
The control unit starts a program stored in the second storage unit, and cooperates with the audio processing by the audio processing unit and the program and / or data stored in the first storage unit The communication terminal according to claim 3, wherein:   The program and / or data input to the communication terminal from outside the communication terminal and stored in the second storage unit are transferred from a server to which the communication terminal communicates. The communication terminal as described in any one of thru | or 4. The communication terminal transmits information necessary for creating language information for voice processing to the server,
6. The communication terminal receives the language information for voice processing created on the server side that has received the language information for voice processing, and performs voice processing in the voice processing unit. The communication terminal described in 1. The first storage unit stores a program and / or data for generating language information for speech processing including at least one of a dictionary, a grammar, and a language model used in the speech processing unit. ,
The communication terminal includes a speech processing language information creation unit that creates language information for speech processing using at least a program and / or data stored in the first storage unit. The communication terminal according to any one of claims 1 to 4. The second storage unit includes a program and / or data that is input from the outside of the communication terminal and that generates language information including at least one of a dictionary, a grammar, and a language model that is used in the voice processing unit. Remembered,
The speech processing language information creation unit creates language information for speech processing using a program and / or data stored in the first storage unit and the second storage unit. The communication terminal according to claim 7.   The communication terminal synthesizes speech processing language information created by the speech processing language information creation unit of the communication terminal and one or more speech processing language information input from outside the communication terminal. The communication terminal according to claim 7, further comprising a language information integration unit for voice processing that creates the language information.   The language information for speech processing input from the outside of the communication terminal to the communication terminal is created by a server to which the communication terminal is connected for communication and transferred from the server to the communication terminal. The communication terminal according to claim 9.   The server apparatus which carries out communication connection with the communication terminal as described in any one of Claims 1 thru | or 10. A communication terminal that has a first storage unit that stores at least a program and / or data that realizes a predetermined function predetermined on the communication terminal, and that performs voice processing of voice recognition and / or voice synthesis,
From the outside of the communication terminal, at least a program that defines how to link the program and / or data stored in the first storage unit and the voice processing is input, and the input program is stored in the second storage. Storing in the department;
Using the program and / or data input from the outside of the communication terminal and the program and / or data stored in the first storage unit, voice processing by the voice processing unit, and the first A step of performing a control for cooperative operation of the function based on the program and / or data stored in the storage unit;
A speech processing method characterized by comprising:   The program stored in the second storage unit is activated, and the activated program calls the program stored in the first storage unit or uses data to generate a sound from the sound processing unit. The voice processing method according to claim 12, further comprising a step of causing a process and a program and / or data stored in the first storage unit to perform a cooperative operation. A communication terminal that has a first storage unit that stores at least information held by the communication terminal and performs voice processing for voice recognition and / or voice synthesis,
Inputting at least a program defining a procedure for creating language information for speech processing from the outside of the communication terminal, and storing the input program in a second storage unit;
Starting a program that defines a procedure for creating language information for speech processing, which is stored in the second storage unit;
A program that defines a procedure for creating the language information for voice processing stored in the second storage unit is at least used by the voice processing unit using information stored in the first storage unit. Creating language information used for voice processing of
Performing the speech processing using the created language information;
A speech processing method characterized by comprising: The second storage unit defines how to link the program and / or data input to the communication terminal from the outside of the communication terminal and stored in the first storage unit with the audio processing. Program to remember,
The first storage unit stores a program executed on the communication terminal and realizing a predetermined function or the program and data,
Including a step of activating a program stored in the second storage unit and causing the voice processing and the program and / or data stored in the first storage unit to operate in cooperation with each other. The voice processing method according to claim 14.   13. The program and / or data input to the communication terminal from outside the communication terminal and stored in the second storage unit are transferred from a server to which the communication terminal communicates. The voice processing method according to any one of 1 to 15. The communication terminal transmits information necessary for creating language information for voice processing to the server;
The server receives the information and creates language information for the voice processing;
The communication terminal receives the language information for voice processing created on the server side, and performs voice processing in the voice processing unit;
The voice processing method according to claim 16, further comprising: The first storage unit stores a program and / or data for generating language information for speech processing including at least one of a dictionary, a grammar, and a language model used in the speech processing unit. ,
16. The voice processing method according to claim 12, further comprising a step of creating the language information using at least a program and / or data stored in the first storage unit. . A program for generating language information for speech processing including at least one of a dictionary, a grammar, and a language model, which is input from the outside of the communication terminal and used in the speech processing unit, in the second storage unit, and Data is stored,
16. The method according to claim 12, further comprising a step of creating the language information using a program and / or data stored in the first storage unit and the second storage unit. The voice processing method described in 1.   Creates language information obtained by synthesizing the speech processing language information created by the speech processing language information creation unit of the communication terminal and one or more speech processing language information input from outside the communication terminal. The voice processing method according to claim 19, further comprising a step.   The speech processing language information input to the communication terminal from the outside of the communication terminal is created by a server to which the communication terminal communicates and is transferred from the server to the communication terminal. The voice processing method according to claim 20. A communication terminal, and a server for communication connection with the communication terminal,
The communication terminal includes voice processing means for performing recognition and / or synthesis voice processing;
Means for creating language information used by the voice processing means based on programs and / or data stored in advance in the communication terminal;
Means for operating the program and / or data stored in advance in the communication terminal and the voice processing using the language information in cooperation with the program and / or data downloaded from the server;
A communication system characterized by comprising: A communication terminal, and a server for communication connection with the communication terminal;
With
The communication terminal is
A first storage unit for storing a program and / or data stored and held in advance by the communication terminal;
A speech processing unit that performs speech processing of at least one of speech recognition and speech synthesis;
Voice processing language information creating means for creating language information used in the voice processing unit according to the program and / or data stored in the first storage unit;
Means for obtaining a program and / or data from the server;
A second storage unit for storing the program and / or data acquired from the server;
Based on the program and / or data stored in the second storage unit, the voice processing unit and control means for controlling the program and / or data in the first storage unit in cooperation with each other;
Including
The server is
Means for receiving information transmitted from the communication terminal and transmitting the program and / or data generated by the server to the communication terminal;
From the data stored on the server side based on the data transmitted from the communication terminal, voice processing language information creating means for creating language information used in the voice processing unit;
A communication system comprising: A first storage unit that stores programs and / or data stored in advance in the communication terminal;
A speech processing unit that performs speech processing of at least one of speech recognition and speech synthesis;
Speech processing language information creating means for creating language information used in the speech processing unit according to the program and / or data stored in the first storage unit;
Means for receiving programs and / or data from outside the communication terminal;
A second storage unit for storing a program and / or data received from the outside of the communication terminal; a program and / or data stored in the second storage unit; Control means for linking programs and / or data in the storage unit of
A communication terminal comprising: A server communicatively connected to the communication terminal according to claim 24,
Means for receiving information transmitted from the communication terminal and transmitting a program and / or data generated by a server to the communication terminal;
Voice processing language information creating means for creating language information used in the voice processing unit from data stored on the server side based on data transmitted from the communication terminal;
A server characterized by including: The communication terminal according to claim 24,
The first storage unit stores at least a state of the communication terminal,
The speech processing language information creating means creates the language information based on data that varies according to the state of the communication terminal. 26. The server of claim 25, wherein
The voice processing language information creation means receives from the communication terminal data that varies according to the state of the communication terminal stored in the first storage unit of the communication terminal, and the received data; A server characterized in that language information is created based on data stored and managed on the server side, and the created language information is transmitted to the communication terminal. A communication terminal generates language information used in voice processing based on a program and / or data transmitted from the outside of the communication terminal to the communication terminal and a program and / or data stored in advance in the communication terminal And a process of
The communication terminal using the language information for the voice processing to execute a program and / or data stored in advance in the communication terminal and a process for causing the voice processing to cooperate with each other;
A speech processing method characterized by comprising: In the computer that constitutes the communication terminal,
A process of storing a program and / or data stored in advance in the communication terminal;
A process of receiving a program and / or data generated outside the communication terminal;
Processing to store the received program and / or data;
Processing for performing at least one of speech recognition and speech synthesis;
Processing for generating speech processing language information for performing speech processing based on a program and / or data stored in advance in the communication terminal;
A process of linking audio processing with a program and / or data stored in advance in the communication terminal by the received program and / or data;
A program for running A communication terminal and a server;
The communication terminal is
A first storage unit for storing programs and / or data stored and held in advance in the communication terminal;
A speech processing unit that performs speech processing of at least one of speech recognition and speech synthesis;
Means for obtaining a program and / or data from the server;
A second storage unit for storing a program and / or data acquired from the server;
Voice processing language information creating means for creating language information used in the voice processing unit according to the program and / or data stored in the first storage unit and the second storage unit;
Control means for linking the audio processing unit and the program and / or data in the first storage unit using the program and / or data stored in the second storage unit,
Including
Means for the server to receive information from the communication terminal side and transmit the program and / or data generated by the server to the communication terminal side;
Based on the data transmitted from the communication terminal, voice processing language information creating means for creating a dictionary for voice processing from data stored on the server side;
A communication system comprising: A first storage unit for storing programs and / or data stored and held in advance in the communication terminal;
A speech processing unit that performs speech processing of at least one of speech recognition and speech synthesis;
Means for acquiring a program and / or data from outside the communication terminal;
A second storage unit for storing a program and / or data acquired from the outside of the communication terminal;
Voice processing language information creating means for creating language information used in the voice processing unit according to the program and / or data stored in the first storage unit and the second storage unit;
Control means for linking the sound processing unit and the program and / or data stored in the first storage unit using the program and / or data stored in the second storage unit ,
A communication terminal comprising: Means for receiving the information transmitted from the communication terminal according to claim 29 and transmitting the program and / or data generated on the server side to the communication terminal;
Voice processing language information creating means for creating a dictionary used in the voice processing unit from data transmitted from the communication terminal and data stored on the server side;
A server characterized by including: The communication terminal obtains a program and / or data downloaded from outside the communication terminal;
The communication terminal generates language information for voice processing used in voice processing based on the downloaded program and / or data and the program and / or data stored in the communication terminal in advance. When,
Using the language information for voice processing, the communication terminal causes a program stored in advance in the communication terminal, a program and / or data downloaded from the outside of the communication terminal, and the voice processing to cooperate with each other. Process,
A speech processing method characterized by comprising: In the computer that constitutes the communication terminal,
Processing for acquiring programs and / or data from outside the communication terminal;
Processing for generating language information for speech processing used in speech processing based on the acquired program and / or data and the program and / or data stored in advance in the communication terminal;
Processing for performing at least one of speech recognition and speech synthesis;
Using the language information for voice processing, a program stored in advance in the communication terminal, a program and / or data downloaded from the outside of the communication terminal, and a process for operating the voice processing in cooperation with each other;
A program for running Including a communication terminal and one or more servers,
The server receives information transmitted from the communication terminal, and transmits a program and / or data generated on the server side to the communication terminal;
Voice processing language information creating means for creating a dictionary for voice processing from data transmitted from the communication terminal and data stored on the server side;
Including
The communication terminal is
A first storage unit for storing programs and / or data stored and held in advance in the communication terminal;
A speech processing unit that performs speech processing of at least one of speech recognition and speech synthesis;
Means for obtaining a program and / or data from the server;
A second storage unit for storing a program and / or data acquired from the server;
Speech processing language information creating means for creating language information used in the speech processing unit in accordance with programs and / or data stored in both the first storage unit and the second storage unit;
Using the program and / or data stored in the second storage unit, the voice processing unit and control means for linking the program and / or data stored in the first storage unit;
Speech processing language information integrating means for combining speech processing language information created by the speech processing language information creating means and speech processing language information created by the speech processing language information creating means of the server; ,
A communication system comprising: A speech processing language for creating a speech processing dictionary from transmission / reception means for transmitting a program and / or data generated on the server side to the communication terminal, and data transmitted from the communication terminal and data stored on the server side A communication terminal that is communicatively connected to a server including the information creating means;
A first storage unit for storing programs and / or data stored and held in advance in the communication terminal;
A speech processing unit that performs at least one of speech recognition and speech synthesis;
Means for obtaining a program and / or data from the server;
A second storage unit for storing a program and / or data acquired from the server;
Speech processing language information creating means for creating language information used in the speech processing unit in accordance with a program and / or data stored in both the first storage unit and the second storage unit;
Using the acquired program and / or data, control means for linking the audio processing unit and the program and / or data of the first storage unit;
Speech processing language information integration means for synthesizing speech processing language information created by the speech processing language information creation means and speech processing language information created by the speech processing language information creation means of the server;
A communication terminal comprising: The communication terminal uses at least one of the program and / or data downloaded from one or more servers and the program and / or data stored in advance in the communication terminal in the communication terminal and / or the plurality of servers. Integrating the language information for speech processing generated in
The communication terminal uses the language information for voice processing to link the program and / or data stored in advance in the communication terminal, the program and / or data downloaded from the server, and the voice processing. A process of
A speech processing method characterized by comprising: In the computer that constitutes the communication terminal,
Processing for storing a program and / or data stored in advance in the communication terminal in the first storage unit;
Processing to receive programs and / or data from one or more servers outside the communication terminal;
A process of storing the received program and / or data in a second storage unit;
Processing to perform at least one of speech recognition and speech synthesis;
For voice processing for performing voice processing based on the program and / or data stored in the second storage unit and the program and / or data stored in advance in the first storage unit Processing to generate language information in the communication terminal;
Processing for integrating speech processing language information generated in the communication terminal or in the server;
A process of linking audio processing with a program and / or data stored in advance in the first storage unit by means of the program and / or data of the second storage unit;
A program for running A first processing unit that performs predetermined processing,
A first storage unit that is executed on the communication terminal and stores at least a program and / or data for realizing a predetermined function;
A program that defines the way in which the program and / or data input to the communication terminal from the outside of the communication terminal and stored in the first storage unit and the process by the first processing unit are linked; A second storage unit for storing at least data;
Using the program and / or data stored in the second storage unit, and further storing in the first storage unit using the program and / or data stored in the first storage unit A control unit that performs control for causing the function based on the program and / or data being performed and the processing performed by the first processing unit to operate in cooperation with each other;
A communication terminal characterized by comprising:   40. The communication terminal according to claim 39, wherein the first processing unit performs speech recognition and / or speech synthesis processing.   The server apparatus which carries out communication connection with the communication terminal of Claim 39 or 40, and transfers the said program and / or data memorize | stored in the said 2nd memory | storage part of the said communication terminal with respect to the said communication terminal. A first processing unit that performs predetermined processing,
A first storage unit for storing at least a program and / or data for realizing a predetermined function on the electronic device;
A program that defines how to link the program and / or data input to the electronic device from the outside of the electronic device and stored in the first storage unit with the processing by the first processing unit And / or a second storage unit that stores at least data;
Using the program and / or data stored in the second storage unit, and further storing in the first storage unit using the program and / or data stored in the first storage unit A control unit that performs control for causing the function based on the program and / or data being performed and the processing performed by the first processing unit to operate in cooperation with each other;
An electronic device comprising: 43. The electronic device according to claim 42, wherein the first processing unit performs speech recognition and / or speech synthesis processing.

Images