JPH10177468A - Mobile terminal voice recognition and data base retrieving communication system - Google Patents

Abstract

(57) [Summary] (Modified) [PROBLEMS] To realize a speech recognition / database search function as a user interface with practical accuracy and cost in a communication environment using a mobile terminal. SOLUTION: An audio signal from an input unit 109 is a PHS
The data is sent to the voice control host device 108 via the network 103, the mobile terminal control host device 104, and the Internet 105. The voice signal is recognized by the sentence voice recognition unit 117, the search keyword is extracted by the search keyword extraction unit 118, and the search keyword is returned to the mobile terminal 101. The user selects one of them and sends it back to the voice control host device 108. The search control unit 118 executes a search process on the database 120 via the SQL DBMS 119. The obtained search result sentence data is returned to the mobile terminal 101, and the communication unit 11
1 and received by the control unit 110 and displayed on the output unit 112.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a technology for recognizing voice such as a voice input in a mobile (portable) terminal device and searching a database based on the recognition result.

[0002]

2. Description of the Related Art There are various voice recognition technologies for recognizing voice signals, converting them into character data and storing the data, and providing recognition results to use of various services. It has been demanded in various industrial fields.

In recent years, with the development of speech recognition algorithms, speech recognition systems using a mainframe computer or a workstation computer have been developed.

[0004] These systems include various systems such as a bank balance inquiry system and a seat reservation system using telephone voice as input, and a delivery package sorting system that recognizes workers' voices and automatically delivers packages. It is being adopted by other industrial fields.

However, such a speech recognition system has
Only under the environment of a large-scale computer system as described above has reached a level having practical recognition accuracy, and under the environment of a small-sized computer system such as a so-called personal computer, the practical recognition accuracy has been reached. At present, an inexpensive speech recognition system having recognition accuracy has not been realized yet.

[0006] On the other hand, in parallel with the above information processing technology, mobile terminals such as car phones, mobile phones, and PHS (Personal Handy Phone System) have been rapidly spreading in recent years.

[0007] In particular, the PHS is small in size, and at the same time, the call charge is lower than that of a car phone or a mobile phone.
It has the feature of being able to talk with high quality "anytime, anywhere, with anyone" and is exploding. In addition, PHS is an ISDN (Integrated Services)
Since it is a public network using a Digital Network (Integrated Services Digital Network) as a backbone, high-speed digital communication at a transmission rate of 32 kilobits / second is possible, and expectations for its application to the multimedia communication field are increasing.

Further, in order to take advantage of the convenience of the mobile terminal,
It is also expected that it can be used as a multimedia information management / communication terminal device that can be used not only as a mobile phone device but also as a mobile information management device. Specifically, such a mobile terminal is expected to have not only a call function / fax function, but also a homepage access function and an e-mail communication function as an access function to the Internet and a company network. It is expected to have information management functions such as address management, schedule management, and database search / storage.

[0009] Such a mobile terminal is required to have a natural user interface that is as gentle as possible to a person so that the user can use it easily. As user interfaces currently realized, finger operation input using a keyboard or a mouse, handwriting input using an electronic pen, and the like have been put to practical use, but if they can respond to voice input and the like, they are ideal as user interfaces. . That is, for example, if it becomes possible to process a voice signal indicating the content of a call as data while utilizing a call function as a basic function, and to search a database on a mainframe based on the processing result, etc. The convenience can be greatly increased. Here, the value of applying a voice recognition function as a user interface to a mobile terminal can be found.

[0010] However, while the mobile terminal is small and has limited information processing capability, as described above, in the current speech recognition processing, the mobile terminal must be installed in a mainframe computer or workstation computer class environment. It is difficult to achieve practical recognition accuracy. Therefore, at present, there is a problem that it is very difficult to realize a voice recognition function as a user interface of a mobile terminal.

An object of the present invention is to realize a speech recognition function as a user interface with practical accuracy and cost in a communication environment using a mobile terminal, and to enable database search.

[0012]

The present invention first has a mobile terminal having the following configuration. In other words, the host connection means (the control unit 110 and the communication unit 111) is connected to a relay network (PHS network 10) composed of one or both of a wireless network and a wired network.
3 and the Internet 105) indirectly or directly to a voice control host device (voice control host device 108) as a host device without going through a relay network.

The voice input means (input section 109) inputs voice. After the connection operation by the host connection unit, the audio data transmission unit (the control unit 110 and the communication unit 111) transmits the audio data input from the audio input unit to the audio control host device.

Search keyword receiving means (control unit 110,
The communication unit 111) receives the search keyword returned from the voice control host device. Search keyword selection means (control unit 110, output unit 112, input unit 109) allows the user to select a desired search keyword from the received search keywords as a selected search keyword.

[0015] Selected search keyword transmission means (control unit 11
0, the communication unit 111) transmits the selected search keyword selected by the search keyword selection means to the voice control host device.

Search result data receiving means (control unit 110,
The communication unit 111) receives the search result data returned from the voice control host device. Search result data display / edit means (control unit 110, output unit 112) displays or edits the received search result data.

Next, the present invention has a voice control host device 108 having the following configuration. That is, mobile terminal connection means (packet transmitting / receiving section 115, mobile terminal communication control section 11
6) responds to the connection operation by the host connection means in the mobile terminal, and identifies and connects the mobile terminal.

Voice data receiving means (packet transmitting / receiving section 1)
15. The mobile terminal communication control unit 116) receives voice data for each currently connected mobile terminal. Voice recognition means (mobile terminal communication control unit 116, sentence voice recognition unit 117)
Executes voice recognition processing on voice data received by the voice data receiving means for each mobile terminal currently connected.

The search keyword extracting means (mobile terminal communication control section 116, search keyword extracting section 121), for each currently connected mobile terminal, extracts a search keyword from recognized voice data obtained by voice recognition processing by voice recognition means. Extract.

The search keyword extracting means includes, for example,
It includes the following search index creation means and search keyword output means. That is, first, the search index creation unit classifies the recognized speech data obtained by the speech recognition processing by the speech recognition unit for each currently connected mobile terminal according to a predetermined classification rule, and more specifically, for example, A search index is created by classifying words that appear in the input data in the order of the number of appearances. The search keyword output unit extracts, for each currently connected mobile terminal, a phrase that satisfies a predetermined extraction criterion from the search index created by the search index creation unit. The above words or words having a rank equal to or higher than a predetermined number of appearances are extracted, a predetermined unnecessary keyword is removed from the extracted phrases, and a new one of the resulting phrases is extracted as a search keyword. Note that the search index creation processing and the unnecessary keyword removal processing may be performed in the reverse order, which is also within the scope of the present invention.

The search keyword returning means (mobile terminal communication control unit 116, packet transmission / reception unit 115) sends, for each currently connected mobile terminal, the search keyword extracted by the search keyword extraction means to the corresponding mobile terminal. Send back.

The selected search keyword receiving means (packet transmitting / receiving section 115, mobile terminal communication control section 116) receives the selected search keyword for each currently connected mobile terminal.

Database means (SQL DBMS 11)
9, database 120) stores search result data as a database corresponding to a plurality of selected search keywords. The search result data includes various multimedia data such as voice and image in addition to text data.

The search control means (search control unit 118) searches, for each currently connected mobile terminal, search result data corresponding to the selected search keyword received by the selected search keyword receiving means from the database means. .

The search result data returning means (mobile terminal communication control section 116, packet transmitting / receiving section 115) corresponds to the search result data obtained by the search processing by the search control means for each currently connected mobile terminal. Reply to mobile terminal.

The mobile terminal according to the present invention including the mobile terminal and the voice control host device according to the present invention has a sophisticated voice recognition / database inspection system.
It is possible to provide a speech recognition / database search function having practical accuracy without providing a database search environment at a low cost.

In particular, according to the present invention, the user can easily specify only the really necessary search keyword as the selected search keyword, and can cause the voice control host device to execute the database search for the really necessary search keyword. Can be easily obtained.

The configuration of the invention described above can include the following limitations. That is, first, the mobile terminal has a personal handyphone system communication function (communication unit 111).

Next, the relay network includes a personal handy phone system communication network (PHS network 103) and the Internet (Internet 105). The voice control host device connects to the Internet.

Then, the host connection means in the mobile terminal comprises:
A mobile terminal control host device (mobile terminal control host device 104) having a gateway function between a public network including the personal handyphone system communication network and the Internet via a personal handyphone system communication network.
The mobile terminal control host device connects to the voice control host device via the Internet using the communication protocol on the Internet by calling and connecting to the voice control host device.

With this limited configuration, it is possible to provide a practically accurate voice recognition function at a lower cost and through a personal handyphone system communication network and the Internet, which are now spreading nationwide and worldwide. At the same time, the functions provided by the present invention can be seamlessly combined with the personal handyphone system call function and the Internet access function.

Further, the above communication protocol can include the following limitations. That is, the communication protocol is
It is a hierarchical protocol that includes an Internet Protocol (IP) layer and a Transmission Control Protocol (TCP) layer.

Next, an Internet Protocol datagram (IP datagram) which is packet data of an Internet protocol layer transmitted on the Internet.
The header (IP header) field stores a source Internet protocol address and a destination Internet protocol address designating addresses of a mobile terminal and a voice control host device on the Internet, and the data field of the Internet protocol datagram contains And a transmission control protocol segment which is packet data of the transmission control protocol layer.

The transmission control protocol segment (TCP segment) header (T
The CP header field stores a source port number and a destination port number for specifying a communication protocol for speech recognition / database search processing. The data field of the transmission control protocol segment includes a data field for identifying a mobile terminal. Terminal identification code, voice data, search keyword, selected search keyword, or search result data.

With this limited configuration, the mobile terminal and the voice control host device can be easily specified worldwide,
The coexistence of the voice recognition / database search processing service and other information processing services can be easily realized.

In the configuration of the present invention described above, the voice control host devices are mutually connected by a network, and mobile terminal connecting means, voice data receiving means, voice recognizing means, search keyword extracting means, search keyword reply means, selective search are provided. Keyword receiving means, database means, search control means,
In addition, the present invention can be realized by a plurality of host computers that realize the functions corresponding to the search result data returning means in a distributed manner.

With this limited configuration, load distribution on the host device side can be easily realized. Note that the above-described mobile terminal and voice control host device alone are also within the scope of the present invention.

[0038]

Embodiments of the present invention will be described below in detail with reference to the drawings. In the present embodiment, in a mobile terminal having a built-in PHS function, a voice signal input from a microphone during a call or in an off-line state is connected to a LAN in a specific voice service provider from the PHS network via the Internet. After being sent to the voice control host device, where the voice signal is recognized, the recognition result is automatically classified to extract search keywords, which are displayed on the mobile terminal. The user selects a desired one of the search keywords, and the selected search keyword obtained as a result is sent to the voice control host device again. In the voice control host device, the database is searched based on the selected search keyword, and database information on the selected search keyword selected by the user's intention in connection with the content of the input voice is returned to the mobile terminal in real time. And used for information management processing in the mobile terminal. This is,
This is a major feature related to the present invention. With such a system, mobile terminals do not need to provide a sophisticated voice recognition environment, can receive practically accurate voice recognition functions at low cost, and have an automatic database search function. can do. <System Configuration> FIG. 1 is an overall system configuration diagram of an embodiment of the present invention.

The mobile terminal 101 has a PHS terminal function.
It is connected to the HS network 103. The wireless base 102 is a public wireless base provided on a public telephone booth, a telephone pole, a building rooftop, an underground passage, or the like on a street, or a parent-child telephone device in a subscriber's house. When connected to the parent-child telephone device, the PH
It is directly connected to the public telephone network without going through the S network. In addition,
Instead of the wireless base 102, a configuration may be adopted in which the wireless base 102 is connected to the PHS network 103 or the public telephone network by wired communication via a wired connection device.

The PHS network 103 is a public telephone network or an ISD
It is interconnected with N networks, and a mobile terminal control host device 104 connected to the Internet 105 by a high-speed digital leased line or the like is connected to these networks.

The mobile terminal 101 is connected to the radio base 102 and the P
By automatically dialing up the mobile terminal control host device 104 connected to the public telephone network or the ISDN network via the HS network 103, it is possible to connect to the Internet 105.

A predetermined voice service provider LA is connected to the Internet 105 via a high-speed digital leased line or the like.
The router device 106 connected to N107 is connected. LAN 107 is an Ethernet system, ATM (As
Synchronous Transfer Mode) or FDDI. LAN 107
Is connected to a voice control host device 108.

After automatically dialing up the mobile terminal 101 to the mobile terminal control host device 104, the mobile terminal 101 sends the Internet 105, the router device 106, and the LAN 10
7 can communicate with the voice control host device 108.

Now, when the user instructs communication with the voice control host device 108 from the touch panel on the input unit 109 in the mobile terminal 101, the control unit 110 sends the voice control host device 108 to the communication unit 111. Request to start communication.

When the communication unit 111 is requested by the control unit 110 to start communication, the current mobile terminal control host device 104
If not connected to a wireless base (or wired connection device)
After making a wireless (or wired) call to 102 and connecting to the PHS network 103, a dial-up call is made by specifying the access telephone number of the mobile terminal control host device 104.

When the mobile terminal control host device 104 receives an incoming call, the communication unit 111 in the mobile terminal 101 first communicates with the connection establishing unit 113 in the mobile terminal control host device 104, thereby establishing a standard communication protocol on the Internet 105. Negotiation for establishing a connection by the TCP / IP and PPP methods. As a result, an IP address, which is an identification address on the Internet 105, is assigned from the mobile terminal control host device 104 to the communication unit 111 in the mobile terminal 101, and the mobile terminal 101
Can access the Internet 105.

If the communication unit 111 in the mobile terminal 101 is already connected to the mobile terminal control host device 104, the above dial-up transmission is omitted. Then, the mobile terminal 10
1 includes a “destination IP address” that is a preset IP address of the voice control host device 108 and a “source IP address” that is an IP address assigned by the mobile terminal control host device 104. And a "terminal identification code" (for example, a PHS telephone number) for identifying the mobile terminal 101, and a command to start a sentence speech recognition / database search process or a request to end a sentence speech recognition / database search process based on a user's designation. The TCP / IP packet storing the command and the
Send it out to 5.

Based on the “destination IP address” stored in the TCP / IP packet, the voice service is provided by the routing unit 114 in the mobile terminal control host device 104 and the relay host device (not shown) in the Internet 105. Router device 106 in the provider
After that, the packet is further transferred to the packet transmitting / receiving unit 115 in the voice control host device 108 via the LAN 107.

The packet transmitting / receiving unit 115 receives the received TC
From the P / IP packet, the “source IP address”
The terminal identification code and the sentence speech recognition / database search processing start request command or the sentence speech recognition / database search processing end request command are extracted and transferred to the mobile terminal communication control unit 116 in the speech control host device 108. .

The mobile terminal communication control unit 116 transmits the delivered “source IP address”, “terminal identification code”,
After registering information relating to a sentence speech recognition / database search process start request command or a sentence speech recognition / database search process end request command in a processing terminal registration table (FIG. 15) described later, the information is transmitted to the packet transmission / reception unit 115. A request is sent to the mobile terminal 101 to return a TCP / IP packet storing the permission data.

The packet transmitting / receiving unit 115
The P / IP packet is transmitted to the IP address corresponding to the mobile terminal 101. In this way, the voice control host device 108 can execute sentence voice recognition / database search processing on voice data transferred from the mobile terminal 101.

When the communication unit 111 in the mobile terminal 101 receives a TCP / IP packet storing the above-mentioned transmission permission data from the voice control host device 108, it passes the transmission permission data stored therein to the control unit 110.

After the transmission permission data is delivered, the control unit 110 in the mobile terminal 101 transmits the voice of the voice data input from the microphone to the communication unit 111 by a call operation or a voice input operation in an offline state. Request transmission to the control host device 108.

The communication unit 111 transmits the TCP / IP packet storing the voice data to the voice control host device 10.
8 is transmitted to the IP address corresponding to No. 8. This TC
The P / IP packet has the “destination IP” stored therein.
Based on the address, via the routing unit 114 in the mobile terminal control host device 104, the relay host device (not shown) in the Internet 105, the router device 106 in the voice service provider, and the LAN 107,
Packet transmission / reception unit 115 in voice control host device 108
Transferred to

The packet transmitting / receiving section 115 receives the received TC
The voice data stored in the P / IP packet is extracted and delivered to the mobile terminal communication control unit 116 in the voice control host device 108.

The mobile terminal communication control section 116 delivers the delivered speech data to the sentence speech recognition section 117. The sentence speech recognition unit 117 executes sentence speech recognition processing on the delivered speech data, and delivers the recognized speech sentence data as a recognition result to the search keyword extraction unit 121. The search keyword extraction unit 121 classifies the recognized voice sentence data into indices for each mobile terminal 101, extracts a search keyword while referring to an unnecessary keyword dictionary based on the search index obtained as a result, and extracts the search keyword from the mobile terminal communication control unit. Hand over to 116.

The mobile terminal communication control unit 116 transmits the TCP / IP packet storing the search index to the mobile terminal 1
Request a reply to 01. The packet transmitting / receiving unit 115
The corresponding TCP / IP packet is transmitted to the IP address corresponding to the mobile terminal 101.

When the communication unit 111 in the mobile terminal 101 receives a TCP / IP packet storing the above search keyword from the voice control host device 108, the communication unit 111 transfers the search keyword stored therein to the control unit 110.

The control unit 110 in the mobile terminal 101 outputs the search keyword to the output unit 112. Output unit 1
Numeral 12 displays the search keyword on the LCD display unit.
The user can select one of the displayed search keywords
A search keyword for which a database search is to be performed is selected as a selected search keyword by selecting the search keyword by touching the electronic pen or the like.

The control unit 110 in the mobile terminal 101 requests the communication unit 111 to transmit the selected search keyword to the voice control host device 108. Communication unit 111
Is the TCP / IP in which the selected search keyword is stored
The packet is transmitted to the IP corresponding to the voice control host device 108.
Send to address.

Based on the “destination IP address” stored in the TCP / IP packet, the TCP / IP packet has a routing unit 114 in the mobile terminal control host device 104, a relay host device (not shown) in the Internet 105,
Router device 106 in the voice service provider, and L
The data is transferred to the packet transmitting / receiving unit 115 in the voice control host device 108 via the AN 107.

The packet transmitting / receiving section 115 receives the received TC
The selected search keyword stored in the P / IP packet is extracted and delivered to the mobile terminal communication control unit 116 in the voice control host device 108.

The mobile terminal communication control unit 116 transfers the delivered selected search keyword to the search control unit 118.
The search control unit 118 controls the S
A request is made to a QLDBMS (structured query language database management system) 119 (FIG. 1) to make an inquiry to the database 120 in the voice control host device 108 using the passed selected search keyword. As a result, the search control unit 118
Based on the search results returned from the DBMS 119, search result sentence data is generated, and is generated by the mobile terminal communication control unit 11.
Hand over to 6.

The mobile terminal communication control section 116 requests the mobile terminal 101 to return a TCP / IP packet storing the search result text data. Packet transmitting / receiving unit 115
Transmits the corresponding TCP / IP packet to the mobile terminal 101.
To the IP address corresponding to.

When the communication unit 111 in the mobile terminal 101 receives the TCP / IP packet storing the search result sentence data from the voice control host device 108, the communication unit 111 sends the search result sentence data stored therein to the control unit 110. hand over.

The control unit 110 in the mobile terminal 101 outputs the search result sentence data to the output unit 112. The output unit 112 outputs a sentence corresponding to the search result sentence data to L
Display on the CD display. The user writes this sentence data
It can be arbitrarily stored or processed.

On the other hand, in addition to communication with the voice control host device 108, the mobile terminal 101 transmits a dial-up call to the mobile terminal control host device 104 using a homepage browsing tool or an e-mail provided with the voice control host device 108. , It is possible to freely access desired resources on the Internet 105. <External Configuration of Mobile Terminal 101> FIG. 2 is an external view of the mobile terminal 101 of FIG.

The mobile terminal 101 has the appearance of a compact portable information management device, and has a microphone 201 also serving as a transmitter for inputting voice, and a microphone 201 for inputting an image which is not particularly related to the present invention. 1. The camera 202, an LCD display unit 203 having a touch panel function of displaying various information and receiving a touch input or a pen input, and a speaker 204 also serving as a receiver for outputting voice. It has a wireless antenna 205 for transmitting to the base 102 and a socket 206 for connecting to a wired connection device instead of the wireless base 102.

Further, an I for inserting various IC cards is provided.
An optical transceiver 208 for performing infrared optical communication between the C card slot 207 and another mobile terminal 101 or a personal computer or the like is provided.

The switch 209 is a power switch. <Functional Block Configuration of Mobile Terminal 101> FIG. 3 is a functional block diagram of the mobile terminal 101.

As shown in FIG. 1, the mobile terminal 101 comprises an input unit 109, a control unit 110, a communication unit 111, and an output unit 112, and are connected to each other by a bus 326.

First, the input unit 109 includes a portion for inputting voice, a portion for inputting an image not particularly related to the present invention, and a portion of a touch panel mechanism described later in the operation of the output unit 112. .

The part for inputting sound is the microphone 301, A
It comprises a / D conversion unit 302 and a microphone control unit 303. The microphone 301 (corresponding to 201 in FIG. 2) is a PHS
Also serves as a telephone transmitter, and inputs a voice uttered by the user.

The A / D converter 302 converts an analog audio signal input from the microphone 301 into digital audio data, and further converts the digital audio data into an ADPCM (Adaptive Differential) signal which is a PHS standard audio encoding system.
l Pulse Code Modulation: Adaptive differential linear pulse coding
Encode according to the method. This part has already been put to practical use as an LSI integrated circuit constituting a PHS terminal.

The microphone control unit 303 transfers the coded voice data to the communication control unit 321 in the communication unit 111 during a call and places it on a communication channel. Control unit 1
10 to the RAM 317.

On the other hand, a portion for inputting an image is a CCD (Ch
arge Coupled Device) Camera 304, A / D converter 3
05, a memory 306, and a camera control unit 307.

The CCD camera 304 captures an arbitrary image based on a user operation. The A / D conversion unit 305
An analog video signal captured by the CCD camera 304 is converted into digital image data.

The memory 306 stores digital image data in frame units. The camera control unit 307 includes a CCD
Camera 304, A / D converter 305, and memory 306
Control the operation of.

Next, the output unit 112 is composed of a part for outputting a sound and a part for outputting an image. The part that outputs audio includes a speaker 308, a D / A converter 309,
And a speaker control unit 310.

The speaker control unit 310 transmits the PHS call voice data received from the communication control unit 321 in the communication unit 111 or the synthesized voice data received from the RAM 317 in the control unit 110 to the D / A conversion unit 309. Forward.

The D / A converter 309 decodes the received audio data, converts it into an analog audio signal, and emits it as sound from the speaker 308 (corresponding to 204 in FIG. 2).

The portion for outputting the image is the LCD display unit 20
3, LCD driver 312, memory 313, and LCD
It comprises a control unit 314. The LCD control unit 314
Various image data such as character data, image data, and command button data received from the RAM 317 in the control unit 110 are stored in the memory 313 in frame units.
The CD driver 312 is started.

The LCD driver 312 displays the image data read from the memory 313 in frame units on the LCD display unit 311 (corresponding to 203 in FIG. 2). In addition,
A transparent touch panel is provided on the surface of the LCD display unit 311 (203 in FIG. 2). The user touches the touch panel with his / her finger or pen in accordance with command button data displayed on the LCD display unit 311. Command input. This input signal is transferred by the touch panel control unit 315 to the RAM 317 in the control unit 110.

Subsequently, the control unit 110 controls the CPU 316,
It comprises a RAM 317 and a ROM 318, an IC card interface unit 319, and an IC card 320 inserted into an IC card slot 207 (FIG. 2) as required.

The CPU 316 controls the operation of the entire mobile terminal 101 in accordance with the control program stored in the ROM 318 while using the RAM 317 as a work area.

The IC card interface unit 319
The input / output of data to / from the C card 320 is controlled. Lastly, the communication unit 111 includes a communication control unit 321, a wireless driver 322, a wireless antenna 323, a wired driver 324,
And a socket 325.

The communication control unit 321 executes a PHS call process and a TCP / IP communication process (described later) with the Internet 105, and controls the wireless driver 322 or the wired driver 324.

The wireless driver 322 mutually converts communication data with a PHS wireless signal transmitted and received via the wireless antenna 323 (corresponding to 205 in FIG. 2) during wireless communication. The PHS radio signal has a radio frequency of 1.9 GHz, a carrier frequency interval of 300 kHz, a TDMA-TDD radio access scheme of 4 channels / carrier,
π / 4 shift QPSK modulation method and 384 kbits /
This is a wireless signal based on the wireless transmission speed of sec.

On the other hand, at the time of wired communication, the wired driver 324 mutually converts communication data with a wired signal transmitted / received via the socket 325 (corresponding to 206 in FIG. 2). This is a typical telephone band modem modulated signal.

The operation of the embodiment of the present invention having the above configuration will be described in detail below. <Process of Mobile Terminal 101> First, the process of the mobile terminal 101 will be described.

FIG. 4 shows the CPU 3 in the control unit 110 shown in FIG.
16 stores the ROM 318 in the control unit 110 after the power is turned on.
4 is an overall operation flowchart showing a control operation realized as an operation of executing a control program stored in the control program.

A control program for realizing each function shown in the operation flowcharts of FIGS. 4, 5, 6, and 9 and data necessary for the control program are, for example, attached to and detached from the IC card slot 207 shown in FIG. Flexible IC card 3
20 is stored in the form of a program code readable by the CPU 316, and the program code is stored in the CPU 3.
16 or the program code is stored in RAM 317 or a writable ROM as needed.
318 may be configured to be loaded and executed by the CPU 316. Alternatively, the above-described control program and data necessary for the control program are received from another device via a communication unit 111 from a wireless or wired communication line or an optical transceiver 208 (FIG. 2), and stored in a RAM 317 or a writable ROM 318. It may be configured to be loaded and executed by the CPU 316.

First, steps 401 → 412 → 402 →
In the repetition loop of 403 → 404 → 401, the touch panel control unit 315 shown in FIG. 3 determines whether or not a touch panel input has been detected (401), and receives a search keyword from the voice control host device 108 (FIG. 1). (412), whether or not search result sentence data has been received from the voice control host device 108 (FIG. 1) (402), other reception / display processing (403), and necessary A data transmission process (404) is executed.

Touch panel control section 315 notifies of touch panel input detection, and determination in step 401 is YES.
Then, in step 405 or 406, the touch panel input is performed by the CCD camera 304 in FIG. 3 (202 in FIG. 2).
It is determined whether or not the input instruction is the input instruction of the microphone 301 of FIG. 3 (201 of FIG. 2).

The touch panel input is the CCD camera 3 shown in FIG.
04 (202 in FIG. 2) and the
If the determination at 5 is YES, at step 407, the camera control unit 307 in the input unit 109 in FIG. 3 is instructed to start input processing of, for example, a handwritten character image. Thereafter, the process proceeds to the transmission process of step 404. Since the image input processing is not particularly related to the present invention, a detailed description thereof will be omitted.

The touch panel input is the microphone 301 shown in FIG.
If the input instruction is (201 in FIG. 2) and the determination in step 406 is YES, in step 408, the microphone control unit 303 in the input unit 109 in FIG. 3 is instructed to start the voice input process. This voice input processing start instruction is
For example, it is an instruction to start a PHS call process or an instruction to start a speech input process in an offline state for executing a sentence speech recognition / database search process.

The microphone control unit 303 is provided with the CPU 31
6, the microphone 301 (201 in FIG. 2)
And instruct the A / D converter 302 to start voice input. As a result, the audio data input from the microphone 301 (201 in FIG. 2) is output from the A / D converter 302.

After that, when the above-mentioned voice input processing start instruction is an instruction to start a PHS call, the above-mentioned voice data is loaded on a predetermined communication channel by a transmission processing (not shown) of the communication control unit 321. And sent to the other party.

When the above-described voice input processing start instruction includes a voice input processing start instruction for sentence voice recognition / database search processing, the microphone 301 (201 in FIG. 2) thereafter inputs the voice input processing start instruction. The audio data output from the control unit 303 is transmitted to the audio control host device 108 in the transmission processing in step 404 described below.

The touch panel input is the CCD camera 3 shown in FIG.
04 (202 in FIG. 2), the microphone 30 in FIG.
If the input instruction is not the input instruction 1 (201 in FIG. 2), the determinations in steps 405 and 406 are NO, and in step 409, another key input processing is executed. Thereafter, the process proceeds to the transmission process of step 404.

On the other hand, the voice control host device 108 (FIG. 1)
From the RAM 31 in the control unit 110 via the communication unit 111
7, the search index is received, and steps 401 → 4
If the determination in step 412 in the iteration loop of 12 → 402 → 403 → 404 → 401 is YES, in step 413, the search keyword is stored in the RAM 31
7 to the memory 313 in the output unit 112
The display of the search keyword is instructed to the D control unit 314.

As a result, under the control of the LCD control unit 314, the received search keyword is displayed on the LCD display unit 311 (203 in FIG. 2) from the memory 313 via the LCD driver 312. A plurality of the search keywords are displayed at the same time, and the touch panel on the LCD display unit 203 is activated by the touch panel control unit 315 so that the user can select any one of the search keywords. In response to this selection operation, the determination in step 401 becomes YES, and the operation is detected in step 409 described later.

Also, the voice control host device 108 (FIG. 1)
From the RAM 31 in the control unit 110 via the communication unit 111
7, the search result sentence data is received, and step 401 →
If the determination in step 402 in the iteration loop of 412 → 402 → 403 → 404 → 401 is YES, in step 410, the search result sentence data is
M317 is transferred to the memory 313 in the output unit 112, and the LCD control unit 314 is instructed to display the search result sentence data.

As a result, under the control of the LCD control unit 314, the received search result sentence data is displayed on the LCD display unit 311 (203 in FIG. 2) from the memory 313 via the LCD driver 312.

Next, the transmission processing in step 404 will be described. 5 and 6 are operation flowcharts showing the details of the transmission processing. First, in step 501, it is determined whether or not a key input from the touch panel processed by the other key input processing in step 409 in FIG. 4 is accompanied by a transmission instruction. The key input for selecting a search keyword is performed in step 514 of FIG.
Is detected by

If this determination is NO, step 50
Proceed to step 5. If the determination in step 501 is YES, in step 502, it is determined whether the mobile terminal 101 is currently connected to the mobile terminal control host device 104 in FIG.

If mobile terminal 101 is currently connected to mobile terminal control host device 104 in FIG. 1 and the determination in step 502 is YES, CPU 316 in control unit 110 in FIG.
Requests the communication control unit 321 in the communication unit 111 in FIG. 3 to transmit a “terminal identification code” of the mobile terminal 101 and a command corresponding to the key input process in step 504. As a result, the communication control unit 321 generates a TCP / IP packet storing the “terminal identification code” and the command, and transmits the TCP / IP packet to a predetermined host connected to the Internet 105 (for example, the voice control host device in FIG. 1). 1
08).

If the mobile terminal 101 is not currently connected to the mobile terminal control host device 104 in FIG. 1 and the determination in step 502 is NO, the CPU 316 in the control unit 110 in FIG.
Requests the communication control unit 321 in the communication unit 111 of FIG. 3 to perform a transmission process in step 503, and then executes step 504.

As will be described in detail later, the transmission instruction of the command to start the sentence speech recognition / database search processing and the transmission instruction of the termination request command of the sentence speech recognition / database search processing based on the user's designation are made in step 504 described above.
Issued at

As described above, the determination at step 501 is N
In the case of O or after the processing of step 504, step 50
In step 5, in step 408 of FIG.
An instruction to start voice input processing for database search processing has been executed, and the voice control host device 1 for voice data has been executed.
08 (FIG. 1) is determined.

If this determination is NO, step 51
Proceed to process 0. If the determination in step 505 is YES, in step 506, it is determined whether or not transmission permission data, which is a response to the command to start the sentence voice recognition / database search processing, has already been returned from the voice control host device. You.

If this determination is NO, the voice control host device 108 still recognizes the sentence voice recognition /
Since the preparation for the database search process start request command has not been completed, the process proceeds to step 510.

If the transmission permission data, which is a response to the command for requesting the start of the sentence speech recognition / database search process, has already been returned from the speech control host device 108, and if the determination in step 506 is YES, the process proceeds to step 50.
At 7, it is determined whether the mobile terminal 101 is currently connected to the mobile terminal control host device 104 of FIG.

If mobile terminal 101 is currently connected to mobile terminal control host device 104 in FIG. 1 and the determination in step 507 is YES, CPU 316 in control unit 110 in FIG.
In step 509, the microphone control unit 303 in the input unit 109 shown in FIG.
The communication control section 321 in the communication section 111 is requested to transmit the voice data transferred to the communication section 7. As a result, the communication control unit 321 generates a TCP / IP packet in which the above-mentioned voice data is stored, and transmits the TCP / IP packet to the voice control host device 10 of FIG.
Send to 8

If mobile terminal 101 is not currently connected to mobile terminal control host device 104 in FIG. 1 and the determination in step 507 is NO, CPU 316 in control unit 110 in FIG.
Requests the communication control unit 321 in the communication unit 111 of FIG. 3 to perform a transmission process in step 508, and then executes step 509.

As will be described in detail later, an instruction to transmit voice data for sentence voice recognition / database search processing is issued in step 509 described above. As described above, when the determination in step 505 or 506 is NO or after the processing in step 509, in step 510, a start instruction of the image input processing is executed by step 407 in FIG. The Internet 105
It is determined whether or not a transmission instruction to an image control host device (not shown) connected to the image control host device is transmitted.

If the determination is NO, the process proceeds to step 514 in FIG. YES at step 510
In step 511, it is determined whether the mobile terminal 101 is currently connected to the mobile terminal control host device 104 in FIG.

If the mobile terminal 101 is currently connected to the mobile terminal control host device 104 in FIG. 1 and the determination in step 511 is YES, the CPU 316 in the control unit 110 in FIG.
Requests the communication control unit 321 in the communication unit 111 to transmit the image data obtained in the memory 306 in the input unit 109 shown in FIG. As a result, the communication control unit 321 generates a TCP / IP packet storing the image data, and transmits the TCP / IP packet to the image control host device 108 (not shown) connected to the Internet 105.

If mobile terminal 101 is not currently connected to mobile terminal control host device 104 in FIG. 1 and the determination in step 511 is NO, CPU 316 in control unit 110 in FIG.
Requests the communication control unit 321 in the communication unit 111 of FIG. 3 to perform a transmission process in step 512, and then executes step 513.

Note that the image data transmission instruction in step 513 is not particularly related to the present invention, and therefore detailed description thereof is omitted. As described above, if the determination in step 510 is NO or after the processing in step 513, in step 514 in FIG. 6, the key input from the touch panel processed by the other key input processing in step 409 in FIG. Is determined to accompany the transmission instruction.

If the determination is NO, the transmission processing of step 404 in FIG. 4 ends. If the determination in step 514 is YES, in step 515, the mobile terminal 10
1 is currently connected to the mobile terminal control host device 104 in FIG.

If mobile terminal 101 is currently connected to mobile terminal control host device 104 in FIG. 1 and the determination in step 515 is YES, CPU 316 in control unit 110 in FIG.
Sends an instruction to transmit the selected search keyword selected in step 517 by the key input process to the communication unit 11 in FIG.
1 to the communication control unit 321. As a result, the communication control unit 321 generates a TCP / IP packet in which the selected search keyword is stored, and transmits the TCP / IP packet to the voice control host device 1 of FIG.
08.

If mobile terminal 101 is not currently connected to mobile terminal control host device 104 in FIG. 1 and the determination in step 515 is NO, CPU 316 in control unit 110 in FIG.
Requests the communication control unit 321 in the communication unit 111 of FIG. 3 to perform a transmission process in step 516, and then executes step 517.

As described above, the determination at step 514 is N
In the case of O or after the processing of step 517, the transmission processing of step 404 in FIG. 4 ends. <Format of Communication Data> FIG.
And mobile terminal control host device 104 and Internet 1
FIG. 5 is a format diagram of communication data communicated with the MFP 05 (voice control host device 108).

Communication data between the mobile terminal 101 and the mobile terminal control host device 104 is PPP (Point-to-P
oint Protocol).
Using the PPP frame (transferred from left to right in the figure) shown in FIG.
It is transmitted over a digital communication channel having a transmission rate of s / sec.

"Flag", which constitutes the PPP frame,
In each of the "address" and "control" fields, each fixed bit string shown in FIG. 7A is set. The FCS having a data length of 2 octets is called a frame check sequence, and is data for error detection / correction of PPP frame data. “Information” of a PPP frame transferred after a PPP link is established between the mobile terminal 101 and the mobile terminal control host device 104
The field (having a variable data length) includes I, which is a basic transmission unit of data on the Internet 105 (FIG. 1).
P datagram is stored, in which case a "protocol" field having a data length of 2 octets contains:
A hexadecimal value “0021” indicating that the IP datagram is stored is stored in the “information” field.

"Information" of PPP frame
The IP datagram is stored in the field as described above. This IP datagram is a basic transmission unit of data on the Internet 105 as described above. IP
The datagram is defined in accordance with the Internet Protocol (IP), provides a function for uniquely transferring data stored in its “data” field to a destination host device on the Internet 105, and has an address on the Internet 105. , The function of transferring the IP datagram itself to the host specified by the "destination IP address" through a fixed route on the Internet 105, and the fragmentation (division) and reassembly of the IP datagram itself. It has functions and the like.

An IP datagram is composed of an IP header field and a data field, as shown in FIG. The IP header field contains all the information necessary to deliver the IP datagram containing it. FIG. 8A is a format diagram of the IP header.

The IP header has a data length of 5 to 6 words with 32 bits as one word. This data length is stored in the “header length” field of the first word. The length is stored in the first word of the "total length of IP datagram" field.

In the "version" field of the first word, the version of the Internet Protocol (IP) that defines the method of transmitting the IP datagram is set. The current version is 4.

In the “service type” field of the first word, information indicating the priority of delivery is stored.
This is not particularly relevant to the present invention. Each field of the second word indicates that the IP datagram is
The control information in the case of fragmentation (division) due to the above-described transfer restriction is defined. First, the “identification number”
The field contains this fragmented I
A unique integer for identifying the undivided IP datagram to which the P datagram belongs is set. Next, in the “fragment offset” field, offset information indicating which part of the IP datagram before the division corresponds to the IP datagram that is the divided fragment is set. In the "flag string" field, it is set whether or not another fragment constituting the undivided IP datagram to which this divided IP datagram belongs is followed by the divided fragment. With the above information, even if the IP datagram is fragmented at the relay host on the Internet 105,
The IP datagram before division can be accurately restored on the receiving side.

The “lifetime” of the third word (TTL: Time
In the “To Live” field, time information in seconds indicating how long the IP datagram is allowed to exist on the Internet 105 is set. The relay host on the Internet 105 subtracts the above field value each time the IP datagram is processed, and discards the IP datagram whose value becomes 0 or less from the Internet 105. This suppresses the occurrence of excessive traffic on the Internet 105. The retransmission control for the discarded IP datagram is executed in a control process for a TCP segment stored in the IP datagram.

In the "protocol" field of the third word, an integer value for defining the format of data stored in the "data" field of the IP datagram is set. In the case of the present embodiment, as shown in FIG. 7 (c), since the TCP data is stored in the "data" field of the IP datagram, an integer value 6 defining the format is set. You.

In the “checksum of header” field of the third word, checksum data for detecting an error in the data of the IP header is set. A 32-bit “source IP address” is set in the fourth word. For example, when an IP datagram is transferred from the mobile terminal 101 to the voice control host device 108, the “source IP address” is assigned from the mobile terminal control host device 104 to the mobile terminal 101 by a transmission process described later. The set IP address is set. By storing the “source IP address”, the voice control host device 108 in FIG. 1 can return a search keyword or search result text data to the mobile terminal 101 via the Internet 105.

In the fifth word, a 32-bit “destination IP”
For example, when an IP datagram is transferred from the mobile terminal 101 to the voice control host device 108, the “destination IP address” is fixedly assigned to the voice control host device 108. The IP address is set.The routing unit 114 in the mobile terminal control host device 104, each relay host device on the Internet 105, and the router device 106 in the voice service provider are stored in the received IP datagram. By identifying the "destination IP address", the delivery route of the IP datagram is determined in advance according to the routing control table information of each device, and finally the IP datagram is transferred to the voice control host device in the voice service provider. 108.

The "IP option" field of the sixth word is optional, and is used for information for testing or debugging each network constituting the Internet 105 and for controlling or monitoring a delivery route on the Internet 105. Information and the like are set, but this is not particularly relevant to the present invention.

In the “padding” field of the sixth word, padding data for adjusting the data length is set. Next, TCP segment data is stored in the "data" field of the IP datagram. This T
The CP segment is defined according to the Transmission Control Protocol (TCP), and its "data"
The data stored in the field is transferred to the Internet 105
A function is provided for accurately delivering the packet to the above destination host device in an appropriate order. IP datagrams provide only the unique transfer function of data on the Internet 105, and the function of ensuring data reliability (retransmission control function, etc.)
Is provided, whereas the TCP segment provides a function for ensuring data reliability.

As described above, the communication data has the hierarchical structure of the IP datagram (with the PPP frame) and the TCP segment. Therefore, it is necessary to efficiently deliver the data on the Internet 105 under a processing load as small as possible. Yes, in order to efficiently address the different demands of achieving the most reliable data delivery end-to-end. As a result, the relay host device on the Internet 105 refers to only the IP header of the IP datagram, so that the information (TCP segment) stored in the “data” field of the IP datagram is as fast and efficiently as possible. To the destination host device, and between end-to-end (source host device and destination host device), TCP
By referring to the TCP header of the segment, highly reliable data communication such as retransmission control can be realized.

As shown in FIG. 7B, the TCP segment is composed of a TCP header field and a data field. FIG. 8B is a format diagram of the TCP header.

As in the case of the IP header, the TCP header has a data length of 5 to 6 words with 32 bits as one word, and this data length is stored in the “header length” field of the fourth word. Also, the data length of the entire IP datagram is stored in the “total length of IP datagram” field of the first word.

In the “source port number” field and the “destination port number” field of the first word, a 16-bit integer value specifying a communication protocol for sentence speech recognition / database search processing is set.

The packet transmission / reception unit 115 (FIG. 1) in the voice control host device 108 includes, in addition to the TCP segment storing the voice data and the selected search keyword for the sentence voice recognition / database search processing, the e-mail data In order to transmit and receive various TCP segments storing various data including the TCP segment, by recognizing the value of the “destination port number” field set in the TCP header of the received TCP segment, It is possible to determine to which application to be executed by the voice control host device the data stored in the "data" field.

The packet transmission / reception unit 115 indicates that the value of the “destination port number” field set in the TCP header of the received TCP segment indicates a value corresponding to the communication protocol for sentence speech recognition / database search processing. If so, the voice data or the selected search keyword stored in the “data” field of the TCP segment can be delivered to the mobile terminal communication control unit 116.

Similarly, the communication control unit 321 (FIG. 3) in the communication unit 111 of the mobile terminal 101 also includes homepage data and e-mail data in addition to the TCP segment in which the search result sentence data is stored. To send and receive various TCP segments storing various data,
By recognizing the value of the “destination port number” field set in the TCP header of the received TCP segment, the data stored in the “data” field of the TCP segment can be transmitted to any application executed by the mobile terminal 101. You can decide to hand it over.

Then, the communication control unit 321 transmits the received T
If the value of the “destination port number” field set in the TCP header of the CP segment indicates a value corresponding to a communication protocol for sentence speech recognition / database search processing, the control unit 110 (FIG. 1, FIG. In FIG. 3), the reception of data for sentence speech recognition / database search processing is notified, and the search keyword or search result sentence data stored in the “data” field of the TCP segment can be delivered.

Furthermore, the packet transmission / reception unit 115 in the voice control host device 108 and the communication unit 111 of the mobile terminal 101
The communication control unit 321 in the above can confirm the source application by confirming the “source port number” set in the TCP header of the received TCP segment.

Next, the second TCP header shown in FIG.
The word "sequence number" field contains the current TC
From the entire byte stream transmitted from the transmission side to the reception side in the P connection, the transmission side determines from which byte of the byte stream the data stored in the "data" field of this TCP segment corresponds. This is a field for notifying the receiving side. Conversely, the “acknowledgement number” field of the third word indicates the number of bytes that the receiving side has received without error in the entire byte stream transmitted from the transmitting side to the receiving side in the current TCP connection. This is a field for notification from the receiving side to the transmitting side. As a result, for example, the voice data can be transferred from the mobile terminal 101 to the voice control host device 108 in the correct order and with high reliability.

In the "flag string" field of the fourth word, a value indicating the type of the TCP segment is set. T
In the CP communication, for example, various control data for acknowledgment is transmitted at the start or end of the connection, for example, and the type of the control data is set in the “flag string” field.

The "window" field of the fourth word is a field for notifying the transmitting side of window data indicating how many bytes of data the receiving side can currently receive. is there. This enables data flow control from the receiving side to the transmitting side. For example, when the load on the voice control host device 108 is high, the transmission of voice data to the mobile terminal 101 is suppressed. Fine control is possible.

The “reserved” field of the fourth word is a field for reservation. The “checksum” field of the fifth word stores checksum data for detecting an error in the data stored in the TCP header and the “data” field. Thereby, for example, the voice control host device 108 can correctly receive the search keyword or the voice data from the mobile terminal 101.

The fifth word "urgent pointer" is control data for communicating urgent data (interrupt data, abort data, etc.), but this is not particularly relevant to the present invention.

The "option" field of the sixth word is used, for example, to specify the maximum segment length that can be communicated between the transmitting and receiving devices, but this is not particularly relevant to the present invention.

[0153] In the "padding" field of the sixth word, padding data for adjusting the data length is set. The communication (termination) processing function of the TCP segment having the above-described configuration is realized by the communication control unit 321 (FIG. 3) in the communication unit 111 in the mobile terminal 101, and the packet transmission / reception unit 115 ( 1). Note that the control program executed by the CPU 316 in the mobile terminal 101 may be configured to realize the above processing functions. <Outgoing Call Processing> As described above, the control unit 1 of the mobile terminal 101
The CPU 316 in FIG. 10 (FIG. 3)
5, the mobile terminal 101 is not currently connected to the mobile terminal control host device 104 of FIG. 1 and steps 502, 507, 511, or 51 of the transmission process shown in FIG.
If the determination of No. 5 is NO, steps 503, 50
8, 512 or 516, the communication unit 111 of FIG.
Request the communication control unit 321 in the terminal to perform a transmission process. The transmission process executed by the communication control unit 321 in response to this request is shown by the operation flowchart in FIG.

First, in step 901, a link establishment phase is executed. In this phase, the access telephone number of the mobile terminal control host device 104 is automatically dialed up and the mobile terminal control host device 104
Is received, the link control protocol (LC
P), a maximum data length of a PPP frame (FIG. 7 (a)) used for communication, a non-transparent character to be escaped, a "protocol" field of the PPP frame (FIG. 7). (a)) Determine whether to compress the data length from 2 octets to 1 octet, and omit (compress) the "address" field (FIG. 7 (a)) having the fixed value "11111111" of the PPP frame. A negotiation regarding the determination of the presence / absence is performed with the connection establishment unit 113 (FIG. 1) in the mobile terminal control host device 104. In this case, communication between the communication control unit 321 in the communication unit 111 of the mobile terminal 101 and the connection establishment unit 113 in the mobile terminal control host device 104 is performed by using a PPP frame having the format shown in FIG. By setting a hexadecimal value “c021” specifying the LCP in the “protocol” field and setting necessary control data in the “information field”,
Be executed.

Next, in step 902, an authentication phase is executed. In this phase, PAP (Pass
word Authentication Protocol) or CHAP (Challe
The authentication of the user using the mobile terminal 101 is performed from the connection establishment unit 113 (FIG. 1) in the mobile terminal control host device 104 to the mobile terminal 101 using an authentication protocol called ngeHandshake Authentication Protocol). As a result, the Internet provider operating the mobile terminal control host device 104 becomes the mobile terminal 101
Can be determined whether or not the user who uses is a contracted user. In this case, communication between the communication control unit 321 in the communication unit 111 of the mobile terminal 101 and the connection establishment unit 113 in the mobile terminal control host device 104 is performed by using a PPP frame having the format shown in FIG. A hexadecimal value “c023” specifying PAP or a hexadecimal value “c223” specifying CHAP is set in the “protocol” field, and necessary authentication data is set in the “information field”. Will be executed.

Finally, at step 903, a network layer protocol phase is executed. In the case of the present embodiment, in this phase, a protocol called an IP control protocol (IPCP) is used, and T
Whether or not the CP header (see FIG. 8B) is compressed is determined, and one of free (unused) IP addresses that can be allocated by the mobile terminal control host device 104 is allocated to the mobile terminal 101. In addition, necessary route information is set in the communication control unit 321 (FIG. 3) in the communication unit 111 of the mobile terminal 101 and the routing unit 114 (FIG. 1) in the mobile terminal control host device 104. After this,
By using the IP address, the mobile terminal 101 can access the voice control host device 108 connected to the Internet 105 and any resource desired by the user on the Internet 105. In this case, communication between the communication control unit 321 in the communication unit 111 of the mobile terminal 101 and the connection establishment unit 113 in the mobile terminal control host device 104 is performed by using a PPP frame having the format shown in FIG. The "protocol" field is used to set a hexadecimal value "8021" for specifying the IPCP, and the "information field" is set with data for negotiating a necessary IP address, and executed.

By the above series of operations, the mobile terminal 101
Can exchange a PPP frame storing a TCP / IP packet for communication with the routing unit 114 in the mobile terminal control host device 104, and the mobile terminal 101 can freely use resources on the Internet 105. Can be accessed.

In order to enable access to the voice control host device 108 and the like even during a PHS call, the mobile terminal 101 can be configured to have, for example, a two-channel simultaneous communication function.

The communication control unit 321 (FIG. 3) in the communication unit 111 of the mobile terminal 101 communicates with the mobile terminal control host device 104 when the transmission / reception data is not detected for a predetermined time (for example, 10 minutes). May be configured to automatically disconnect the PPP link. <Detailed operation of transmission / reception processing of mobile terminal 101 related to sentence speech recognition / database search processing>
The detailed operation of the transmission / reception processing executed by the mobile terminal 101 when the start of sentence speech recognition / database search processing is instructed by operating the touch panel 1 and thereafter will be described.

The operation of the touch panel described above is detected by the touch panel control unit 315 of FIG.
In the control operation corresponding to the above-described operation flowchart of FIG. 4 executed by the CPU 316 (FIG. 3) in the CPU 10, the determination in step 401 is YES, and the determinations in steps 405 and 406 are NO.
09 is detected by executing another key input process. Further, in the transmission processing in step 404, the determination in step 501 in FIG. 5 described above is YES, and the transmission processing is executed in step 503 as necessary. Then, in step 504, the “terminal identification code” of the mobile terminal 101 is transmitted. Then, the communication control unit 321 in the communication unit 111 of FIG. 3 is requested to send a command corresponding to the key input process indicating the start instruction of the sentence speech recognition / database search process.

As a result, the communication control section 321 first generates a TCP segment having the format shown in FIG. In this case, in the TCP header having the format shown in FIGS. 7 (c) and 8 (b), the "source port number" field and the "destination port number" field contain text / speech recognition / database search processing. A 16-bit integer value specifying the communication protocol is set. In the “data” field of the TCP segment, a “terminal identification code” (for example, the PHS telephone number) for identifying the mobile terminal 101 and a command for starting a sentence voice recognition / database search process based on a user's designation are included. Is stored.

Next, the communication control unit 321 executes the above-described TCP
FIG. 7 with segments stored in the “data” field
An IP datagram having the format shown in (b) is generated. In this case, in the IP header having the format shown in FIG. 7B and FIG. 8A, the “protocol” format defines the format of the TCP segment data stored in the “data” field. Numerical value 6 is set. In the “source IP address” field, the connection establishment unit 113 in the mobile terminal control host device 104 transmits the communication unit of the mobile terminal 101 by the transmission processing already executed (see the description of step 903 in FIG. 9). The assigned IP address is set for the communication control unit 321 in 111. Further, the "destination IP address" field contains the voice control host device 1
08 is set.

Then, the communication control unit 321 transmits the IP
The datagram is stored in the "Information" field, and the IP
Hexadecimal value "00" indicating that the datagram is stored
FIG. 7A in which “21” is stored in the “protocol” field
Is generated in accordance with the path information set in the communication control unit 321 (see the description of step 903 in FIG. 9).
The P frame is transmitted to the mobile terminal control host device 104. Hereinafter, when a data unit including the above-described TCP segment, IP datagram, and PPP frame is transferred in the Internet 105, the data unit is simply referred to as a TCP / IP packet.

This TCP / IP packet is routed to the routing unit 114 in the mobile terminal control host device 104 and to the Internet 105 in the Internet 105 based on the “destination IP address” stored in the IP header of the IP datagram constituting the TCP / IP packet. In particular, by a relay host device (not shown),
After being transferred to the router device 106 in the voice service provider, the data is further transferred to the packet transmitting / receiving unit 115 in the voice control host device 108 via the LAN 107.

The packet transmitting / receiving section 115 has its own IP address set in the "destination IP address" field of the IP header of the IP datagram constituting the transferred TCP / IP packet. That the TCP / I
Receive a P packet.

The packet transmitting / receiving unit 115 stores a communication protocol for text / speech recognition / database search processing in the “destination port number” field and the “source port number” field of the TCP segment constituting the received TCP / IP packet. By confirming that a 16-bit integer value for specifying is set, the reception notification is notified to the mobile terminal communication control unit 116 (FIG. 1).

Along with this notification, the packet transmitting / receiving unit 115
Extracts the “source IP address” from the IP header of the IP datagram constituting the received TCP / IP packet, and reads “terminal identification code” from the “data” field of the TCP segment constituting the TCP / IP packet. The voice recognition / database search process start request command is extracted, and the data is transferred to the mobile terminal communication control unit 116.

As a result, a TCP / IP packet storing transmission permission data is returned from the voice control host device 108 to the mobile terminal 101 as described later. This TCP / IP packet is composed of the IP
“Destination IP” stored in the IP header of the datagram
After being transferred to the routing unit 114 in the mobile terminal control host device 104 by the router device 106 in the voice service provider and the relay host device (not shown) in the Internet 105 based on the "address", the PHS network Mobile terminal 1 via 103 (FIG. 1)
01 to the communication control unit 321 (FIG. 3) in the communication unit 111.

The communication control unit 321 in the communication unit 111 of the mobile terminal 101 stores its own mobile terminal 101 (in the “destination IP address” field of the IP header of the IP datagram constituting the transferred TCP / IP packet. By identifying that the IP address assigned to it (temporarily or dynamically) is set.
/ IP packet is received.

Then, the communication control unit 321 transmits the received T
In the “destination port number” field and the “source port number” field of the TCP segment constituting the CP / IP packet, a 16-bit integer value for specifying a communication protocol for sentence speech recognition / database search processing is set. By confirming this, the CPU 316 in the control unit 110 of the mobile terminal 101 is notified of the reception notification.

At the same time as this notification, the communication control unit 321 extracts the transmission permission data from the “data” field of the TCP segment constituting the received TCP / IP packet, and delivers it to the CPU 316.

The CPU 316 processes the above-described reception notification and transmission permission data in step 403 of FIG.
The transmission permission data is stored in the RAM 317. In the mobile terminal 101, when the user operates the touch panel to instruct the start of the sentence speech recognition / database search processing, the CPU 316 causes the above-described step 408 in FIG.
Then, the microphone control unit 303 in the input unit 109 in FIG. 3 is instructed to start a PHS call process or to start a speech input process in an off-line state for executing a sentence speech recognition / database search process. . As a result, the user has started inputting sound from the microphone 301 (201 in FIG. 2) through a call operation or a sound input operation in an offline state.

Thereafter, the CPU 316 executes steps 401 → 412 → 402 → 403 → 404 in FIG.
→ In the transmission processing of step 404 executed as one processing of the repetition loop of 401, step 50 of FIG.
The determinations at Steps 5 and 506 are YES, and the transmission processing is executed again at Step 508 as necessary.
At 09, an instruction to transmit the audio data transferred from the microphone control unit 303 in the input unit 109 shown in FIG. 3 to the RAM 317 in the control unit 110 is requested to the communication control unit 321 in the communication unit 111. Is done.

As a result, the communication control section 321 first generates a TCP segment having the format shown in FIG. In this case, in the TCP header having the format shown in FIGS. 7 (c) and 8 (b),
In the “source port number” field and the “destination port number” field, a 16-bit integer value specifying a communication protocol for sentence speech recognition / database search processing is set. Then, the “data” field of the TCP segment includes the microphone control unit 303 in the input unit 109 and the RAM 317 in the control unit 110 shown in FIG.
Is stored.

Next, the communication control unit 321 executes the above-described TCP
FIG. 7 with segments stored in the “data” field
An IP datagram having the format shown in (b) is generated. In this case, in the IP header having the format shown in FIG. 7B and FIG. 8A, the “protocol” format defines the format of the TCP segment data stored in the “data” field. Numerical value 6 is set. In the “source IP address” field, the connection establishment unit 113 in the mobile terminal control host device 104 transmits the communication unit of the mobile terminal 101 by the transmission processing already executed (see the description of step 903 in FIG. 9). The assigned IP address is set for the communication control unit 321 in 111. Further, the "destination IP address" field contains the voice control host device 1
08 is set.

Then, the communication control unit 321 transmits the IP
The datagram is stored in the "Information" field, and the IP
Hexadecimal value "00" indicating that the datagram is stored
FIG. 7A in which “21” is stored in the “protocol” field
Is generated in accordance with the path information set in the communication control unit 321 (see the description of step 903 in FIG. 9).
The P frame is transmitted to the mobile terminal control host device 104.

This TCP / IP packet is routed to the routing unit 114 in the mobile terminal control host device 104 and to the Internet 105 in the Internet 105 based on the “destination IP address” stored in the IP header of the IP datagram constituting the TCP / IP packet. In particular, by a relay host device (not shown),
After being transferred to the router device 106 in the voice service provider, the data is further transferred to the packet transmitting / receiving unit 115 in the voice control host device 108 via the LAN 107.

The packet transmitting / receiving section 115 has its own IP address set in the “destination IP address” field of the IP header of the IP datagram constituting the transferred TCP / IP packet. That the TCP / I
Receive a P packet.

The packet transmission / reception unit 115 stores a communication protocol for text / speech recognition / database search processing in the “destination port number” field and the “source port number” field of the TCP segment constituting the received TCP / IP packet. By confirming that a 16-bit integer value for specifying is set, the reception notification is notified to the mobile terminal communication control unit 116 (FIG. 1).

Along with this notification, the packet transmitting / receiving unit 115
Extracts the “source IP address” from the IP header of the IP datagram constituting the received TCP / IP packet, extracts the audio data from the “data” field of the TCP segment constituting the TCP / IP packet, and Is transferred to the mobile terminal communication control unit 116.

As a result, the mobile terminal communication control unit 116
The delivered speech data is delivered to the sentence speech recognition unit 117. The sentence speech recognition unit 117 executes sentence speech recognition processing on the delivered speech data, and delivers the recognized speech sentence data as a recognition result to the search keyword extraction unit 121. The search keyword extraction unit 121 classifies the recognized voice sentence data into indices for each mobile terminal 101, extracts a search keyword while referring to an unnecessary keyword dictionary based on the search index obtained as a result, and extracts the search keyword from the mobile terminal communication control unit. Hand over to 116. Then, the mobile terminal communication control unit 116 returns a TCP / IP packet storing the search keyword obtained from the search keyword extraction unit 121 to the mobile terminal 101 as described later.

The TCP / IP packet, based on the “destination IP address” stored in the IP header of the IP datagram constituting the TCP / IP packet, and the TCP / IP packet in the voice service provider, After being transferred by the relay host device (not shown) to the routing unit 114 in the mobile terminal control host device 104, the communication control unit 321 (in the communication unit 111 of the mobile terminal 101) via the PHS network 103 (FIG. 1). It is transferred to Fig. 3).

The communication control unit 321 in the communication unit 111 of the mobile terminal 101 stores its own mobile terminal 101 (in the “destination IP address” field of the IP header of the IP datagram constituting the transferred TCP / IP packet). By identifying that the IP address assigned to it (temporarily or dynamically) is set.
/ IP packet is received.

Then, the communication control unit 321 transmits the received T
In the “destination port number” field and the “source port number” field of the TCP segment constituting the CP / IP packet, a 16-bit integer value for specifying a communication protocol for sentence speech recognition / database search processing is set. By confirming this, the CPU 316 in the control unit 110 of the mobile terminal 101 is notified of the reception notification.

[0185] Along with this notification, the communication control unit 321 extracts the retrieval keyword (plural) from the "data" field of the TCP segment constituting the received TCP / IP packet, and delivers it to the CPU 316.

The CPU 316 processes the above-described reception notification and search keyword in the above-described step 412 in FIG.
In step 413, the search keyword is displayed on the LCD display unit 311 (203 in FIG. 2).

By operating the touch panel of the mobile terminal 101, the user selects any one or more of the displayed search keywords as a selected search keyword.

The operation of the touch panel is detected by the touch panel control section 315 of FIG.
In the control operation corresponding to the above-described operation flowchart of FIG. 4 executed by the CPU 316 (FIG. 3) within 0, the determination in step 401 is YES, and the determination in steps 405 and 406 is NO.
9 is detected by executing another key input process. Further, in the transmission processing of step 404, the determination of the above-described step 514 in FIG. 6 is YES, and the transmission processing is executed in step 516 as necessary. Then, in step 5174, the transmission instruction of the selected search keyword is The request is made to the communication control unit 321 in the communication unit 111 in FIG.

As a result, the communication control section 321 first generates a TCP segment having the format shown in FIG. In this case, in the TCP header having the format shown in FIGS. 7 (c) and 8 (b),
In the “source port number” field and the “destination port number” field, a 16-bit integer value specifying a communication protocol for sentence speech recognition / database search processing is set. The “search” keyword extracted via the touch panel control unit 315 is stored in the “data” field of the TCP segment.

Next, the communication control unit 321 executes the TCP
FIG. 7 with segments stored in the “data” field
An IP datagram having the format shown in (b) is generated. In this case, in the IP header having the format shown in FIG. 7B and FIG. 8A, the “protocol” format defines the format of the TCP segment data stored in the “data” field. Numerical value 6 is set. In the “source IP address” field, the connection establishment unit 113 in the mobile terminal control host device 104 transmits the communication unit of the mobile terminal 101 by the transmission processing already executed (see the description of step 903 in FIG. 9). The assigned IP address is set for the communication control unit 321 in 111. Further, the "destination IP address" field contains the voice control host device 1
08 is set.

Then, the communication control unit 321 transmits the IP
The datagram is stored in the "Information" field, and the IP
Hexadecimal value "00" indicating that the datagram is stored
FIG. 7A in which “21” is stored in the “protocol” field
Is generated in accordance with the path information set in the communication control unit 321 (see the description of step 903 in FIG. 9).
The P frame is transmitted to the mobile terminal control host device 104.

The TCP / IP packet is routed to the routing unit 114 in the mobile terminal control host device 104 and to the Internet 105 in the Internet 105 based on the “destination IP address” stored in the IP header of the IP datagram constituting the TCP / IP packet. In particular, by a relay host device (not shown),
After being transferred to the router device 106 in the voice service provider, the data is further transferred to the packet transmitting / receiving unit 115 in the voice control host device 108 via the LAN 107.

The packet transmitting / receiving section 115 has its own IP address set in the “destination IP address” field of the IP header of the IP datagram constituting the transferred TCP / IP packet. That the TCP / I
Receive a P packet.

The packet transmitting / receiving unit 115 stores a communication protocol for text / speech recognition / database search processing in the “destination port number” field and the “source port number” field of the TCP segment constituting the received TCP / IP packet. By confirming that a 16-bit integer value for specifying is set, the reception notification is notified to the mobile terminal communication control unit 116 (FIG. 1).

Along with this notification, the packet transmitting / receiving unit 115
Extracts the “source IP address” from the IP header of the IP datagram that constitutes the received TCP / IP packet, extracts the selected search keyword from the “data” field of the TCP segment that constitutes the TCP / IP packet, The data is transferred to the mobile terminal communication control unit 116.

As a result, the mobile terminal communication control unit 116
As will be described later, the selected search keyword is input to the search control unit 1.
Hand over to 18. The search control unit 118 sends a response to the SQL DBMS (structured query language database management system) 119 (FIG. 1) in the voice control host device 108 to the database 120 in the voice control host device 108 based on the passed selected search keyword. Request an inquiry. As a result, the search control unit 118 generates search result sentence data based on the search result returned from the SQL DBMS 119, and delivers it to the mobile terminal communication control unit 116. Then, as described later, the mobile terminal communication control unit 116 controls the TCP in which the search result sentence data obtained from the search control unit 118 is stored.
/ IP packet is returned to the mobile terminal 101.
This TCP / IP packet is routed to a router 106 in the voice service provider and a relay (not shown) in the Internet 105 based on the “destination IP address” stored in the IP header of the IP datagram constituting the TCP / IP packet. After being transferred by the host device to the routing unit 114 in the mobile terminal control host device 104, the communication control unit 321 (FIG. 3) in the communication unit 111 of the mobile terminal 101 via the PHS network 103 (FIG. 1). Transferred to

The communication control unit 321 in the communication unit 111 of the mobile terminal 101 stores its own mobile terminal 101 (in the “destination IP address” field of the IP header of the IP datagram constituting the transferred TCP / IP packet. By identifying that the IP address assigned to it (temporarily or dynamically) is set.
/ IP packet is received.

Then, the communication control section 321
In the “destination port number” field and the “source port number” field of the TCP segment constituting the CP / IP packet, a 16-bit integer value for specifying a communication protocol for sentence speech recognition / database search processing is set. By confirming this, the CPU 316 in the control unit 110 of the mobile terminal 101 is notified of the reception notification.

At the same time as this notification, the communication control unit 321 extracts the retrieval result sentence data from the “data” field of the TCP segment constituting the received TCP / IP packet, and delivers it to the CPU 316.

The CPU 316 processes the above-described reception notification and the search result sentence data in step 402 of FIG. 4 described above, and in step 410, converts the search result sentence data to the LC
It is displayed on the D display unit 311 (203 in FIG. 2).

The user operates the touch panel of the mobile terminal 101 to request the voice control host device 108 to end the sentence speech recognition / database search processing, thereby requesting the end of the sentence speech recognition / database search processing. Can be indicated.

In this case, the operation of the touch panel described above is detected by the touch panel control section 315 of FIG. 3, and then executed by the CPU 316 (FIG. 3) of the control section 110. In the control operation corresponding to the flowchart, the determination is made by YES in Step 401 and NO in Steps 405 and 406, and the other key input processing of Step 409 is executed. Further, in the transmission processing of step 404, the determination of step 501 in FIG.
In step 503, the mobile terminal 10 becomes an ES, and if necessary, a calling process is executed in step 503.
The transmission control unit 321 in the communication unit 111 of FIG. 3 is requested to transmit the “terminal identification code” of No. 1 and the above-described sentence speech recognition / database search processing end request command.

As a result, the communication control unit 321 first
A TCP segment having a format shown in FIG. 7C in which a “terminal identification code” for specifying the mobile terminal 101 and a command for requesting termination of sentence speech recognition / database search processing in a “data” field are generated, next,
An IP datagram having the format shown in FIG. 7B in which the TCP segment is stored in the "data" field is generated, and the IP datagram is stored in the "information" field in FIG. 7A. A PPP frame having the format shown in (1) is generated, and a TCP / IP packet including the PPP frame is transmitted.
In this case, the TCP header (FIG. 7 (c), FIG. 8 (b)), I
P header (Fig. 7 (b), Fig. 8 (a)) and "protocol"
Each piece of information set in the field (FIG. 7A) is the same as each piece of information set when the above-described sentence speech recognition / database search processing start request command is transmitted.

As a result, the above-described TCP / IP packet is transmitted via the Internet 105 in exactly the same manner as the TCP / IP packet storing the above-described sentence speech recognition / database search processing start request command and the like. The packet is transferred to the packet transmission / reception unit 115 in the control host device 108.

The packet transmitting / receiving unit 115 transmits the transferred TCP / IP in exactly the same manner as when the TCP / IP packet storing the above-described sentence speech recognition / database search processing start request command and the like is transferred. It receives the packet and notifies the mobile terminal communication control unit 116 (FIG. 1) of a reception notification.

With this notification, the packet transmitting / receiving unit 115
Extracts the “terminal identification code” and the command to end the sentence / speech recognition / database search process from the “data” field of the TCP segment constituting the received TCP / IP packet, and extracts the data from the mobile terminal communication control unit. Hand over to 116.

As a result, the mobile terminal communication control unit 116
As described later, the sentence speech recognition / database search processing for the mobile terminal 101 ends. <Schematic Operation of Mobile Terminal Communication Control Unit 116, Sentence / Speech Recognition Unit 117, and Search Control Unit 118> Next, the mobile terminal communication control unit 116, sentence / speech recognition unit 117, and search keyword extraction unit in the sound control host device 108 The general operations of the search control unit 121 and the search control unit 118 will be described.

The mobile terminal communication control unit 116 transmits the “terminal identification code” assigned to the mobile terminal 101 that has transmitted the command to start the sentence speech recognition / database search process.
For each (stored in the TCP segment to which the above command has been transferred), an entry is registered in the processing terminal registration table having the data structure shown in FIG.
Various types of buffer files to be described later are stored in the audio control host device 1.
08 is created on the file system managed. Also,
When the registration of the entry and the file is successful, the mobile terminal communication control unit 116
"Source IP address" stored in the datagram
The transmission permission data is returned to the mobile terminal 101.

[0209] The mobile terminal communication control unit 116 converts the voice data received from the mobile terminal 101 thereafter into a processing terminal corresponding to the "source IP address" (stored in the IP datagram to which the data was transferred). Write additionally to the audio buffer file specified from the entry in the registration table.

For each entry in the processing terminal registration table shown in FIG. 15, the sentence / speech recognition unit 117 performs a sentence / speech recognition process on the speech data if the speech data is received in the speech buffer file specified from each entry. Then, the recognition voice text data obtained as a result is additionally written to the text buffer file corresponding to each of the above entries.

The search keyword extraction section 121 (FIG. 1)
For each entry in the processing terminal registration table shown in FIG. 15, if the recognized voice sentence data is obtained in the sentence buffer file specified from each entry, a search keyword extraction process is executed on the data, and the result is obtained. The search keyword is additionally written to the search keyword buffer file corresponding to each entry.

[0212] For each entry in the processing terminal registration table, the mobile terminal communication control unit 116, if a search keyword is obtained in the search keyword buffer file specified from each entry, stores the search keyword in each entry. A reply is sent to the mobile terminal 101 having the “source IP address”.

Subsequently, mobile terminal communication control section 116 sets the selected search keyword received from mobile terminal 101 thereafter in its “source IP address” (stored in the IP datagram to which it was transferred). Additional writing is performed on the selected search keyword buffer file specified from the corresponding entry of the processing terminal registration table.

The search control unit 118 (FIG. 1) performs, for each entry in the processing terminal registration table shown in FIG. 15, if the selected search keyword is received in the selected search keyword buffer file specified from each entry, Then, a database search process is executed, and the search result sentence data obtained as a result is additionally written into a search result buffer file corresponding to each entry.

[0215] For each entry in the processing terminal registration table, the mobile terminal communication control unit 116 registers the search result text data in the search result buffer file specified in each entry, if the data is obtained in each entry. A reply is sent to the mobile terminal 101 having the “source IP address”.

The mobile terminal communication control unit 116 determines whether the entry in the processing terminal registration table that has received the end request command for the sentence speech recognition / database search processing or the last access time is earlier than the time that is a fixed time before the current time. For the entry in the processing terminal registration table that is the time, the contents of the entry are deleted, and each buffer file specified from that entry is deleted. <Detailed Operation of Mobile Terminal Communication Control Unit 116> FIGS.
4 is an operation flowchart showing a control operation performed by the mobile terminal communication control unit 116 to realize the above function. This operation flowchart is realized as an operation in which a processor (not shown) that controls the mobile terminal communication control unit 116 executes a control program (not shown).

First, in step 1001, it is determined whether or not a reception notification has been sent from the packet transmitting / receiving unit 115 (FIG. 1) in the voice control host device. As described above, the packet transmitting / receiving unit 115 communicates with the Internet 105
I that constitutes the TCP / IP packet transferred from
In the "destination IP address" field of the IP header of the P datagram, the IP address of the voice control host
By identifying that the address is set,
A 16-bit that receives the TCP / IP packet and specifies a communication protocol for sentence speech recognition / database search processing in the "destination port number" field and the "source port number" field of the TCP segment constituting the TCP / IP packet By confirming that an integer value is set, the mobile terminal communication control unit 116 is notified of the reception notification. The reception notification includes a sentence speech recognition / database search process start request command, speech data targeted for sentence speech recognition / database search process, a selected search keyword targeted for database search, or sentence speech recognition / database search process.
This is a reception notification regarding any one of the end request commands for the database search process.

When the reception notification is notified from the packet transmission / reception unit 115 and the determination in step 1001 is YES, in step 1002, the data transferred together with the reception notification from the packet transmission / reception unit 115 is fetched. In this case, if the reception notification is a reception notification of a sentence speech recognition / database search process start request command, the “source IP address”, the “terminal identification code”, and the above command are fetched. When the reception notification is a reception notification of audio data, the “source IP address” and the audio data are captured. When the reception notification is a reception notification of the selected search keyword, the “source IP address” and the selected search keyword are fetched. Furthermore,
If the reception notification is a reception notification of a command for requesting termination of sentence speech recognition / database search processing, the “terminal identification code” and the command are fetched.

After the processing in step 1002, the judgments in step 1003 in FIG. 10, steps 1007 and 1009 in FIG. 11, or step 1011 in FIG. 12 are sequentially examined, and any judgment result becomes YES. That is, if the data transferred from the packet transmitting / receiving unit 115 in step 1002 is related to a command requesting start of sentence speech recognition / database search processing, step 10
03 is YES and steps 1004 to 100
6 is executed, if it is related to voice data, the determination in step 1007 in FIG. 11 is YES and step 1008 is executed. If it is related to a selected search keyword, the determination in step 1009 in FIG.
S, step 1010 is executed, and the sentence speech recognition /
If the command is related to the end request command of the database search process, the determination in step 1011 of FIG.
In S, steps 1012 and 1013 are executed.

If the packet transmission / reception unit 115 has not received the reception notification and the determination in step 1001 is NO,
Alternatively, after the processing corresponding to the reception of each command, voice data, or selected search keyword described above, the transmission processing of the search keyword is performed in steps 1014 and 1015 in FIG. 13, and the search result text is processed in steps 1016 and 1017 in FIG. The data transmission process is executed, and the subsequent step 10
In steps 18 and 1019, after the processing for terminating communication with the mobile terminal 101 whose last access time is a fixed time or more is performed, the flow returns to the determination processing in step 1001 of FIG. 10 again.

When the determination in step 1001 is YES,
In the case where the data transferred from the packet transmitting / receiving unit 115 in step 1002 is related to a command for requesting the start of sentence speech recognition / database search processing, the determination in step 1003 is YES and the processing in steps 1004 to 1005 is executed. explain.

First, in step 1004, an audio buffer file which is a buffer file for receiving audio data, a text buffer file which is a buffer file for temporarily storing recognized voice text, and a search control unit 118
A search index buffer file and a searched keyword buffer file, a search keyword buffer file that is a buffer file for transmitting search keywords, and a selected search keyword buffer file that is a buffer file for receiving selected search keywords.
The search result buffer file, which is a buffer file for transmitting the search result sentence data, is the voice control host device 1
08 is created on the file system managed.

Next, at step 1005, one entry (data of one horizontal row) is stored in the processing terminal registration table having the data structure shown in FIG. Pairs) are secured. Then, in the entry, "terminal identification code"
, “Source IP address”, last access time,
Voice buffer file name, text buffer file name, search index buffer file name, searched keyword buffer file name, search keyword buffer file name, selected search keyword buffer file name, and search result buffer file name are registered. Is done.
The “terminal identification code” is data transferred from the packet transmitting / receiving unit 115 in step 1002, and is stored in the “data” field of a TCP segment constituting a TCP / IP packet transferred from the mobile terminal 101. (See FIG. 7 (c)). The “source IP address” is also the data passed from the packet transmitting / receiving unit 115 in step 1002, and was stored in the IP header of the IP datagram constituting the TCP / IP packet transferred from the mobile terminal 101. (See FIGS. 7 (b) and 8 (a)). The current time is set as the last access time. Each buffer file name is a file name indicating each file created in step 1004.

After the processing of step 1005, step 1
In step 006, in step 1002, the packet
5, the transmission permission data is returned to the "source IP address" registered in the above entry of the processing terminal registration table.

Specifically, mobile terminal communication control section 116
Requests the packet transmission / reception unit 115 (FIG. 1) to return transmission permission data to the “source IP address”. As a result, the packet transmitting / receiving unit 115 first
Generate a TCP segment having the format shown in (c). In this case, in the TCP header having the format shown in FIGS. 7 (c) and 8 (b), the "source port number" field and the "destination port number" field contain text / speech recognition / database search processing. A 16-bit integer value specifying the communication protocol is set. Then, transmission permission data is stored in the “data” field of the TCP segment.

Next, the packet transmitting / receiving unit 115 generates an IP datagram having the format shown in FIG. 7B in which the above-mentioned TCP segment is stored in the “data” field. In this case, in the IP header having the format shown in FIGS. 7 (b) and 8 (a),
In the “protocol” format, an integer value 6 defining the format of the TCP segment data stored in the “data” field is set. The “source IP address” field contains the voice control host device 1
08 is set. Further, in the “destination IP address” field, the “source IP address” passed from the packet transmission / reception unit 115 in step 1002 of FIG. 10 is set.

Then, the packet transmitting / receiving unit 115 generates a frame according to the protocol on the LAN 107 in which the above-described IP datagram is stored, and transmits the frame to the LAN 107
To send to. For example, if the LAN 107 is a local area network based on the Ethernet system, the frame is an Ethernet frame.

The above frame, IP datagram and TCP
A TCP / IP packet composed of a segment and
After being transferred to the mobile terminal control host device 104 via the router device 106 and the Internet 105 based on the “destination IP address” stored in the IP header of the IP datagram constituting the PHS network,
03 and the communication control unit 32 in the communication unit 111 of the mobile terminal 101 via the wireless base (or wired connection device) 102
1 (FIG. 3).

Thereafter, voice data is transferred from the mobile terminal 101 to the voice control host device 108 as described above. After the processing of step 1006, FIG.
In steps 1014 and 1015 of step 3, transmission processing of a search keyword is performed, and in steps 1016 and 1017 of FIG. 14, transmission processing of search result text data is executed. In steps 1018 and 1019, the last access time is longer than a predetermined time. Processing for terminating communication with a certain mobile terminal 101 is performed, and the flow returns to the determination processing of step 1001 in FIG. 10 again.

Next, the determination at step 1001 in FIG. 10 is YES, and the
A description will be given of the processing of step 1008, which is executed when the determination at step 1007 in FIG. 11 is YES when the data delivered from 15 is audio data.

That is, in step 1008, an entry of the processing terminal registration table (FIG. 15) in which the same “source IP address” passed from the packet transmitting / receiving section 115 in step 1002 of FIG. 10 is stored is searched. The audio data transferred from the packet transmission / reception unit 115 in step 1002 in FIG. 10 is additionally written to the audio buffer file (see step 1004 in FIG. 10) corresponding to the audio buffer file name stored in the corresponding entry. . The size of the audio buffer file at the time of additional writing is automatically adjusted by the file system managed by the audio control host device 108.

At step 1008, the last access time stored in the corresponding entry is updated to the current time. In this way, the mobile terminal communication control unit 116 sends the sentence voice recognition unit 11 via the voice buffer file for each mobile terminal 101 (for each “terminal identification code”).
7 (FIG. 1) is delivered. As will be described later, the sentence speech recognition unit 117 performs, for each entry in the processing terminal registration table shown in FIG. 15, if speech data is received in a speech buffer file specified from each entry, the sentence speech recognition unit 117 The process is executed, and the resulting recognized voice text data is additionally written to the text buffer file corresponding to each of the entries. Further, as described later, the search keyword extraction unit 121 (FIG. 1) obtains the recognized voice text data in a text buffer file specified from each entry in the processing terminal registration table shown in FIG. Then, a search keyword extraction process is executed for the search keyword, and the search keyword obtained as a result is additionally written to the search keyword buffer file corresponding to each entry.

After the processing in step 1008, transmission processing of a search keyword is executed in steps 1014 and 1015 in FIG. 13, and transmission processing of search result text data is executed in steps 1016 and 1017 in FIG. At 1019, a process for terminating communication with the mobile terminal 101 whose last access time is a predetermined time or more is performed, and the process returns to the determination process of step 1001 in FIG. 10 again.

Next, the determination at step 1001 in FIG. 10 is YES, and the
In the case where the data passed from No. 15 is the selected search keyword, the determination in step 1009 in FIG.
The process of step 1010 executed as an ES will be described.

That is, in step 1010, an entry of the processing terminal registration table (FIG. 15) in which the same “source IP address” passed from the packet transmitting / receiving unit 115 in step 1002 of FIG. 10 is stored is searched. , A selected search keyword buffer file corresponding to the selected search keyword buffer file name stored in the corresponding entry (see step 1004 in FIG. 10)
In step 1002 of FIG.
The selected search keyword passed from 5 is additionally written. The size of the selected search keyword buffer file at the time of additional writing is automatically adjusted by the file system managed by the voice control host device 108.

At step 1010, the last access time stored in the relevant entry is updated to the current time. In this way, the selected search keyword is delivered from the mobile terminal communication control unit 116 to the search control unit 118 (FIG. 1) via the selected search keyword buffer file for each mobile terminal 101 (for each “terminal identification code”). As described later, the search control unit 118
If the selected search keyword is obtained in the selected search keyword buffer file specified from each entry for each entry of the processing terminal registration table shown in, a database search process is executed on the selected search keyword, and the search result sentence obtained as a result is obtained. The data is additionally written to the search result buffer file corresponding to each of the entries.

After the processing of step 1010, the transmission processing of the search keyword is executed in steps 1014 and 1015 of FIG. 13, the transmission processing of the search result text data is executed in steps 1016 and 1017 of FIG. At 1019, a process for terminating communication with the mobile terminal 101 whose last access time is a predetermined time or more is performed, and the process returns to the determination process of step 1001 in FIG. 10 again.

Next, in the case where the determination in step 1001 in FIG. 10 is YES, and the data passed from the packet transmitting / receiving unit 115 in step 1002 is related to a command for requesting termination of sentence speech recognition / database search processing, FIG. The determination in step 1011 of 11 is YES
The processing of steps 1012 and 1013 executed as follows will be described.

First, at step 1012, all the contents of the entries of the processing terminal registration table (FIG. 15) storing the same “terminal identification code” transferred from the packet transmitting / receiving unit 115 at step 1002 of FIG. Deleted.

Next, at step 1013, the audio buffer file name, the sentence buffer file name, the searched keyword buffer file name,
Each buffer file corresponding to the search index buffer file name, the search keyword buffer file name, the selected search keyword buffer file name, and the search result buffer file name is deleted from the file system managed by the voice control host device.

After the processing of step 1013, the transmission processing of the search keyword is executed in steps 1014 and 1015 of FIG. 13, the transmission processing of the search result text data is executed in steps 1016 and 1017 of FIG. At 1019, a process for terminating communication with the mobile terminal 101 whose last access time is a predetermined time or more is performed, and the process returns to the determination process of step 1001 in FIG. 10 again.

The reception notification is not notified from packet transmitting / receiving section 115, and the determination in step 1001 in FIG. 10 is NO.
Or the processing of steps 1014 and 10115 of FIG. 13 and the processing of steps 1016 and 1017 of FIG. 14, and the processing of steps 1016 and 1017 of FIG. The processing of the following steps 1018 and 1019 will be described.

In these processes, the transmission process of the search keyword obtained from the search keyword extraction unit 121 and the transmission process of the search result sentence data obtained from the search control unit 118 are executed.

First, in step 1014 of FIG. 13, it is determined whether or not there is an entry in which a search keyword exists in the search keyword buffer file corresponding to the search keyword buffer file name in the processing terminal registration table (FIG. 15). . If there is no such entry and the determination in step 1014 is NO, step 1015
In FIG. 14, the processing for transmitting the search keyword is not executed.
The process proceeds to step 1016.

If there is one or more entries as described above and the determination in step 1014 is YES, step 1
At 015, for each corresponding entry, the search keyword in the search keyword buffer file corresponding to the search keyword buffer file name stored in the entry is directed to the “source IP address” stored in that entry. The sent search keyword is deleted from the search keyword buffer file. Note that the size of the search keyword buffer file at the time of deletion is automatically adjusted by the file system managed by the voice control host device 108.

After the processing in step 1015 or when the determination in step 1014 is NO, step 1016 in FIG. 14 is executed. In step 1016,
In the processing terminal registration table (FIG. 15), it is determined whether or not there is an entry in which the search result text data exists in the search result buffer file corresponding to the search result buffer file name.

There is no such entry and step 101
If the determination in step 6 is NO, the transmission processing of the search result sentence data in step 1017 is not executed, and step 1
The process proceeds to 018 and 1019.

If there is one or more entries as described above and the determination in step 1016 is YES, step 1
In step 017, for each corresponding entry, the search result sentence in the search result buffer file corresponding to the search result buffer file name stored in the entry is directed to the “source IP address” stored in the entry. The data is transmitted, and the transmitted search result sentence data is deleted from the search result buffer file. The size of the search result buffer file at the time of deletion is automatically adjusted by the file system managed by the voice control host device 108.

After the processing in step 1017 or when the determination in step 1016 is NO, step 1
018 is executed. Here, among the entries in the processing terminal registration table (FIG. 15), an entry whose last access time is a time that is earlier than the current time by a predetermined time is detected, and the contents of the entry are all deleted.

In step 1019, the audio buffer file name, the sentence buffer file name, the searched keyword buffer file name,
Each buffer file corresponding to the search index buffer file name, the search keyword buffer file name, the selected search keyword buffer file name, and the search result buffer file name is deleted from the file system managed by the voice control host device.

After the processing in step 1019, the processing in FIG.
The process returns to the determination processing of step 1001. <Detailed Operation of Sentence Speech Recognition Unit 117> FIG. 16 is a functional block diagram of the sentence speech recognition unit 117.

As described above, this sentence speech recognition unit 117 performs, for each entry in the processing terminal registration table shown in FIG. 15, if speech data is received in a speech buffer file specified from each entry. Then, sentence speech recognition is executed, and the resulting recognized speech sentence data is additionally written into a sentence buffer file corresponding to each of the above entries.

The reading of the voice data from the voice buffer file for each entry and the writing of the recognized voice text data to the text buffer file are controlled by the input / output control unit 1609 in FIG. First, the input / output control unit 1609
Will be described. FIG. 17 shows the input / output control unit 1
609 is an operation flowchart illustrating a control operation executed by the control unit 609. This operation flowchart is based on the input / output control unit 160
9 is realized as an operation of executing a control program (not shown).

First, in step 1701, it is determined whether or not an entry in which audio data is stored in the audio buffer file corresponding to the audio buffer file name exists in the processing terminal registration table (FIG. 15).

If such an entry exists and step 17
If the determination of 01 is YES, in step 1702, for each corresponding entry, the “terminal identification code” stored in the entry and the audio buffer file name corresponding to the audio buffer file name stored in the entry Of the input buffer queue 160 in FIG.
1 and the audio data is deleted from the audio buffer file.

The input buffer queue 1601 stores the audio data queued by the input buffer queue 1601 in the audio section detector 16.
02. Voice section detector 16
02 and later, a speech analysis unit 1603, a phoneme recognition unit 1604, a word recognition unit 1606, and a sentence recognition unit 1
Reference numeral 607 forms a data processing pipeline, and has a function of processing input data independently of each other. Each of the parts 1602 to 1607 can recognize the “terminal identification code” (input from the input buffer queue 1601) corresponding to the audio data currently being processed. Therefore, finally, a set of the “terminal identification code” and the recognized voice text data is output from the text recognition unit 1607 to the output buffer queue 1608.

After the processing in step 1702 or when the determination in step 1701 is NO, step 1703
Then, it is determined whether or not a set of “terminal identification code” and recognized speech text data has been obtained in the output buffer queue 1608 in FIG.

If such a set has been obtained, step 17
If the determination in step 03 is YES, in step 1704, for each set in the output buffer queue 1608, for the entry in the processing terminal registration table corresponding to the "terminal identification code" of that set, the text buffer stored in that entry The set of recognized speech text data in the output buffer queue 1608 is additionally written to the text buffer file corresponding to the file name.

After the processing in step 1704 or when the determination in step 1703 is NO, step 170 is executed again.
1 is performed. As described above, the sentence / speech recognition unit 117 can efficiently execute the sentence / speech recognition process on the speech data requested by the plurality of mobile terminals 101 in a streamlined manner.

Next, 1 for realizing the sentence speech recognition process is described.
The function of each part of 602 to 1607 will be described below. Each of the methods described below can be realized by referring to, for example, the document “Electronic / Information Engineering Introduction Series 2, Sound and Speech Engineering” (Furui, Modern Science Co., Chapter 14).

[0261] Voice section detection section 1602 detects a section in which voice exists, in a sample time series of voice data input from input buffer queue 1601. More specifically, the voice section detection unit 1602 calculates an average power (power) for each predetermined sample (for example, 32 to 256 samples for 8 kHz sampling data),
A section in which the state in which the average power exceeds a predetermined threshold continues for a predetermined number of times or more is detected as a voice section. Thereby, it is possible to prevent a sentence voice from being erroneously recognized in a section where no voice exists.

[0262] The voice analysis section 1603 includes the voice section detection section 1
A feature amount parameter vector is detected by performing a feature analysis on the audio data output from 602. Any of the following well-known analysis methods can be adopted as the voice analysis method. (1) A method of smoothing each output of a band filter bank to which a time series of audio data is input, and using each smoothed output as an element of a feature parameter vector. (2) Smoothing each short-time spectrum component calculated by Fast Fourier Transform (FFT) which receives an audio data time series of a predetermined number of continuous samples as input, and converts each smoothed component value into a feature amount parameter vector. Element method. (3) A method in which a cepstrum coefficient group is calculated by cepstrum analysis using a time series of audio data for each successive predetermined sample as an input, and these are used as elements of a feature parameter vector. (4) A method of calculating a Δ (delta) cepstrum (differential coefficient of a cepstrum) group for the cepstrum coefficient group in addition to the cepstrum coefficient group of the above (3), and adding them to the element of the feature amount parameter vector. (5) LPC analysis is performed by linear prediction analysis (LPC analysis, more specifically, line spectrum pair analysis: LSP analysis) that receives a time series of audio data for each successive predetermined sample as input.
(LSP) A method of calculating coefficient groups and using them as elements of a feature parameter vector. (6) An autocorrelation function is calculated by an autocorrelation analysis using a time series of voice data of each successive predetermined sample as an input, and a pitch fundamental frequency pattern of voice detected based on the autocorrelation function is calculated as one element of a feature parameter vector. Method to add to Next, the phoneme recognition unit 1604 stores the feature parameter vector output from the speech analysis unit 1603 at every predetermined frame period (predetermined sample) and the phoneme standard pattern dictionary 160
5. The similarity (distance) of the feature parameter vector of each phoneme stored in No. 5 with the standard pattern is calculated, and as a result, a set of phonemes with high similarity obtained at every predetermined frame period is determined along with the phonetic lattice along with the similarity. Output as data.
The phoneme recognizing unit 1604 does not determine the final phoneme at every predetermined frame period in order to avoid occurrence of a phoneme recognition error. Instead, the phoneme recognizer 1604 stores the result data in the form of phoneme lattice data that lists phoneme candidates. Is output.

The word recognizing unit 1606 receives the phoneme lattice data output from the phoneme recognizing unit 1604 at every predetermined frame period, and outputs word lattice data by listing word candidates at every predetermined frame period. As the word recognition method, any of the following well-known analysis methods can be adopted. (1) The word recognition unit 1606 performs time normalization on the time series of phoneme lattice data over a plurality of frame periods output from the phoneme recognition unit 1604 and the entire phoneme standard pattern sequence stored in the word dictionary ( DP matching or
DTW (Dynamic Time Warping) is executed to output word lattice data. Also in this case, the word recognition unit 1606
Outputs the result data in the form of word lattice data in which word candidates are tabulated without determining a final word at every predetermined frame period in order to avoid utterance of a word recognition error. (2) The word recognition unit 1606 uses the HMM (Hidden Markov)
Model), all the words are modeled, and the phoneme recognition unit 1
A time series of phoneme lattice data spanning a plurality of frame periods output from 604 is input to the HMM analysis unit, and each word corresponding to a plurality of models from a large occurrence probability is set as word lattice data as a word candidate. Output.

Finally, the sentence recognizing unit 1607 uses the first
As a rounding process, word lattice data output from the word recognition unit 1606 is sequentially input, and Japanese (English may be used).
According to the intra-phrase grammar (word order rule) for the phrase structure of
The possibility of various phrases is calculated as phrase lattice data. Then, the sentence recognizing unit 1607 analyzes the semantic dependency between the sentences according to the inter-segment grammar, determines the recognized speech sentence data, and sequentially transmits the data from the input buffer queue 1601 as the second stage processing. Output buffer queue 1608 for the pair with the "terminal identification code"
Write to. <Detailed Operation of Search Keyword Extraction Unit 121> FIG.
FIG. 4 is a functional block diagram of a search keyword extraction unit 121.

As described above, the search keyword extracting unit 121, for each entry in the processing terminal registration table shown in FIG. 12, stores the sentence buffer data specified by Is obtained, a search keyword extraction process is executed for the search keyword, and the search keyword obtained as a result is additionally written to the search keyword buffer file corresponding to each entry.

The reading of the recognized voice sentence data from the sentence buffer file for each entry and the writing of the search keyword to the search keyword buffer file are described in FIG.
Is controlled by an input / output control unit 1806. First, the control operation of the input / output control unit 1806 will be described. FIG.
9 is an operation flowchart illustrating a control operation executed by an input / output control unit 1806. This operation flowchart is realized as an operation in which a processor (not shown) for controlling the input / output control unit 1806 executes a control program (not shown), and the input / output control unit 117 shown in FIG. A control operation similar to that of the output control unit 1609 is realized.

First, in step 1901, it is determined whether or not an entry storing recognized speech text data exists in the text buffer file corresponding to the text buffer file name in the processing terminal registration table (FIG. 15). .

If such an entry exists and step 19
If the determination of 01 is YES, in step 1902, for each corresponding entry, the “terminal identification code” stored in the entry and the text buffer file name corresponding to the text buffer file name stored in the entry Is written to the input buffer queue 1801 in FIG. 18, and the recognized voice sentence data is deleted from the text buffer file.

The input buffer queue 1801 has a function of sequentially pouring the recognized voice sentence data queued therein into the search index creation unit 1802. The search index creation unit 1802 and the search keyword extraction unit 1803 form a data processing pipeline similarly to the configuration of the sentence speech recognition unit 117 shown in FIG. It has the function of processing. Further, each of the portions 1802 and 1803 is a “terminal identification code” (input from the input buffer queue 1801) corresponding to the currently recognized speech text data.
Can be recognized. Therefore, finally, a set of the “terminal identification code” and the search keyword is output from the search keyword extraction unit 1803 to the output buffer queue 1805.

After the processing in step 1902 or when the determination in step 1901 is NO, step 1903
Then, it is determined whether a set of “terminal identification code” and a search keyword has been obtained in the output buffer queue 1805 in FIG.

Since such a set has been obtained, step 19
If the determination in step 03 is YES, in step 1904, for each set in the output buffer queue 1805, for the entry in the processing terminal registration table corresponding to the "terminal identification code" of that set, the search keyword stored in that entry A set of search keywords in the output buffer queue 1805 is additionally written to the search keyword buffer file corresponding to the buffer file name.

After the processing in step 1904 or when the determination in step 1903 is NO, step 190 is executed again.
1 is performed. As described above, similarly to the case of the sentence speech recognition unit 117, the search keyword extraction unit 121 is efficiently obtained in the work flow and is obtained by the sentence speech recognition unit 117 based on the requests from the plurality of mobile terminals 101. A search keyword extraction process can be performed on the recognized speech sentence data.

Next, the function of each of the parts 1802 to 1804 for realizing the search keyword extraction processing will be described below. For each set of the “terminal identification code” and the recognized speech text data that are sequentially input from the input buffer queue 1801, the search index creation unit 1802 creates an entry in the processing terminal registration table corresponding to the “terminal identification code” of the set. The search index buffer file obtained from the search index buffer file name stored in is used to create a search index, which is a list in which words constituting the recognized speech text data are classified for each mobile terminal 101 according to a predetermined criterion, It is output to the search keyword extracting unit 1803 together with the above-mentioned “terminal identification code”. Specifically, for example, the search index creation unit 1802 counts the number of appearances of each of the words constituting the set of recognized speech text data, thereby obtaining a search index as a word table listed in descending order of the number of appearances. Create In this case, in the search index buffer file, search indexes after the command to start text / speech recognition / database search processing from one mobile terminal 101 are stored, and the search index and the recognition input this time are stored. A new search index is created based on each word constituting the voice sentence data, and is stored in the search index buffer file. Therefore, words appearing in the voice input from one mobile terminal 101 after the designation of the command are listed on the search index according to a certain standard, that is, for example, in descending order of the number of appearances. The recognized voice sentence data particularly includes word delimiter information (not shown). The word delimiter information is stored in the sentence buffer file and the input buffer queue 18.
01, the sentence recognition unit 14 in the sentence speech recognition unit 117
07, it is easy to identify the delimiter of each word on the recognized voice sentence data.

Next, the search keyword extraction unit 1803
For each set of the “terminal identification code” and the search index output from the search index creation unit 1802, a word that satisfies a predetermined criterion in the search index of the set, for example, the number of occurrences of which is equal to or more than a certain number (or a certain number) Extract words whose rank is equal to or higher than the rank of appearance frequency. Further, the search keyword extraction unit 1803 determines, among the extracted words,
A word that is not registered in the unnecessary keyword dictionary 1804 is extracted, and a search stored in an entry of the processing terminal registration table corresponding to the “terminal identification code” output from the search index creation unit 1802 together with the search index. Then, a word other than the searched keyword registered in the searched keyword buffer file obtained from the completed keyword buffer file name is extracted, and the extracted word is output to the output buffer queue 1805 together with the above-mentioned “terminal identification code” as a search keyword. The search keyword extraction unit 1803 registers the search keyword in the searched keyword buffer file.

In the unnecessary keyword dictionary 1804, words such as ordinary verbs, adjectives, adverbs, auxiliary verbs, particles, conjunctions, and prepositions are registered. By referring to this dictionary, it is possible to prevent the meaningless words from being searched in the database, and it is possible to provide only significant search keywords to the mobile terminal 101.

In the searched keyword buffer file, search keywords that have been subjected to database search processing after one mobile terminal 101 has designated a command to start sentence speech recognition / database search processing have been registered. By referring to this file, it is possible to prevent the same search keyword from being subjected to database search processing redundantly. <Detailed Operation of Search Control Unit 118> FIG. 20 is a functional block diagram of the search control unit 118.

As described above, the search control unit 118 determines, for each entry of the processing terminal registration table shown in FIG. 15, if the selected search keyword is received in the selected search keyword buffer file specified from each entry. In response to this, a database search process is executed, and the search result sentence data obtained as a result is additionally written to the search result buffer file corresponding to each of the entries.

The reading of the selected search keyword from the selected search keyword buffer file for each entry and the writing of the search result sentence data to the search result buffer file are controlled by the input / output control unit 2004 in FIG. First,
The control operation of the input / output control unit 2004 will be described.
FIG. 21 is an operation flowchart illustrating a control operation performed by the input / output control unit 2004. This operation flowchart is realized as an operation in which a processor (not shown) for controlling the input / output control unit 2004 executes a control program (not shown).
A control operation similar to that of the input / output control unit 1609 shown in FIG.

First, in step 2101, it is determined whether or not an entry in which the selected search keyword is stored in the selected search keyword buffer file corresponding to the selected search keyword buffer file name exists in the processing terminal registration table (FIG. 15). Is determined.

If such an entry exists and step 21
If the determination of 01 is YES, in step 2102, for each applicable entry, the "terminal identification code" stored in the entry and the selected search corresponding to the selected search keyword buffer file name stored in the entry The selected search keyword in the keyword buffer file is written into the input buffer queue 2001 in FIG. 20, and the selected search keyword is deleted from the selected search keyword buffer file.

The input buffer queue 2001 has a function of sequentially flowing the selected search keywords queued therein to the search execution unit 2002. Search execution unit 20
No. 02 can recognize the “terminal identification code” (input from the input buffer queue 2001) corresponding to the recognized speech text data currently being processed. Therefore, from the search execution unit 2002 to the output buffer queue 2003,
A set of “terminal identification code” and search result text data is output.

After the processing in step 2102 or when the determination in step 2101 is NO, step 2103
Then, it is determined whether a set of “terminal identification code” and search result text data is obtained in the output buffer queue 2003 of FIG.

When such a set is obtained, step 21 is executed.
If the determination in step 03 is YES, in step 2104, for each set in the output buffer queue 2003, for the entry in the processing terminal registration table corresponding to the "terminal identification code" of that set, the search result stored in that entry A set of search result text data in the output buffer queue 2003 is additionally written to the search result buffer file corresponding to the buffer file name.

After the processing in step 2104 or when the determination in step 2103 is NO, step 210 is executed again.
1 is performed. As described above, similarly to the case of the sentence speech recognition unit 117, the search control unit 118 can efficiently and efficiently execute the database search process for the selected search keywords received from the plurality of mobile terminals 101 in a workflow. it can.

Next, the function of the search execution unit 2002 for realizing the database search processing will be described below. The search execution unit 2002
For each set of the “terminal identification code” and the selected search keyword output from No. 1, SQL query data including the selected search keyword of the set is generated, and
L to the DBMS 119 (FIG. 1). What is SQL?
Abbreviation for Structured Query Language, JIS (Japanese Industrial Standard:
Japanese Industrial Standards, ANSI (American National Stand)
ards Institute: American National Standards Institute, and ISO (Intern
This is a database processing language standardized by the International Organization for Standardization).

[0286] The SQL DBMS 119 searches the database 120 based on the SQL query data.
Information related to the selected search keyword included therein is extracted as a search result, and the result is returned to the search execution unit 2002.

[0287] The search execution unit 2002 creates search result sentence data from the returned search results and writes it in the output buffer queue 2003 together with the above-mentioned "terminal identification code". <Other Embodiments> In the embodiment described above, mobile terminal 101 is a PHS terminal, and mobile terminal 101 and voice control host device 108 are connected via PHS network 103 and Internet 105. I have. However, the present invention is not limited to this, and the present invention may be applied to any form connected to the voice control host device 108 indirectly or directly by wireless or wired. Can be applied.

Further, in the present embodiment, a system including an SQL DBMS and a database accessed by the SQL DBMS is adopted as the database system, but any database system may be used. Further, these database systems do not necessarily need to exist in the voice control host device 108.

[0289]

According to the present invention, it is possible for a mobile terminal to provide a voice recognition / database search function having practical accuracy at a low cost without having to provide an advanced voice recognition / database search environment. It becomes possible.

In particular, according to the present invention, the user can easily designate only a really necessary search keyword as a selected search keyword from the mobile terminal, and can cause the voice control host device to execute a database search in real time. , It is possible to easily obtain the really necessary information.

Also, according to the present invention, by way of the personal handyphone system communication network and the Internet which are now spreading nationwide and worldwide,
The voice recognition / database search function having practical accuracy can be provided at lower cost and easily, and at the same time, the function provided by the present invention, the personal handyphone system call function, and the Internet access function can be seamlessly combined. It becomes possible to combine.

Further, according to the present invention, the mobile terminal and the voice control host device can be easily specified worldwide,
It is possible to easily realize coexistence of the voice recognition / database search processing service and other information processing services.

In addition, according to the present invention, it is possible to easily realize load distribution on the host device side.

[Brief description of the drawings]

FIG. 1 is an overall system configuration diagram.

FIG. 2 is an external view of a mobile terminal.

FIG. 3 is a functional block diagram of a mobile terminal.

FIG. 4 is an overall operation flowchart of processing of a mobile terminal.

FIG. 5 is an operation flowchart (part 1) of a transmission process.

FIG. 6 is an operation flowchart (part 2) of a transmission process.

FIG. 7 is a format diagram of communication data.

FIG. 8 is a format diagram of an IP header and a TCP header.

FIG. 9 is an operation flowchart of a calling process using PPP.

FIG. 10 is an operation flowchart (part 1) of a mobile terminal communication control unit.

FIG. 11 is an operation flowchart (part 2) of the mobile terminal communication control unit.

FIG. 12 is an operation flowchart (part 3) of the mobile terminal communication control unit.

FIG. 13 is an operation flowchart (part 4) of the mobile terminal communication control unit.

FIG. 14 is an operation flowchart (part 5) of the mobile terminal communication control unit.

FIG. 15 is a data configuration diagram of a processing terminal registration table.

FIG. 16 is a configuration diagram of a sentence speech recognition unit.

FIG. 17 is an operation flowchart of an input / output control unit in the sentence speech recognition unit.

FIG. 18 is a configuration diagram of a search keyword extraction unit.

FIG. 19 is an operation flowchart of an input / output control unit in a search keyword extraction unit.

FIG. 20 is a configuration diagram of a search control unit and an SQL DBMS.

FIG. 21 is an operation flowchart of an input / output control unit in the search control unit.

[Explanation of symbols]

Reference Signs List 101 mobile terminal 102 wireless base (wired connection device) 103 PHS network (public telephone network, ISDN network) 104 mobile terminal control host device 105 Internet 106 router device 107 LAN (local area network) 108 voice control host device 109 input unit 110 control Unit 111 communication unit 112 output unit 113 connection establishment unit 114 routing unit 115 packet transmission / reception unit 116 mobile terminal communication control unit 117 sentence speech recognition unit 118 search control unit 119 SQL DBMS 120 database 121 search keyword extraction unit 201, 301 microphone 202, 304 Camera (CCD camera) 203, 311 LCD display unit 204, 308 Speaker 205, 323 Wireless antenna 206, 325 Socket (for communication) 207 IC card slot 08 Optical transceiver (for optical communication) 302, 305 A / D conversion unit 303 Microphone control unit 306, 313 memory 307 Camera control unit 309 D / A conversion unit 310 Speaker control unit 312 LCD driver 314 LCD control unit 315 Touch panel control unit 316 CPU 317 RAM 318 ROM 319 IC card interface section 320 IC card 321 Communication control section 322 Wireless driver 324 Wired driver 1601, 1801, 2001 Input buffer queue 1602 Voice section detection section 1603 Voice analysis section 1604 Phoneme recognition section 1605 Phoneme standard pattern dictionary 1606 Word recognition unit 1607 Sentence recognition unit 1608, 1805, 2003 Output buffer queue 1609, 1806, 2004 Input / output control unit 1802 Search index creation unit 1803 search keyword extraction unit 1804 unnecessary keyword dictionary 2002 search execution unit

──────────────────────────────────────────────────続 き Continued on the front page (51) Int.Cl. ⁶ Identification code FIG06F 15/403 310Z H04B 7/26 109M

Claims (8)

[Claims] 1. A communication system in which a mobile terminal communicates with a host device, wherein the mobile terminal indirectly or via a relay network comprising one or both of a wireless network and a wired network. A host connecting means for directly connecting to the voice control host device, which is the host device, without going through a relay network; a voice input means for inputting voice; and a connection operation by the host connecting means, after the voice input means, Voice data transmitting means for transmitting input voice data to the voice control host device; search keyword receiving means for receiving a search keyword returned from the voice control host device; and a desired one from the received search keywords Search keyword selecting means for selecting as a selected search keyword; and a selected search keyword selected by the search keyword selecting means Selected search keyword transmitting means for transmitting to the voice control host device; search result data receiving means for receiving search result data returned from the voice control host device; search for displaying or editing the received search result data A mobile terminal connection means for identifying and connecting to the mobile terminal in response to a connection operation by a host connection means in the mobile terminal, comprising: a result data display / edit means; For each currently connected mobile terminal, voice data receiving means for receiving the voice data, and for each currently connected mobile terminal, perform voice recognition processing on voice data received by the voice data receiving means. Voice recognition means to be executed, and a recognized voice obtained by voice recognition processing by the voice recognition means for each of the currently connected mobile terminals. Search keyword extraction means for extracting a search keyword from data, and search keyword return means for returning, for each currently connected mobile terminal, the search keyword extracted by the search keyword extraction means to the corresponding mobile terminal. A selected search keyword receiving means for receiving the selected search keyword for each mobile terminal currently connected; a database means for storing search result data as a database corresponding to the plurality of selected search keywords; Search control means for searching, from the database means, search result data corresponding to the selected search keyword received by the selected search keyword receiving means for each mobile terminal, Search result data obtained by means of search processing And a search result data returning means for returning a search result data to the mobile terminal corresponding thereto. 2. The mobile terminal used in a communication system in which the mobile terminal communicates with a host device, wherein the mobile terminal is indirectly or via a relay network including one or both of a wireless network and a wired network. Host connection means for directly connecting to the audio control host device which is the host device without passing through the relay network; voice input means for inputting voice; and after the connection operation by the host connection means, the voice input means Voice data transmitting means for transmitting voice data input from the voice control host device to the voice control host device; search keyword receiving means for receiving a search keyword returned from the voice control host device; Search keyword selecting means for selecting an object as a selected search keyword; and a selected search keyword selected by the search keyword selecting means. Selected search keyword transmitting means for transmitting a search result to the voice control host device; search result data receiving means for receiving search result data returned from the voice control host device; and displaying or displaying the received search result data. A search result data display / editing means for editing; 3. The host device used in a communication system in which a mobile terminal communicates with a host device, wherein the host device is indirectly or via a relay network including one or both of a wireless network and a wired network. Mobile terminal connection means for identifying and connecting to the mobile terminal in response to a connection operation performed by the mobile terminal directly without passing through the relay network; and voice data for each currently connected mobile terminal. Voice data receiving means for receiving voice data, voice recognition means for performing voice recognition processing on voice data received by the voice data receiving means, for each currently connected mobile terminal, A search keyword extraction unit for extracting a search keyword from recognition voice data obtained by the voice recognition processing by the voice recognition unit for each terminal; A search keyword returning means for returning a search keyword extracted by the search keyword extraction means to a corresponding mobile terminal; and a selected search keyword receiving means receiving the selected search keyword for each currently connected mobile terminal. Database means for storing search result data as a database corresponding to a plurality of selected search keywords; and a search corresponding to the selected search keyword received by the selected search keyword receiving means for each currently connected mobile terminal. Search control means for searching result data from the database means; and search result data for returning search result data obtained by the search processing by the search control means to the corresponding mobile terminal for each currently connected mobile terminal. Replying means; and voice comprising: Control host device. 4. The mobile terminal has a personal handyphone system communication function; the relay network includes a personal handyphone system communication network and the Internet; the voice control host device connects to the Internet; The host connection means in the terminal transmits and connects to the mobile terminal control host apparatus having a gateway function between the public network including the personal handyphone system communication network and the Internet via the personal handyphone system communication network. The mobile terminal control host device connects to the voice control host device via the Internet by using the communication protocol on the Internet. Mobile terminal speech recognition / database search described in section Shin system, mobile terminal, or voice control host device. 5. The communication protocol is a hierarchical protocol including an Internet protocol layer and a transmission control protocol layer, and includes a header field of an Internet Protocol datagram, which is packet data of the Internet protocol layer transmitted on the Internet. Stores a source Internet protocol address and a destination Internet protocol address designating addresses of the mobile terminal and the voice control host device on the Internet, and the transmission control protocol is stored in a data field of the Internet protocol datagram. A transmission control protocol segment, which is packet data of a layer, is stored. The header field of the control protocol segment stores a source port number and a destination port number that specify a communication protocol for the voice recognition / database search processing, and the data field of the transmission control protocol segment stores the mobile terminal. The mobile terminal voice recognition / mobile terminal according to claim 4, wherein a terminal identification code for identifying the mobile terminal, the voice data, the search keyword, the selected search keyword, or the search result data is stored.
Database search communication system, mobile terminal, or voice control host device. 6. The voice control host device is mutually connected by a network, and the mobile terminal connection means, the voice data reception means, the voice recognition means, the search keyword extraction means, the search keyword reply means, the selection 6. A computer system comprising a plurality of host computers that implement functions corresponding to a search keyword receiving unit, the database unit, the search control unit, and the search result data returning unit in a distributed manner. The mobile terminal voice recognition / database search communication system, the mobile terminal, or the voice control host device according to any one of the above. 7. The search keyword extracting means classifies, for each currently connected mobile terminal, recognition speech data obtained by speech recognition processing by the speech recognition means according to a predetermined classification rule, thereby forming a search index. A search index creation unit to be created, and a phrase that satisfies a predetermined extraction criterion is extracted from the search index created by the search index creation unit for each mobile terminal currently connected, and a predetermined unnecessary phrase is extracted from the extracted phrase. The mobile terminal voice according to any one of claims 1 to 6, further comprising: a search keyword output unit that removes a keyword and extracts a new one of the resulting phrases as a search keyword. Recognition / database search communication system, mobile terminal, or voice control host device. 8. The search index creating means creates the search index by classifying words appearing in input data in order of appearance frequency, and the search keyword output means includes: A word whose appearance number is equal to or more than a predetermined number or a word whose rank is equal to or more than a predetermined appearance number rank, removes a predetermined unnecessary keyword from the extracted phrase, and replaces a new word among the resulting phrases. The mobile terminal voice recognition / database search communication system, the mobile terminal, or the voice control host device according to claim 7, wherein the mobile terminal is extracted as a search keyword.