JP7531241B2 - Information providing system, terminal device, and server - Google Patents

Description

CROSS-REFERENCE TO RELATED APPLICATIONS

This international application claims priority based on Japanese Patent Application No. 2012-137065, Japanese Patent Application No. 2012-137066, and Japanese Patent Application No. 2012-137067, filed with the Japan Patent Office on June 18, 2012, and the entire contents of Japanese Patent Application No. 2012-137065, Japanese Patent Application No. 2012-137066, and Japanese Patent Application No. 2012-137067 are incorporated by reference into this international application.

The present invention relates to an information providing system, a terminal device, and a server .

There is known a technology that searches a dictionary for an answer to an input question and outputs the searched answer by voice (see, for example, Patent Document 1). There is also known a technology that generates an answer to a question based on the content of a dialogue with a user (see, for example, Patent Document 2).

Patent No. 4832097 Patent No. 4924950

One aspect of the present invention is to provide an information providing system, a terminal device, and a server that are more user-friendly.

In a first aspect of the present invention , there is provided an information providing system including a request source device that is a request source of a request, a providing source device that is an information providing source, and a server capable of communicating with the request source device and the providing source device,
The requesting device
a designation request transmission unit configured to transmit a designation request, which is a request designating the providing device, to the server;
The server,
a provision request sending unit configured to send a provision request, which is a request for information provision, to the providing device specified in the specification request;
a response sending unit configured to obtain provision information from the provision source device in response to the provision request and to provide information to the request source device based on the provision information;
Equipped with
The providing device is
A determination unit configured to determine whether or not the information provision is permitted;
a providing unit configured to transmit to the server information to be provided in response to the request for information provision when the information provision is permitted, and to refuse to provide the information when the information provision is not permitted;
Equipped with.
In another aspect of the invention,
A voice response system including a request source device that is a request source of a request, a providing source device that is an information providing source, and a server capable of communicating with the request source device and the providing source device,
the providing device is configured to be able to set whether or not to permit information provision;
an information acquisition unit configured to acquire a request from the request source device, and if the request includes a request that specifies the providing source device, transmit the request to the specified providing source device, and if the information provision is permitted, acquire the provided information provided by the providing source device;
a providing unit configured to generate a voice response based on the provided information as a response to a request from the requesting device and provide the response to the requesting device;
Equipped with.
In another aspect of the invention,
A voice response device that responds to input text information by voice,
A response acquisition means for acquiring a plurality of different responses to the character information;
a voice output means for outputting the plurality of different responses in different voice tones;
The present invention is characterized by comprising:

This type of voice response device can output multiple responses in different tones of voice, so even if a single answer cannot be identified for a single piece of text information, different answers can be output in different tones of voice in a way that is easy for the user to understand. This makes it easier for the user to use.

The voice response device of the present invention may be configured, for example, as a terminal device carried by a user, or as a server that communicates with this terminal device. Furthermore, text information may be input using an input means such as a keyboard, or may be input by converting voice into text information.

In the above voice response device, as in the second aspect of the invention,
A voice input means for a user to input voice, and converting the input voice into text information;
a voice transmission means for transmitting a plurality of different responses to the text information to an external device which generates the plurality of different responses and transmits the responses to the voice response device;
Equipped with
The response acquisition means may acquire the response from the external device.

With such a voice response device, voice can be input into the voice response device, so text information can be input by voice. Also, a response can be generated in an external device, so the processing load on the voice response device can be reduced.

In the voice transmitting means, the operation of "converting the input voice into text information" may be performed by the voice response unit or an external device.
Further, in the above voice response device, as in the third aspect of the invention,
The voice response device or the external device includes a response recording means for recording a plurality of different responses, including positive and negative responses, for each of a plurality of pieces of text information;
the response acquisition means acquires the positive response and the negative response as the plurality of different responses;
The audio output means may reproduce the positive response and the negative response in different tones of voice.

With such a voice response device, responses with different positions, such as positive and negative responses, can be reproduced in different tones of voice, making it possible to reproduce the voice as if it were being spoken by a different person. This makes it less likely that the user listening to the voice will feel uncomfortable.

The tone of voice may be changed depending on the type of response or the words used when responding. For example, if the response is to be made in a gentle tone, a calm female voice may be used, and if the response is to be made in a fierce tone, a brave male voice may be used. In other words, the content of the response may be associated with a personality, and the tone of voice may be set according to the personality.

The voice response device can also be configured to be used at the reception desk of a workplace or company, as in the fourth aspect of the invention, or to convey things that a user finds difficult to say directly to someone.

When using a voice response unit at the reception desk, the name of the salesperson and the name of the company can be recorded in advance on the voice response unit or an external device, and if the person at the reception gives this name or company name, a response can be generated to play an audio refusal statement.

In addition, if you want to configure it to say things that are difficult to say, for example, before a date, you can tell the device what you want to say today, and the voice response device will speak for you (play back the audio) at an appropriate time (for example, at a preset time or when a certain amount of time has passed since the conversation stopped).

Alternatively, the device may be configured to speak words that trigger something difficult to say, such as "By the way, didn't you say that you were going to tell her something?" In other words, instead of outputting a response immediately, the device may output a response when a playback condition is met, such as after a certain period of time has passed.

Furthermore, in the above voice response device, as in a fifth aspect of the invention, the external device or the voice response device may obtain information for generating a response to text information from another voice response device. Also, in the above voice response device, as in a sixth aspect of the invention, when information for generating a response to text information is requested by another voice response device, the voice response device may return information in response to this request.

In this case, the voice response device can be equipped with sensors for detecting location information, temperature, humidity, illuminance, noise level, etc., as well as a database of dictionary information, etc., and can extract the necessary information upon request.

Such a voice response device (external device) can obtain information for generating a response from another voice response device. In this case, information specific to the other voice response device, such as the location of the other voice response device, can be obtained.

Also, the voice response unit can transmit its own unique information to other voice response units.
Furthermore, in the above voice response device, as in the seventh aspect of the invention, a response (e.g., a positive response or a negative response) output by the voice response device itself or another voice response device may be input as text information, and a response for refuting this response may be generated. In other words, from the user's point of view, it is possible to hear arguments from both pro and con positions. Then, after listening to this argument, the user can make a final decision.

This configuration can be realized using one or more voice response devices. In this case, the voice can be directly input and output to exchange voice between the multiple voice response devices, or wireless communication can be used.

In an eighth aspect of the invention,
A voice response device that responds to input text information by voice,
a personality information acquiring means for acquiring personality information in which the personality of a user or a related person representing a person related to the user is associated with the user in accordance with a preset classification;
A response acquisition means for acquiring response candidates representing a plurality of different responses to the text information;
a voice output means for selecting a response to be output from among response candidates in accordance with the personality information and outputting the selected response;
The present invention is characterized by comprising:

This type of voice response device can respond differently depending on the personality of the user and those related to the user (stakeholders). This makes it easy for users to use.

In the voice response device, as in a ninth aspect of the invention,
a first personality information generating means for generating personality information of the user or the related person based on answers to a plurality of questions set in advance,
The personality information acquiring means may acquire the personality information generated by the personality information generating means.

With such a voice response device, personality information can be generated in the voice response device. When generating the personality information, well-known personality analysis techniques (Rorschach test, Szondi test, etc.) may be used. When generating the personality information, aptitude test techniques used by companies and the like for employment examinations may also be used.

Further, in the above voice response device, as in a tenth aspect of the invention,
a second personality information generating means for generating personality information of the user or the related person based on a character string included in the input character information;
The personality information acquiring means may acquire the personality information generated by the personality information generating means.

According to such a voice response device, personality information can be generated while the user is using the voice response device.
In the voice response device, as in an eleventh aspect of the invention,
a preference information generating means for generating preference information indicating a tendency of a preference of the user or the related person based on a character string included in the character information,
The voice output means may select a response to be output from the response candidates based on the preference information, and output the selected response.

Such a voice response device can respond according to the preferences of the user or the person concerned.
Furthermore, in the above voice response device, as in the twelfth aspect of the invention, the user's actions (conversations, places traveled, and what is captured by the camera) may be learned (recorded and analyzed) to compensate for any shortcomings in the user's conversation.

For example, in a conversation in which the user responds with "Is hamburger steak okay today?" and "I think curry would be good," the device can add, "Because we had hamburger steak yesterday," thereby conveying the reason why the user said that curry was good.

Such configuration may also be implemented while on a phone call and may be configured to participate spontaneously in the user's conversation.
Further, in the above voice response device, as in a thirteenth aspect of the invention,
a response candidate acquisition means for acquiring response candidates from a predetermined server or the Internet;
The device may include:

According to such a voice response device, it is possible to obtain candidate responses not only from the device itself or an external device, but also from any device connected via the Internet or a dedicated line.
In the voice response device, as in a fourteenth aspect of the invention,
A character information generating means for converting a user's action into character information;
The device may include:

Here, the movement referred to in the present invention corresponds to conversation, handwriting, gestures (for example, sign language), and other movements resulting from muscle movements.
Such a voice response unit can convert the user's actions into text information.

Further, in the above voice response device, as in a fifteenth aspect of the invention,
The text information generating means converts the user's speech into text information and accumulates speech habits (pronunciation habits, etc.) as learning information (capturing and recording the characteristics).
This may be done.

According to such a voice response device, since it is possible to generate text information based on learning information, it is possible to improve the accuracy of generating text information.
In the voice response device, as in a sixteenth aspect of the invention,
a transfer means for transferring the learning information to another voice response unit;
The device may include:

With such a voice response device, even if the user uses another voice response device, the learning information recorded by this voice response device can be used. Therefore, the accuracy of generating text information can be improved even when using another voice response device.

Furthermore, in the above voice response device, as in the seventeenth aspect of the invention, any of the user's actions and operations may be detected and learning information or personality information may be generated based on these.

For example, such a voice response device can urge a user to leave the house a few minutes earlier the next day if it detects that the user has jumped on a train for several days in a row, or output voice or music to calm the user down if it detects from conversation that the user has a tendency to get angry easily.

In the voice response device, as in an eighteenth aspect of the invention,
The voice response unit may further include an other device information acquisition unit for acquiring information recorded in another voice response unit from the other voice response unit.

Such a voice response unit can generate a response based on information recorded in another voice response unit.
Further, in the above voice response device, as in a nineteenth aspect of the invention,
a reproduction condition determination means for determining whether or not a state of the voice response unit matches a reproduction condition that is set in advance as a condition for outputting a voice when the character information is not input;
a message reproducing means for outputting a preset message when the reproducing condition is met;
The device may include:

This type of voice response device can output voice even when no text information is input (i.e., when the user does not speak). For example, forcing the user to speak can be used to prevent drowsiness while driving a car. In addition, by determining whether or not a person living alone responds, the safety of the person can be confirmed.

In the above voice response device, as in a twentieth aspect of the invention,
The message reproducing means may acquire news information and output a message relating to the news in the form of a question requesting a response from the user.

Such a voice response device makes it possible to have a conversation about the news, which helps prevent the conversation from becoming the same every time. For example, if information about a certain company's stock price is obtained, the content of the conversation could be something like, "The stock price of XX company went up XX yen today. Did you know?"

Further, in the above voice response device, as in a twenty-first aspect of the invention,
The voice output means or message reproducing means may output a preset message by adding separately acquired external information (such as news or environmental information (temperature, weather, location information, etc.)).

Such a voice response unit can output a response that combines a predetermined message with the acquired information.
In the above voice response device, as in a twenty-second aspect of the invention,
A plurality of messages may be acquired, and a message to be reproduced may be selected and output according to the frequency of reproduction of the message.

With this type of voice response device, it is possible to make it harder to play messages that are played frequently, thereby creating a sense of randomness when playing messages, or to repeatedly play messages that are played frequently to draw attention or help solidify the memory.

Further, in the above voice response device, as in a twenty-third aspect of the invention,
a no-answer transmission means for transmitting information identifying the user and a message indicating that no answer has been received to a pre-set contact point when no response or reply to the message is received;
The device may include:

According to such a voice response device, if no answer is received, it is possible to report to a contact person, and therefore, for example, it is possible to report an abnormality in an elderly person living alone at an early stage.
In the above voice response device, as in a twenty-fourth aspect of the invention,
The message replay means memorizes the contents of the conversation and asks questions to obtain the same contents as those heard (memory confirmation process).
This may be done.

Such a voice response device can check the memory ability of the user and also help the user to solidify the memory.
Further, in the above voice response device, as in a twenty-fifth aspect of the invention,
a speech accuracy detection means for detecting the accuracy of pronunciation and accent of a voice input by a user;
an accuracy output means for outputting the detected accuracy;
The device may include:

Such a voice response device makes it possible to check the accuracy of pronunciation and accent, which is useful, for example, when practicing a foreign language.
In the above voice response device, as in a twenty-sixth aspect of the invention,
The accuracy level output means may output a voice containing the closest word when the accuracy level is equal to or lower than a certain value.

Such a voice response device allows the user to check the accuracy of pronunciation and accent.
Further, in the above voice response device, as in a twenty-seventh aspect of the invention,
The message reproducing means may be configured to output the same question again when the degree of accuracy is equal to or lower than a certain value.

With such a voice response unit, accurate answers can be obtained by outputting the same questions.
In the above voice response device, as in a twenty-eighth aspect of the invention,
a connection control means for identifying a communication partner based on input character information and connecting the communication partner to a communication destination previously set for each communication partner;
The device may include:

Such a voice response device can assist with reception duties and telephone responses.
In particular, in the above voice response device, as in the twenty-ninth aspect of the invention,
The connection control means may distinguish between business activities (sales) and visitors, and play a message to decline the call if it is business activities.

Such a voice response device makes it possible to exclude people who may be disruptive to the user's business without the user having to respond.
Furthermore, in the above voice response device, as in the invention of the thirtieth aspect, a keyword contained in the input character information (particularly voice) may be extracted and a connection may be made to a connection destination corresponding to the keyword. Note that a keyword such as the name of a destination and the connection destination may be associated in advance.

Such a voice response device can assist with tasks such as transferring calls and calling the reception desk.
In the above voice response unit, as in the thirty-first aspect of the invention, the requirements of what the other party is saying may be recognized based on keywords, and an outline of what the other party said may be conveyed to the user.

Such a voice response unit can assist in the intermediate business with customers.
Further, in the above voice response device, as in a thirty-second aspect of the invention,
The voice input device may be provided with an emotion determining means for reading emotions from the tone of voice input by the user and outputting which emotion corresponds to the voice input, usually including at least one of anger, joy, confusion, sadness, and elation.

Such a voice response unit can output a response according to the user's emotions.
Next, the invention of a thirty-third aspect is as follows:
a response generating means for generating a response according to an image captured by capturing an image of the surroundings of the voice response unit when the character information is input;
a voice output means for outputting the response by voice;
The present invention is characterized by comprising:

This type of voice response device can output a voice response in response to the captured image. This improves usability compared to a configuration in which a response is generated from text information only.

Specific configurations of the present invention include, for example, inputting text information to respond with what has been recognized, and outputting a voice indicating what (or who) has been recognized from the captured image.

In the above voice response unit, as in the thirty-fourth aspect of the invention,
a position identification means for searching for an object included in the character information from within the captured image by image processing and identifying the position of the object;
A guide means for guiding the object to a position thereof;
The device may include:

Such a voice response device can guide the user to an object in a captured image.
Further, in the above voice response device, as in a thirty-fifth aspect of the invention,
A voice input moving image acquisition means for acquiring a moving image capturing an image of a user's mouth shape when inputting character information by voice;
a character information conversion means for converting the voice into character information and correcting the character information by estimating unclear parts of the voice based on the moving image;
The device may include:

According to such a voice response device, the contents of the speech can be estimated from the shape of the mouth, so that unclear parts of the speech can be estimated well.
In the above voice response device, as in a thirty-sixth aspect of the invention,
The message reproducing means may detect an unexpected sound produced by the user to detect irritation or agitation of the user, and generate a message for suppressing the irritation or agitation.

According to such a voice response device, when the user becomes irritated or upset, it is possible to suppress these, and therefore it is possible to suppress the occurrence of trouble between the user and those around him or her.
Further, in the above voice response device, as in a thirty-seventh aspect of the invention,
a route information acquisition means for acquiring route information such as weather, temperature, humidity, traffic information, and road surface conditions to the destination when providing guidance to the destination;
The message reproducing means may be configured to output the route information by voice.

Such a voice response device can notify the user of the status (route information) to the destination by voice.
In the above voice response device, as in a thirty-eighth aspect of the invention,
A gaze detection means for detecting a gaze of a user;
a gaze movement request transmitting means for outputting a voice requesting the user to move their gaze to a predetermined position when the user does not move their gaze to a predetermined position in response to a call from the message reproducing means;
The device may include:

Such a voice response device allows the user to view a specific location, thereby enabling the user to reliably check for safety when driving a vehicle.
In the above voice response device, as in the invention of a thirty-ninth aspect,
The device may also be provided with a change request sending means for observing the position of a body part or facial expression, and outputting a voice requesting a change in the position of a body part or facial expression if there is little change in response to the call.

With such a voice response device, it is possible to move a part of the user's body to a specific position or induce the user to make a specific facial expression. The present invention can be used when driving a vehicle or during a physical examination, etc.

Further, in the above voice response device, as in a fortieth aspect of the invention,
A broadcast program acquisition means for acquiring a broadcast program similar to a broadcast program viewed by a user;
a broadcast program complementing means for complementing the interrupted broadcast program by outputting the broadcast program acquired by the broadcast program complementing means when the broadcast program is interrupted;
The device may include:

Such a voice response device can compensate for the interruption of a broadcast program being viewed by a user.
In the voice response device, as in a forty-first aspect of the invention,
The device is provided with a lyrics adding means for comparing a song with lyrics with the lyrics added by the user and outputting the lyrics by voice in a portion where the lyrics of the user are not included, when the user adds lyrics to a song without lyrics and sings the song.

Such a voice response device can compensate for a portion that the user cannot sing in so-called karaoke (a portion where the lyrics are cut off).
Further, in the above voice response device, as in a forty-second aspect of the invention,
a pronunciation output means for, when a captured image contains characters and a user asks how to read the characters, acquiring information about the characters from an external source and outputting the pronunciation of the characters contained in the information by voice;
The device may include:

Such a voice response unit can teach a user how to read characters.
In the voice response device, as in a forty-third aspect of the invention,
The device includes an action environment detection means for detecting an action of a user and an environment surrounding the user,
The message generating means may generate a message in response to a detected action or a surrounding environment.

Such a voice response device can notify dangerous places, restricted areas, etc. Also, it can detect abnormal behavior of the user.
Further, in the above voice response device, as in a forty-fourth aspect of the invention,
A health condition determining means for determining a health condition of a user based on a captured image of the user;
a health message generating means for generating a message according to a health condition;
The device may include:

Such a voice response device makes it possible to manage the health condition of the user.
In the voice response device, as in a forty-fifth aspect of the invention,
A reporting means for reporting to a predetermined contact when the health condition falls below a threshold value;
The device may include:

Such a voice response device can report when the health condition of the user is below a reference value, thereby making it possible to notify others of an abnormality at an earlier stage.
Furthermore, in the above voice response unit, as in a forty-sixth aspect of the invention, information about the user may be output in response to an inquiry from a person other than the user.

With such a voice response device, for example, if the contents of the user's meals and the distance of walking are detected, questions at a hospital or the like can be answered on behalf of the user. It may also be possible to have the device learn information such as the user's health condition and self-introduction.

In addition, each aspect of the invention does not need to be based on other inventions, and can be an independent invention as far as possible.

1 is a block diagram showing a schematic configuration of a voice response system to which the present invention is applied; FIG. 2 is a block diagram showing a schematic configuration of a terminal device. 10 is a flowchart showing a voice response terminal process executed by an MPU of the terminal device. 13 is a flowchart showing a voice response server process executed by a calculation unit of the server. FIG. 11 is an explanatory diagram illustrating an example of a response candidate DB; 11 is a flowchart showing an automatic conversation terminal process executed by an MPU of the terminal device. 13 is a flowchart showing an automatic conversation server process executed by a calculation unit of the server. 10 is a flowchart showing a message terminal process executed by an MPU of a terminal device. 13 is a flowchart showing a message server process executed by a calculation unit of the server. 13 is a flowchart showing a guidance terminal process executed by an MPU of the terminal device. 13 is a flowchart showing a guidance server process executed by a calculation unit of the server. 13 is a flowchart showing a reception process executed by a calculation unit of the server. 10 is a flowchart showing an information providing terminal process executed by an MPU of the terminal device. FIG. 2 is an explanatory diagram illustrating an example of a personality DB. 10 is a flowchart showing a personality information generating process executed by an MPU of the terminal device. FIG. 2 is an explanatory diagram illustrating an example of a preference DB. 13 is a flowchart showing a preference information generating process executed by a calculation unit of the server. FIG. 11 is an explanatory diagram showing an example of a combination of personality categories and preferences. 13 is a flowchart showing a motion character input process executed by a calculation unit of the server. 13 is a flowchart showing another terminal use process executed by a calculation unit of the server. 13 is a flowchart showing a storage confirmation process executed by a calculation unit of the server. 13 is a flowchart showing a pronunciation determination process 1 executed by a calculation unit of the server. 13 is a flowchart showing a pronunciation determination process 2 executed by a calculation unit of the server. 13 is a flowchart showing a pronunciation determination process 3 executed by a calculation unit of the server. 13 is a flowchart showing an emotion determination process executed by a calculation unit of the server. 13 is a flowchart showing an emotion response generation process executed by a calculation unit of the server. 13 is a flowchart showing a guidance process executed by a calculation unit of the server. 13 is a flowchart showing a movement request process 1 executed by a calculation unit of the server. 13 is a flowchart showing a movement request process 2 executed by a calculation unit of the server. 13 is a flowchart showing a broadcast music complementation process executed by a calculation unit of the server. 13 is a flowchart showing a character explanation process executed by a calculation unit of the server. 13 is a flowchart showing an action response terminal process executed by a calculation unit of the server. 13 is a flowchart showing an action response server process executed by a calculation unit of the server.

Hereinafter, an embodiment of the present invention will be described with reference to the drawings.
[First embodiment]
[Configuration of this embodiment]
A voice response system 100 to which the present invention is applied is a system configured to generate an appropriate response in a server 90 in response to a voice input in a terminal device 1, and to output the response by voice in the terminal device 1. In detail, as shown in Fig. 1, the voice response system 100 is configured such that a plurality of terminal devices 1 and the server 90 can communicate with each other via a communication base station 80 and an Internet network 85.

The server 90 has the functions of a normal server device. In particular, the server 90 has a calculation unit 101 and various databases (DB). The calculation unit 101 is configured as a well-known calculation device equipped with a CPU and memories such as ROM and RAM, and performs various processes based on the programs in the memory, such as communication with the terminal device 1 via the Internet network 85, reading and writing data in various DBs, and voice recognition and response generation for conversation with the user of the terminal device 1.

As shown in FIG. 1, the various DBs include a voice recognition DB 102, a predictive conversion DB 103, a voice DB 104, a response candidate DB 105, a personality DB 106, a learning DB 107, a preference DB 108, a news DB 109, a weather DB 110, a playback condition DB 111, a handwritten character/sign language DB 112, a terminal information DB 113, an emotion determination DB 114, a health determination DB 115, a karaoke DB 116, a report destination DB 117, a sales DB 118, and a client DB 119. Details of these DBs will be described each time the processing is explained.

Next, as shown in FIG. 2, the terminal device 1 is configured by providing a behavior sensor unit 10, a communication section 50, a notification section 60, and an operation section 70 in a predetermined housing.
The behavior sensor unit 10 is equipped with a well-known MPU 31 (microprocessor unit), memory 39 such as ROM and RAM, and various sensors, and the MPU 31 performs processing such as activating a heater to optimize the temperature of the sensor elements so that the sensor elements that make up the various sensors can properly detect the test subject (humidity, wind speed, etc.).

The behavior sensor unit 10 includes various sensors, such as a three-dimensional acceleration sensor 11 (3DG sensor), a three-axis gyro sensor 13, a temperature sensor 15 arranged on the rear surface of the housing, a humidity sensor 17 arranged on the rear surface of the housing, a temperature sensor 19 arranged on the front surface of the housing, a humidity sensor 21 arranged on the front surface of the housing, an illuminance sensor 23 arranged on the front surface of the housing, a wetness sensor 25 arranged on the rear surface of the housing, a GPS receiver 27 that detects the current location of the terminal device 1, and a wind speed sensor 29.

The behavior sensor unit 10 also includes various sensors, such as an electrocardiogram sensor 33, a heart sound sensor 35, a microphone 37, and a camera 41. The temperature sensors 15, 19 and the humidity sensors 17, 21 measure the temperature or humidity of the air outside the housing as the test subject.

The three-dimensional acceleration sensor 11 detects acceleration applied to the terminal device 1 in three mutually perpendicular directions (the vertical direction (Z direction), the width direction of the housing (Y direction), and the thickness direction of the housing (X direction)) and outputs the detection results.

The three-axis gyro sensor 13 detects angular acceleration (counterclockwise speed in each direction is considered positive) in the vertical direction (Z direction) and any two directions perpendicular to the vertical direction (the width direction of the housing (Y direction) and the thickness direction of the housing (X direction)) as the angular velocity applied to the terminal device 1, and outputs the detection result.

The temperature sensors 15 and 19 are configured with, for example, a thermistor element whose electrical resistance changes depending on the temperature. In this embodiment, the temperature sensors 15 and 19 detect the temperature in degrees Celsius, and all temperature indications in the following description are in degrees Celsius.

Humidity sensors 17 and 21 are configured, for example, as well-known polymer film humidity sensors. These polymer film humidity sensors are configured as capacitors whose dielectric constant changes as the amount of moisture contained in the polymer film changes in response to changes in relative humidity.

The illuminance sensor 23 is configured as a well-known illuminance sensor including, for example, a phototransistor.
The wind speed sensor 29 is, for example, a known wind speed sensor, and calculates the wind speed from the power (amount of heat dissipation) required to maintain the heater temperature at a predetermined temperature.

The heart sound sensor 35 is configured as a vibration sensor that captures vibrations caused by the beating of the user's heart, and the MPU 31 distinguishes between vibrations and noises caused by beating and other vibrations and noises based on the detection results of the heart sound sensor 35 and the heart sounds input from the microphone 37.

The wetness sensor 25 detects water droplets on the surface of the housing, and the electrocardiogram sensor 33 detects the heartbeat of the user.
The camera 41 is disposed within the housing of the terminal device 1 so that the outside of the terminal device 1 is its imaging range.

The communication unit 50 includes a well-known MPU 51, a wireless telephone unit 53, and a contacts memory 55, and is configured to be able to acquire detection signals from various sensors that make up the behavior sensor unit 10 via an input/output interface (not shown). The MPU 51 of the communication unit 50 then executes processing according to the detection results from the behavior sensor unit 10, input signals input via the operation unit 70, and programs stored in a ROM (not shown).

Specifically, the MPU 51 of the communication unit 50 performs the functions of a motion detection device that detects specific motions performed by the user, a positional relationship detection device that detects the positional relationship with the user, an exercise load detection device that detects the load of the exercise performed by the user, and a function of transmitting the results of processing by the MPU 51.

The wireless telephone unit 53 is configured to be able to communicate with, for example, a mobile phone base station, and the MPU 51 of the communication section 50 outputs the processing results of the MPU 51 to the notification section 60, or transmits them to a pre-set destination via the wireless telephone unit 53.

The contacts memory 55 functions as a storage area for storing location information of places visited by the user. This contacts memory 55 stores information on contacts (such as telephone numbers) to be contacted if an abnormality occurs to the user.

The notification unit 60 includes a display 61, for example, an LCD or an organic EL display, illumination 63, for example, an LED capable of emitting light in seven colors, and a speaker 65. Each component of the notification unit 60 is driven and controlled by the MPU 51 of the communication unit 50.

Next, the operation unit 70 includes a touch pad 71 , a confirmation button 73 , a fingerprint sensor 75 , and a rescue request lever 77 .
The touch pad 71 outputs a signal according to the position and pressure of the touch by a user (the user, the user's guardian, etc.).

The confirmation button 73 is configured so that when pressed by the user, the contacts of a built-in switch are closed, and the communication unit 50 is capable of detecting that the confirmation button 73 has been pressed.

The fingerprint sensor 75 is a well-known fingerprint sensor, and is configured to be able to read fingerprints using, for example, an optical sensor. Note that instead of the fingerprint sensor 75, any means capable of recognizing the physical characteristics of a person (means capable of biometric authentication: means capable of identifying an individual), such as a sensor that recognizes the shape of veins in the palm of the hand, can be used.

It also includes a rescue request lever 77 that, when operated, connects to a specified contact point.
[Processing of this embodiment]
The process performed in the voice response system 100 will be described below.

The voice response terminal process performed by the terminal device 1 is a process of accepting voice input by the user, sending this voice to the server 90, and, upon receiving a response to be output from the server 90, reproducing this response as voice. This process is started when the user inputs via the operation unit 70 that he or she will make a voice input.

In detail, as shown in FIG. 3, first, the microphone 37 is put into a state in which it can receive input (ON state) (S2), and image capture (recording) by the camera 41 is started (S4). Then, it is determined whether or not there has been audio input (S6).

If there is no voice input (S6: NO), it is determined whether a timeout has occurred (S8). Here, a timeout means that the allowable time for waiting for processing has been exceeded, and in this case the allowable time is set to, for example, about 5 seconds.

If the timeout has occurred (S8: YES), the process proceeds to S30, which will be described later. If the timeout has not occurred (S8: NO), the process returns to S6.
If there is voice input (S6: YES), the voice is recorded in memory (S10), and it is determined whether or not the voice input has ended (S12). Here, if the voice is interrupted for a certain period of time or more, or if an instruction to end the voice input is input via the operation unit 70, it is determined that the voice input has ended.

If the voice input has not ended (S12: NO), the process returns to S10. If the voice input has ended (S12: YES), the device transmits data such as an ID for identifying the device, the voice, and captured images to the server 90 in packets (S14). The data transmission process may be performed between S10 and S12.

Next, it is determined whether the data transmission is complete (S16). If the data transmission is not complete (S16: NO), the process returns to S14.
If the transmission is complete (S16: YES), the process judges whether data (packets) transmitted in the voice response server process described below have been received (S18). If no data has been received (S18: NO), the process judges whether a timeout has occurred (S20).

If the timeout has occurred (S20: YES), the process proceeds to S30, which will be described later. If the timeout has not occurred (S20: NO), the process returns to S18.
If data is being received (S18: YES), a packet is received (S22).In this process, one or more different responses to the text information, each of which is associated with a different tone of voice, are obtained.

Then, it is determined whether or not the reception is complete (S24). If the reception is not complete (S24: NO), it is determined whether or not a timeout has occurred (S26).
If the timeout has occurred (S26: YES), a message indicating that an error has occurred is output via the notification unit 60, and the voice response terminal process is terminated. If the timeout has not occurred (S26: NO), the process returns to S22.

Also, if reception is complete (S24: YES), a response based on the received packet is output as voice from the speaker 65 (S28). In this process, if multiple responses are to be played back, each of the multiple responses is played back in a different voice tone. When this process is completed, the voice response terminal process is terminated.

Next, the voice response server process performed by the server 90 (external device) will be described with reference to FIG. 4. The voice response server process receives voice from the terminal device 1, performs voice recognition to convert the voice into text information, and generates a response to the voice and returns it to the terminal device 1. In particular, in this embodiment, multiple responses may be transmitted in association with voices of different tones.

As shown in FIG. 4, the voice response server process first determines whether a packet has been received from any of the terminal devices 1 (S42). If no packet has been received (S42: NO), the process of S42 is repeated.

If a packet has been received (S42: YES), the terminal device 1 with which the communication is to be made is identified (S44). In this process, the terminal device 1 is identified by the ID of the terminal device 1 contained in the packet.

Next, the voice contained in the packet is recognized (S46). Here, in the voice recognition DB 102, many voice waveforms are associated with many characters. In addition, in the predictive conversion DB 103, words that tend to be used following a certain word are associated with each other.

In this process, the voice recognition DB 102 and the predictive conversion DB 103 are referenced to perform a well-known voice recognition process and convert the voice into text information.
Next, the captured image is processed to identify objects in the captured image (S48), and the user's emotions are determined based on the waveform of the voice, the endings of words, etc. (S50).

In this process, by referencing emotion determination DB114, which associates voice waveforms (tone of voice) and endings of words with emotion categories such as anger, joy, confusion, sadness, and elation, it is determined which category the user's emotion falls into, and the determination result is recorded in memory. Next, by referencing learning DB107, words that the user frequently speaks are searched for, and parts of the text information generated by voice recognition that were ambiguous are corrected.

In addition, the learning DB 107 records the characteristics of each user, such as the words the user frequently uses and pronunciation habits. In addition, data is added to and corrected in the learning DB 107 during conversations with the user.

Next, the corrected character information is identified as the input character information (S54), and a sentence similar to the character information is input and searched from the response candidate DB 105 to obtain a response from the response candidate DB 105 (S56). Here, as shown in FIG. 5, the response candidate DB 105 uniquely associates the input character information, the first output, the tone of voice of the first output, the second output, and the tone of voice of the second output.

For example, as shown in the first row of FIG. 5, when text information "Today's weather at ※" is input, the first output "Today's weather at ※ is ※" is output in association with the voice tone of Woman 1. However, the "※" part is obtained by accessing the weather DB 110, which associates area names with weather forecasts for the next few days in those areas.

Furthermore, when the text information "Today's weather is ※", the weather at the time when today's weather changes is also obtained from the weather DB 110, and the second output "However, ※ is ※" is output in association with Man 1's voice tone. If today's weather in Tokyo is sunny and tomorrow's weather is rainy, and "Today's weather in Tokyo" is input, "Today's weather in Tokyo is sunny" is output in Woman 1's voice tone, and "However, it will rain tomorrow" is output in Man 1's voice tone.

In this embodiment, a case where multiple responses are output has been described, but if there is only one answer to the input, there will be only one response. Therefore, it is determined whether there is only one response (S58). If there is only one response (S58: YES), the process proceeds to S62, which will be described later.

If there are multiple responses (S58: NO), the response content and tone of voice are associated (S60). Here, the voice DB 104 stores a database of artificial voices for each tone of voice, and in this process, the tone of voice set for each response is associated with the tone of voice in the database.

Then, the response content is converted into voice (S62). In this process, the response content (text information) is output as voice based on the database stored in the voice DB 104.

Then, the generated response (voice) is transmitted as a packet to the communication partner terminal device 1 (S64). Note that the voice of the response may be transmitted as a packet while being generated.
Next, the conversation content is recorded (S68). In this process, the input character information and the output response content are recorded as the conversation content in the learning DB 107. At this time, keywords (words recorded in the voice recognition DB 102) included in the conversation content, pronunciation characteristics, and the like are recorded in the learning DB 107.

When this processing is completed, the voice response server processing is terminated.
[Effects of this embodiment]
The voice response system 100 described above in detail is a system that responds to input text information by voice, and the terminal device 1 (MPU 31) obtains a plurality of different responses to the text information and outputs the plurality of different responses in different voice tones.

With this voice response system 100, multiple responses can be output in different tones of voice, so even if a single answer cannot be identified for a single piece of text information, different answers can be output in different tones of voice in a way that is easy for the user to understand. This makes it easier for the user to use.

In addition, in the above voice response system 100, the terminal device 1 inputs voice from the user via the microphone 37, and the server 90 (calculation unit 101) converts the input voice into text information, generates multiple different responses to the text information, and transmits them to the terminal device 1. Then, the terminal device 1 obtains the responses from the server 90.

With this voice response system 100, voice can be input into the terminal device 1, so text information can be input by voice. In addition, a response can be generated in the server 90, so the processing load on the voice response system 100 can be reduced.

Furthermore, in the voice response system 100, the server 90 converts the user's speech into text information and accumulates speech habits (pronunciation habits, etc.) as learning information (capturing characteristics and recording these characteristics).

According to such a voice response system 100, since text information can be generated based on learning information, the accuracy of generating text information can be improved.
Furthermore, in the above voice response system 100, the server 90 reads emotions from the tone of voice input by the user, and outputs which emotion corresponds to the voice, usually including at least one of anger, joy, confusion, sadness, and elation.

According to such a voice response system 100, a response can be output according to the user's emotions.
[Modification of the first embodiment]
In this embodiment, voice recognition is used as a configuration for inputting character information, but it is not limited to voice recognition, and input may be made using an input means (operation unit 70) such as a keyboard or a touch panel. Also, the operation of "converting input voice into character information" is performed by the server 90, but may be performed by the terminal device 1.

Furthermore, in the above voice response system 100, the server 90 may include a response candidate DB 105 in which a plurality of different responses, including positive and negative responses to each piece of text information, are recorded, and the terminal device 1 may acquire the positive and negative responses as the plurality of different responses and play back the positive and negative responses in different tones of voice.

For example, as shown in the second row of Figure 5, when a voice is input asking "May I buy this item?", positive information about the item, such as a good reputation, is output in a female voice. On the other hand, negative information, such as a bad reputation, is output in a voice tone different from the female voice associated with the positive information (here, a male voice).

With this voice response system 100, responses with different positions, such as positive and negative responses, can be reproduced in different tones of voice, making it possible to reproduce the voice as if it were being spoken by a different person. This makes it possible to reduce the sense of incongruity felt by the user listening to the voice.

The tone of voice may be changed depending on the type of response or the words used when responding. For example, if the response is to be made in a gentle tone, a calm female voice may be used, and if the response is to be made in a fierce tone, a brave male voice may be used. In other words, the content of the response may be associated with a personality, and the tone of voice may be set according to the personality.

Furthermore, in the above voice response system 100, a response (e.g., a positive or negative response) output by one's own terminal device 1 or another terminal device 1 may be input as text information, and a response may be generated to refute this response. In other words, from the user's perspective, it is possible to hear arguments from both pro and con positions. Then, after listening to this argument, the user can make a final decision.

This configuration can be realized using one or more terminal devices 1. For multiple terminal devices 1 to exchange audio with each other, audio may be input and output directly, or wireless communication may be used. When multiple terminal devices 1 communicate with the server 90, data may be sent to the other terminal devices 1 in the process of S66.

Furthermore, in the voice response system 100, the calculation unit 101 may learn (record and analyze) the user's actions (conversations, places traveled, and what is captured by the camera) and compensate for shortcomings in the user's conversation.

For example, in a conversation in which the user responds with "Is hamburger steak okay today?" and "I think curry would be good," the device can add, "Because we had hamburger steak yesterday," thereby conveying the reason why the user said that curry was good.

Such configuration may also be implemented while on a phone call and may be configured to participate spontaneously in the user's conversation.
Furthermore, in the voice response system 100, the server 90 may obtain the response candidates from a predetermined server or from the Internet.

According to such a voice response system 100, response candidates can be obtained not only from the server 90 but also from any device connected via the Internet, a dedicated line, or the like.
[Second embodiment]
[Processing of the second embodiment]
Next, another embodiment of the voice response system will be described. In the following embodiment, only the parts different from the voice response system 100 of the first embodiment will be described in detail, and the same parts as those of the voice response system 100 of the first embodiment will be denoted by the same reference numerals and will not be described.

In the voice response system of the second embodiment, voice is output even when the user does not input text information. In detail, the terminal device 1 executes the automatic conversation terminal process shown in FIG. 6. The automatic conversation terminal process is a process that is started, for example, when the terminal device 1 is powered on, and is executed repeatedly thereafter.

In the automatic conversation terminal process, first, it is determined whether or not the setting for automatic conversation is ON (S82). Note that the user can set whether or not to perform automatic conversation via the operation unit 70 or by inputting voice.

If the automatic conversation is OFF (S82: NO), the automatic conversation terminal process is terminated. If the automatic conversation is ON (S82: YES), a message indicating that the automatic conversation mode has been set is sent to the server 90 together with an ID for identifying the device (S84).

Then, it is determined whether or not a packet has been received from the server 90 (S86). If a packet has not been received (S86: NO), the process of S86 is repeated. If a packet has been received (S86: YES), the same processes as S22 to S30 described above are carried out, and when these processes are completed, the automatic conversation terminal process is terminated.

The server 90 also executes the automatic conversation server process shown in FIG. 7. The automatic conversation server process is started, for example, when the server 90 is powered on, and is then executed repeatedly.

In the automatic conversation server process, first, it is determined whether or not a notification that the automatic conversation mode has been set has been received from the terminal device 1 (S92). If a notification that the automatic conversation mode has been set has not been received (S92: NO), the process proceeds to S98.

If the device has received a notification that the automatic conversation mode has been set (S92: YES), the device identifies the terminal device 1 that will be the communication partner based on the ID included in the received packet (S94), and sets the device to start an automatic conversation with this communication partner (S96). Next, it is determined whether the playback conditions are met for each of the terminal devices 1 that have been set to start an automatic conversation (S98).

Here, playback conditions refer to, for example, a certain amount of time having passed since the last conversation (voice input), a certain time of day, specific weather, or when any sensor value indicates an abnormality.

If the playback conditions are not met (S98: NO), the automatic conversation server process ends. If the playback conditions are met (S98: YES), a message according to the playback conditions is generated (S100).

Here, the message according to the playback conditions may be, for example, a standard phrase such as "Good morning" or "Hello," or may be about the latest news obtained from the news DB 109, which automatically updates the latest news. In the case of a message about the latest news, for example, if information about the stock price of a certain company has been obtained, the message may be, "The stock price of XX company went up by XX yen today. Did you know?"

When this process is completed, the processes of S42 to S54 described above are carried out. Then, when the process of S54 is completed, it is determined whether or not a predetermined response has been obtained from the terminal device 1 that is the communication partner (S112). Here, the predetermined response may be, for example, some kind of voice or a specific response. For example, a specific response would be "I know" or "I don't know" in response to the question "Do you know?", and an answer containing words indicating the weather, such as "It's raining" or "It's sunny" in response to the question "What's the weather like now?".

If there is a predetermined response (S112: YES), the automatic conversation server process ends. If there is no predetermined response (S112: NO), the message sent in S100 is resent (S114). When resending a message in this way, the tone of voice is changed to produce a stronger, more strict tone of voice.

Then, the report destination DB 117, in which the terminal device 1 and the report destination are previously associated, is referenced, and a message is sent to the specified report destination indicating that no reply was received (S116). When this processing is completed, the automatic conversation server processing is terminated.

[Effects of the second embodiment]
In the above voice response system 100, when no text information is input, the server 90 judges whether or not the situation of the voice response system 100 matches a reproduction condition that is set in advance as a condition for outputting a voice. If the reproduction condition is matched, the server 90 outputs a preset message.

This voice response system 100 can output voice even when no text information is input (i.e., when the user does not speak). For example, forcing the user to speak can be used to prevent drowsiness while driving a car. In addition, by determining whether or not a person living alone responds, the safety of the person can be confirmed.

In the voice response system 100, the server 90 acquires news information and outputs a message relating to the news in the form of a question requesting a response from the user.
According to such a voice response system 100, it is possible to have a conversation about the news, so that it is possible to prevent the conversation from becoming the same over and over again.

Furthermore, in the above voice response system 100, the server 90 outputs a preset message by adding separately acquired external information (such as news and environmental information (temperature, weather, location information, etc.)).

According to such a voice response system 100, it is possible to output a response that combines a predetermined message with the acquired information.
Furthermore, in the voice response system 100, when no response to a response or message is received, the server 90 transmits to a preset contact point information identifying the user and a message indicating that no response was received.

With this type of voice response system 100, if no response is received, a notification can be sent to a contact. Therefore, for example, abnormalities in the lives of elderly people living alone can be reported early on.

[Modification of the second embodiment]
In addition, in the voice response system 100, the server 90 may acquire a plurality of messages and select and output a message to be reproduced depending on the reproduction frequency of the message.

With this type of voice response system 100, it is possible to make it harder to play messages that are played frequently, thereby creating a sense of randomness when playing messages, or to repeatedly play messages that are played frequently to draw attention or help solidify memories.

[Third embodiment]
[Processing of the third embodiment]
Next, in the voice response system of the third embodiment, the terminal device 1 is configured to convey things that the user finds difficult to say directly to someone. For example, if a user speaks to the device before a date about what he or she would like to say today, the voice response system 100 will speak on the user's behalf (play back audio) at an appropriate time (for example, at a preset time or when a certain amount of time has passed since the conversation stopped).

In detail, the terminal device 1 performs the message terminal process shown in FIG. 8, and the server 90 performs the message server process shown in FIG. 9. The message terminal process is started, for example, when the terminal device 1 is powered on, and is then repeatedly executed.

As shown in FIG. 8, the message terminal process first determines whether the message mode has been set by the user (S132). If the message mode has not been set (S132: NO), the process of S132 is repeated.

If the message mode is set (S132: YES), the processes of S2 to S8 are carried out, and if a positive determination is made in S6, the message mode flag is set to ON in the memory of the terminal device 1 (S134). Then, the processes of S10 to S16 are carried out.

If the determination in S16 is positive, it is determined whether or not a packet has been received from the server 90 (S136). If a packet has not been received (S136: NO), the process of S136 is repeated. If a packet has been received (S136: YES), the processes of S24 to S30 are carried out, and the message terminal process is terminated.

Next, the message server process is a process that starts, for example, when the server 90 is powered on, and is executed repeatedly thereafter. In detail, first, it is determined whether or not a packet has been received from any of the terminal devices 1 (S142). If no packet has been received (S142: NO), the process proceeds to S156, which will be described later.

If a packet has been received (S142: YES), the terminal device 1 of the communication partner is identified (S44), and it is determined whether the packet contains a mode flag such as a message mode flag (S144). If no mode flag is present (S144: NO), the process proceeds to S148.

If there is a mode flag (S144: YES), the server 90 also sets the mode by setting the flag corresponding to the communication partner's terminal device 1 to ON (S146). For example, if the message mode flag is the corresponding message mode, the processes of S46 to S152 described below are performed, and if the guidance mode flag described below is the corresponding guidance mode, the processes of S46 to S176 (see FIG. 11) are performed.

Then, it is determined whether the message flag is ON (S148). If the message flag is ON (S148: YES), the processes of S46 to S54 are carried out, and then the message playback conditions are extracted (S150).

Here, the message playback conditions can be set in advance by the user via the operation unit 70 of the terminal device 1, and include, for example, the time and the location. The message playback conditions are transmitted to the server 90 when packets are transmitted for message terminal processing.

Then, the message and the voice (tone of voice) are associated and recorded in memory (S152), and the process proceeds to S156. If the message flag is OFF (S148: NO), the process proceeds to other modes (S154), and it is determined whether playback timing has arrived (S156). Here, playback timing refers to the content set in the message playback conditions.

If it is not playback time (S156: NO), the message server process immediately ends. If it is playback time (S156: YES), steps S62 to S64 are carried out, and the message server process ends.

[Effects of the third embodiment]
According to the voice response system of the third embodiment, the voice input by the user is not immediately reproduced, but is reproduced when the message reproduction condition is met after a certain period of time.

For example, as shown in the third row of Figure 5, if you input "Tell XX that," the sentence you want to convey will be played after XX's voice is recognized (heard).

[Modification of the third embodiment]
In the third embodiment, the contents of what the user said are reproduced, but it may be configured to speak words that trigger something difficult to say, such as "Now that I think of it, didn't you say that you were going to tell her something?" In detail, the terminal device 1 executes the guiding terminal process shown in Fig. 10, and the server 90 executes the guiding server process shown in Fig. 11.

The guidance terminal process is a process that is started, for example, when the terminal device 1 is powered on, and is then repeatedly executed. For example, the guidance terminal process is a process that is started, for example, when the terminal device 1 is powered on, and is then repeatedly executed.

In the guidance terminal process, as shown in FIG. 10, first, it is determined whether or not the guidance mode has been set by the user (S162). If the guidance mode has not been set (S162: NO), the process of S162 is repeated.

If the guidance mode is set (S162: YES), the processes of S2 to S8 are carried out, and if a positive determination is made in S6, the guidance mode flag is set to ON in the memory of the terminal device 1 (S164). Then, the processes of S10 to S16 are carried out.

If the determination in S16 is positive, it is determined whether or not a packet has been received from the server 90 (S166). If a packet has not been received (S166: NO), the process of S166 is repeated. If a packet has been received (S166: YES), the processes of S24 to S30 are carried out, and the guidance terminal process is terminated.

Next, the guidance server process is started, for example, when the server 90 is powered on, and is then repeatedly executed. In detail, the processes of S142 to S146 described above are executed. Then, it is determined whether the guidance flag is ON (S172).

If the guidance flag is ON (S172: YES), the processes of S46 to S54 are carried out, and then the guidance regeneration conditions are extracted (S174).
Here, like the message playback conditions, the user can set the guided playback conditions in advance via the operation unit 70 of the terminal device 1, and the guided playback conditions include, for example, the time and the location. The guided playback conditions are transmitted to the server 90 when a packet is transmitted for the message terminal processing.

Next, prompting contents are generated, and the prompting contents and the voice (tone of voice) are associated and recorded in memory (S176). For example, the prompting contents are obtained by searching for words expressing desires such as "want to" and "hope" contained in the input text information, extracting keywords before these words, and outputting words registered as words that induce these keywords as prompting contents. The keywords and words indicating the prompting contents are associated with each other in advance and recorded in the response candidate DB 105.

Then, the above-mentioned processing from S156 onwards is carried out, and the server processing is terminated. Also, if the guidance flag is OFF (S172: NO), processing related to another mode is carried out (S154), the above-mentioned processing from S156 onwards is carried out, and the server processing is terminated.

According to the configuration of the modified example of the third embodiment, the user can be guided to speak the words he or she wants to say, rather than directly outputting the words he or she wants to say.
[Fourth embodiment]
[Processing of the Fourth Embodiment]
Next, an example of using the terminal device 1 for reception work will be described. In this embodiment, the terminal device 1 is installed at the reception desk of a company. It can also be used for telephone reception for a company's main telephone line or telephone banking. Here, in this embodiment, the process of S56 in the first embodiment is realized by replacing it with the reception process shown in FIG. 12.

As shown in FIG. 12, the reception process first determines whether the text information includes a company name (S192). In this process, it determines whether the text information includes a general name or a company name (recorded in the voice recognition DB 102).

If the text information does not include a company name or an individual name (S192: YES), a response is generated to ask for the company name and individual name (S194), and the reception process ends. In this process, a response such as "Please tell us your name and purpose" is generated, for example.

If the text information contains a company name or an individual name (S192: NO), the company name or individual name is extracted from the sales DB 118 and the client DB 119 (S196). Here, the sales DB 118 records the names of companies and personnel who have made sales visits in the past, or complainers who only complain. The client DB 119 records company names, personnel of those companies, personnel on the user's side of the terminal device 1 (their own side), schedules such as planned meeting times, and contact information for each personnel.

Next, it is determined whether the company name or individual name could be extracted from the sales DB 118, that is, whether the company name or individual name contained in the text information was included in the sales DB 118 (S198). If the company name or individual name could be extracted from the sales DB 118 (S198: YES), a sales refusal response (a response refusing to be referred) is generated to decline the sale (S200), and the reception process ends.

If the company name or individual name cannot be extracted from the sales DB 118 (S198: NO), it is determined whether the person at the reception desk is scheduled to visit at a nearby time (for example, within one hour before or after the current time) in the schedule in the client DB 119 (S202). If the person is scheduled to visit at a nearby time (S202: YES), the contact information of the person in charge of this person is extracted from the client DB 119, and the person is connected to this person so that they can converse (S204). In this process, a connection can be made to the person's internal phone number, mobile phone, etc.

Next, a reception response for the client is generated (S206). Here, as the reception response for the client, for example, a response such as "Dear Mr. XX, thank you for your continued support. We are connecting you to the person in charge, so please wait a moment." is generated. When this process is completed, the reception process ends.

If the person is not coming soon (S202: NO), the system connects to a pre-set contact for reception and connects the person to the receptionist so that the person can talk to the receptionist (S208). Then, a normal reception response is generated (S210).

At this point, a normal reception response would be something like "We are connecting to reception, please wait a moment." When this process is complete, the reception process ends.

[Effects of the fourth embodiment]
The voice response system 100 is configured to be used at the reception desk of a workplace or company. In this configuration, the name of a salesperson and the name of the company are recorded in advance in the sales DB 118 of the server 90, and when a person at the reception gives this name and company name, a response is generated to play back a voice of refusal.

In the voice response system 100, the server 90 identifies the communication partner based on the input character information, and connects the communication partner to a communication destination preset for each communication partner.
Such a voice response system 100 can assist with reception work and telephone correspondence. Also, such a voice response system 100 can remove a person who may be interfering with the user's work without the user having to respond to the person.

Furthermore, in the above voice response system 100, the server 90 extracts keywords contained in the input text information (particularly voice) and connects to the connection destination that corresponds to the keyword. Note that keywords, such as the name of the other party, and the connection destination are associated in advance.

Such a voice response system 100 can assist with tasks such as transferring calls and calling the reception desk.
[Modification of the fourth embodiment]
In the above embodiment, the connection destination is set depending on the other party, but this technology may also be applied to recognize requirements (keywords contained in the text information) when accepting calls for telephone banking, telephone shopping, etc., and change the connection destination depending on the requirements.

In the voice response system 100, the server 90 may recognize what the other party is saying based on keywords and inform the user of an outline of what the other party said.
Such a voice response system 100 can assist in the intermediate business with customers.

[Fifth embodiment]
[Processing of the Fifth Embodiment]
Next, the terminal device 1 may receive a request from another terminal device 1 and provide the information requested by the other terminal device 1.

When configured in this manner, in the process of S56, the server 90 requests the necessary information from the other terminal device 1, and generates a response after acquiring the necessary information from the other terminal device 1. Then, in the terminal device 1 that provides the necessary information, the information providing terminal process shown in FIG. 13 is performed. The information providing terminal process is a process that is started, for example, when a request is received from the server 90.

As shown in FIG. 13, the information providing terminal process first extracts the information providing destination (S222). This information providing destination indicates another terminal device 1 requesting information, and an ID for identifying this other terminal device 1 is included in the request from the server 90.

Next, it is determined whether the person is a person to whom information should be provided (S224). Here, the IDs of people to whom information should be provided, such as family members and friends, are pre-recorded in the terminal information DB 113. In this process, the determination is made by referring to this terminal information DB 113.

If the information is provided to the server 90 and the request is received from the server 90 (S226) (S228), the server 90 receives the requested information from its own memory 39 and various sensors (S226). If the information is not provided to the server 90 and the request is not received (S224: NO), the server 90 receives a message refusing to provide the information (S230).

When this processing is completed, the information provision terminal processing is terminated.
In this configuration, for example, as shown in the fourth row of FIG. 5, in response to the question "What is Mr./Ms. XX doing?", the server 90 requests location information from Mr./Ms. XX's terminal device 1, and the terminal device 1 returns the location information.

Then, the server 90 recognizes Mr. XX's actions based on the location information. For example, if he is moving on the tracks at a speed faster than a human can run, it will determine that he is traveling on a train and generate a response such as "Mr. XX is on a train. It appears that he is on his way home."

[Effects of the Fifth Embodiment]
In the voice response system 100, the server 90 acquires information recorded in another terminal device 1 different from the requesting terminal device 1, and provides the information to the other terminal device 1. That is, in the voice response system 100, the server 90 acquires information for generating a response to text information from the other terminal device 1.

According to such a voice response system 100, a response can be generated based on information recorded in another terminal device 1.
Furthermore, in the voice response system 100, when a terminal device 1 is requested by another terminal device 1 for information for generating a response to text information, the terminal device 1 returns information in response to this request.

In this configuration, the terminal device 1 is equipped with sensors for detecting location information, temperature, humidity, illuminance, noise level, etc., as well as a database of dictionary information, etc., and extracts the necessary information upon request.

With this voice response system 100, it is possible to obtain information specific to the other terminal device 1, such as the location of the other terminal device 1. It is also possible to transmit information specific to the terminal device 1 to the other terminal device 1.

Sixth Embodiment
[Processing of the Sixth Embodiment]
Next, in the voice response system of the sixth embodiment, a personality DB 106 is prepared in which personality information is recorded in which the personalities of the user or related persons who are related to the user are associated with predetermined categories. For example, as shown in Fig. 14, the personality DB 106 records the names of the user and related persons in association with their personality categories.

In addition, in the personality DB 106 shown in FIG. 14, personality tests are administered to users and related parties, and the test results are also recorded. Here, when generating personality information, well-known personality analysis techniques (Rorschach test, Szondi test, etc.) may be used. When generating personality information, aptitude testing techniques used by companies and the like for employment examinations may also be used.

When generating personality information, for example, the personality information generation process shown in FIG. 15 is carried out. The personality information generation process is started when, for example, an instruction to generate personality information is input using the operation unit 70 or the like in the terminal device 1.

As shown in FIG. 15, in the personality information generation process, first, the microphone 37 is turned on (S242), and one of the predetermined multiple-choice questions is output by voice (S244). At this time, the multiple-choice question may be obtained from the server 90, or a question previously recorded in the memory 39 may be asked.

Next, it is determined whether or not there is a voice response from the subject (the user or a related person) (S246). If there is no response (S246: NO), the process of S246 is repeated.
If there is an answer (S246: YES), conversation parameters such as the ending of words and the conversation speed are extracted (S248), and it is determined whether the current question is the last question (S250). If it is not the last question (S250: NO), the next question is selected (S252), and the process returns to S242.

If it is the final question (S250: YES), a personality analysis is performed by answering a multiple-choice question (S254), and a personality analysis is performed using conversation parameters (S256). Here, personality analysis using conversation parameters can capture the tendency for confident people to use strong endings and unconfident people to use weak endings, as well as the tendency for impatient people to speak quickly and easygoing people to speak slowly, etc.

Next, these personality analysis results are analyzed comprehensively, such as by taking a weighted average (S258), and then assigned to a personality category (S260). In more detail, the subject's personality obtained by the test is converted into a score, and each score is assigned to a personality category.

The subject is then associated with the personality category (S262) and recorded in the personality DB 106 (S264). In other words, the relationship between the subject and the personality category is transmitted to the server 90. At this time, the test results are also transmitted to the server 90, and the server 90 constructs a personality DB 106 as shown in FIG. 14. When this process is completed, the personality information generation process is terminated.

When the personality DB 106 generated in this manner is used, the correspondence between personality categories and different responses is prepared in the response candidate DB 105. Then, in the process of S56, the server 90 obtains response candidates representing a plurality of different responses to the text information, selects a response to be output from the response candidates according to the personality information, and in the processes of S60 and S64, outputs the selected response.

[Effects of the sixth embodiment]
In the voice response system 100, the terminal device 1 generates personality information of a user or related person based on answers to a plurality of preset questions, and acquires the generated personality information.

According to such a voice response system 100, personality information can be generated in the server 90 or the terminal device 1.
Furthermore, in the voice response system 100, the calculation unit 101 generates personality information of the user or related person based on the character string included in the input text information.

According to the voice response system 100, the user can generate personality information while using the voice response system 100.
In addition, according to the voice response system 100, different responses can be made depending on the characteristics of the user and those related to the user (concerned persons), which makes the system more user-friendly.

[Modification of the sixth embodiment]
In the sixth embodiment, the response may be narrowed down to one according to the personality and then output, or a plurality of responses may be output in correspondence with voices of different tones.

Furthermore, the processes of S248 and S254 to S264 in the personality information generation process may be performed by the server 90. In this case, as in the first embodiment, etc., the server 90 may identify the terminal device 1, and voice and questions may be exchanged between the terminal device 1 and the server 90.

Furthermore, in the above voice response system 100, the server 90 may detect any of the user's actions and operations and generate learning information or personality information based on them.

For example, such a voice response system 100 can urge a user to leave the house a few minutes earlier from the next day if it detects that the user has jumped on a train for several consecutive days, or can output voice or music to calm the user down if it detects from conversation that the user has a tendency to get angry easily.

[Seventh embodiment]
[Processing of the Seventh Embodiment]
Next, in the voice response system of the seventh embodiment, a preference DB 108 is prepared in which preference information in which the preferences of the user and related parties are associated with each other according to a preset category is recorded. For example, as shown in Fig. 16, the preference DB 108 records the names of the user and related parties and their preferences in association with each of the preference types such as food preferences (food), color preferences (colors), hobbies, etc.

In particular, food preferences are classified as sweet tooth (sweet), spicy (spicy), or somewhere in between; color preferences are classified as warm colors (warm), cool colors (cold), or somewhere in between; and hobbies are classified as indoor hobbies (indoor), outdoor hobbies (outdoor), or both indoor and outdoor hobbies (indoor and outside).

When constructing such a preference DB 108, for example, a preference information generating process shown in Fig. 17 is executed. The preference information generating process is carried out, for example, between S48 and S54.
17, keywords related to preferences are extracted from the character information (S282), and objects identified by image processing are extracted that relate to preferences (S284). Note that, in the preference DB 108, keywords related to preferences are associated with types of preferences and classifications within those types (sweet, normal, spicy, etc. for food preferences), and in these processes, if the extracted keywords or objects are included in the preference DB 108, they are extracted as being related to preferences.

Next, a counter is incremented for each group of preference-related keywords (S288). For example, if a preference type of "food preference" and a type of "spicy" are extracted, such as kimchi, the counters corresponding to "food preference" and "spicy" are incremented.

Then, the preference information (preference DB 108) is updated based on the counter value (S290). In other words, for each "preference type," the "type" with the largest counter value is the one that best matches the preference, and is recorded in the preference DB 108 as a characteristic of the user's or related person's preferences. When this process is completed, the preference information generation process is terminated.

When the preference DB 108 generated in this manner is used, responses corresponding to different preferences are prepared in the response candidate DB 105, and the server 90 acquires response candidates representing a plurality of different responses to the text information in the process of S56, selects a response to be output from the response candidates according to the preference information, and outputs the selected response in the processes of S60 and S64.

[Effects of the Seventh Embodiment]
In the voice response system 100, the server 90 generates preference information indicating the tendency of preferences of the user or related person based on the character string included in the text information, selects a response to be output from the response candidates based on the preference information, and outputs the selected response.

With such a voice response system 100, it is possible to provide a response according to the preferences of the user or related parties. For example, when buying a present for a related party, the user can ask the terminal device 1, "What would Mr./Ms. XX want?" and receive a response according to the preference information.

[Modification of the Seventh Embodiment]
The reply candidate DB 105 may have a table in which personality categories are associated with preference information, as shown in FIG.

For example, in the example shown in FIG. 18, personality categories are associated with color preferences, and products that are estimated to please women as presents are arranged in a matrix.
In the process of S56, a response can be generated taking into account both personality and preferences in this way.

[Eighth embodiment]
[Processing of the Eighth Embodiment]
In the above embodiment, the voice is converted into text information, but the action of the user may be converted into text information.

In detail, the terminal device 1 captures the user's movements as a captured image and transmits it to the server 90, and the server 90 performs, for example, the movement character input process shown in FIG. 19. The movement character input process is a process that is started when a part of the user's body is shown in the captured image in the process of S48.

In the motion character input process, as shown in FIG. 19, first, a captured image is acquired (S302). Then, it is determined whether the user is attempting to input characters by handwriting or by sign language (S304, S308).

In these processes, for example, if the captured image shows the upper body of the user together with their face, it is determined that they are attempting to input characters using sign language, and if the captured image shows the user's hands but not their face, it is determined that they are attempting to input characters by handwriting.

If a character is being input by handwriting (S304: YES), the behavior of the fingertip or pen tip is recorded (S306), and the behavior is converted into character information based on this behavior (S312). Here, the handwritten character/sign language DB112 associates the behavior when writing a character by hand with the character, and also associates hand movements with characters expressed in sign language. In the process of S312, character information is generated by referencing the handwritten character/sign language DB112.

If the user is attempting to input characters using sign language (S304: NO, S308: YES), the handwritten characters/sign language DB 112 is referenced to recognize the sign language content, and the above-mentioned processing of S312 is carried out. If the user is not attempting to input characters using handwriting or sign language (S308: NO), processing for input using another method is carried out (S314).

Then, the character input by the action and the character input by the voice are associated with each other to determine whether there is a similar voice (whether the degree of match between the reference waveform based on the character and the pronunciation waveform is equal to or greater than a reference value) (S316). If such a voice input is present (S316: YES), the accent and pronunciation features used when the user inputs the character are associated with the character and recorded in the learning DB 107 (S318), and the action character input process is terminated.

If there is no such voice input (S316: NO), the action character input process ends.
[Effects of the Eighth Embodiment]
In the voice response system 100, since the actions of the user are converted into text information, the user can input text information without speaking.

[Modification of the eighth embodiment]
The movements in this embodiment may be not only handwritten characters or gestures (for example, sign language) but also movements resulting from muscle movements.

[Ninth embodiment]
[Processing of the ninth embodiment]
When the user uses a terminal device 1 other than the terminal device 1 that he or she normally uses, the contents of the learning DB 107 may be made available to the other terminal device 1. In this case, the other terminal device 1 transmits the ID and password of the terminal device 1 that he or she normally uses to the server 90 together with a request for use.

Then, the server 90 executes the other terminal use process shown in Fig. 20. The other terminal use process is started when a use request is received.
In the other terminal use process, as shown in Fig. 20, first, it is determined whether or not an ID and a password have been input (S332). If an ID and a password have not been input (S332: NO), the process of S332 is repeated.

If an ID and password have been input (S332: YES), it is determined whether authentication using the ID and password has been completed (S334). If authentication has been completed (S334: YES), a message indicating that authentication has been completed is transmitted to the other terminal device 1 (S336), and the other terminal device 1 is set to use the learning DB 107 of the terminal device 1 whose ID and password correspond (S338).

If the authentication is not completed (S334: NO), a message indicating an error is transmitted to the other terminal device 1 (S340), and the other terminal use process is terminated.
[Effects of the ninth embodiment]
In addition, in the voice response system 100, the server 90 transfers learning information of a certain terminal device 1 to other terminal devices 1.

According to such a voice response system 100, even when a user of a certain terminal device 1 uses another terminal device 1, the user can use the learning information recorded in the certain terminal device 1 (the learning information recorded in the server 90). Therefore, the accuracy of generating text information can be improved even when the other terminal device 1 is used. This is particularly effective when a user has multiple terminal devices 1.

Furthermore, in the voice response system 100, the server 90 outputs information about the user in response to an inquiry from a person other than the user.
According to such a voice response system 100, if the contents of meals of a user, the distance of a walk, etc. are detected, it is possible to answer questions at a hospital, etc. on behalf of the user. In addition, it is also possible to learn the health condition, self-introduction, etc.

[Modification of the ninth embodiment]
As in the configuration of the ninth embodiment, when a request to end use and an ID and password are received, use of the learning DB 107 for the terminal device 1 corresponding to the ID and password may be terminated (prohibited).

[Tenth embodiment]
[Processing of the Tenth Embodiment]
In the voice response system of the tenth embodiment, the server 90 stores the contents of the conversation and asks questions to obtain the same contents about the contents heard. In detail, in S100 of the automatic conversation server process shown in FIG. 7, the memory confirmation process shown in FIG. 21 is executed.

In the memory confirmation process, as shown in FIG. 21, past conversation contents are extracted from the learning DB 107 (S352), and a question is generated with an answer being a keyword contained in any of the conversation contents (S353). When this process is completed, the memory confirmation process is terminated.

In the memory verification process, questions such as "What was on the menu for dinner last night?" or "Where did you go three days ago?" may be asked.
[Effects of the Tenth Embodiment]
According to the voice response system 100, the memory of the user can be confirmed and the memory can be consolidated. It is also considered to be effective in preventing the progression of dementia in the elderly.

[Eleventh embodiment]
[Processing of the eleventh embodiment]
Next, the voice response system of the eleventh embodiment is configured so that a user can practice a foreign language by utilizing the terminal device 1 and the server 90.

In detail, pronunciation determination process 1 shown in FIG. 22, pronunciation determination process 2 shown in FIG. 23, and pronunciation determination process 3 shown in FIG. 24 are executed in that order. However, the server 90 executes one of the pronunciation determination processes 1 to 3 each time the voice response server process (FIG. 2) is executed. Furthermore, each of the pronunciation determination processes 1 to 3 is executed as the process of S56 described above.

First, in pronunciation determination process 1, as shown in FIG. 22, a response is generated to instruct the user to input a specific sentence by voice (S362). In this process, for example, a model sentence in a foreign language is generated, and the user is prompted to speak the model sentence by imitating it. When this process is completed, pronunciation determination process 1 is terminated.

Next, when speech is input in accordance with pronunciation determination process 1, pronunciation determination process 2 is carried out. In pronunciation determination process 2, as shown in FIG. 23, the accuracy of the pronunciation and accent is scored (S372). In this process, the speech is treated as a waveform, and the degree of match between the waveform and a model sentence is scored.

This score is then recorded in memory (S374), and pronunciation determination process 2 is terminated. Next, pronunciation determination process 3 is performed. In pronunciation determination process 3, as shown in FIG. 24, first, it is determined whether or not the score is less than a threshold value (S382).

If the score is less than the threshold (S382: YES), a response is generated to instruct the user to input a similar sentence again (S384). In this process, for example, a response is generated to prompt the user to speak by imitating the example again.

If the score is equal to or greater than the threshold (S382: NO), a response is generated to inform the user that the pronunciation was good and to prompt the user to input the next sentence (S386). For example, a response such as "Good pronunciation. Let's move on to the next sentence" is generated.

When this process is completed, the sound generation determination process 3 is completed.
[Effects of the eleventh embodiment]
In the voice response system 100, the server 90 detects the accuracy of the pronunciation and accent of the voice input by the user, and outputs the detected accuracy.

According to the voice response system 100, the accuracy of pronunciation and accent can be checked, which is effective when practicing a foreign language, for example.
Furthermore, in the voice response system 100, the server 90 outputs the same question again if the accuracy is equal to or lower than a certain value.

According to such a voice response system 100, it is possible to obtain an accurate answer by outputting the same question.
[Modification of the eleventh embodiment]
In the voice response system 100, when the accuracy is equal to or lower than a certain value, the server 90 may output a voice including a word that is closest to the pronunciation made by the user for confirmation.

According to such a voice response system 100, the user can check the accuracy of the pronunciation and accent.
[Twelfth embodiment]
[Processing of the twelfth embodiment]
Next, a description will be given of a voice response system according to a twelfth embodiment. In the voice response system according to the twelfth embodiment, the emotion of a user is detected from a voice input by the user, and a response that soothes the user according to the emotion is generated.

In detail, the emotion determination process shown in FIG. 25 and the emotion response generation process shown in FIG. 26 are executed. The emotion determination process is implemented as a detailed process of S50 described above, and as shown in FIG. 25, first, emotions are scored based on tone of voice, stress at the end of sentences, length of a sentence, speed of speech, unexpected words, etc. (S392), and then the emotions are classified according to the score and recorded in memory (S394).

When this process is completed, the emotion determination process is terminated. Next, in the process of S56 described above, the emotion response generation process is executed.
In detail, as shown in Fig. 26, first, the emotion category set in the emotion determination process is determined (S412). If the emotion category is normal (S412: normal), a normal greeting such as "Hello" is generated as a response (message) (S414).

Furthermore, if the emotion category is anger (S412: anger), a response that calms the other person's emotions, such as "Did that bother you?", is generated (S416). Furthermore, if the emotion category is joy (S412: joy), a greeting with a brighter nuance than normal greetings, such as "Today is a fun day, isn't it?", is generated as a response (S418).

If the emotion category is confusion (S412: confusion), a greeting that shows concern for the other person, such as "What's wrong?", is generated as a response (S420). When this process is completed, the emotion response generation process ends.

[Effects of the twelfth embodiment]
In the voice response system 100, the server 90 detects an unexpected voice to detect irritation or agitation of the user, and generates a message to suppress the irritation or agitation.

This voice response system 100 can suppress irritation or agitation in the user when the user is irritated or agitated. This can suppress the user from vocalizing issues that may cause trouble between the user and those around them.

[Thirteenth embodiment]
[Processing of the 13th embodiment]
Next, a voice response system according to a thirteenth embodiment will be described. In the voice response system according to the thirteenth embodiment, a process of guiding a user to an object in a captured image is performed. This process is performed in the server 90 as a detailed process of S56 described above.

When a user voice-inputs something like "Please show me the way to the tower I can see" into the terminal device 1, the guidance process is executed in the process of S56. In the guidance process, as shown in FIG. 27, first, the terminal location information is acquired from the GPS receiver 27 of the terminal device 1, etc. (S432).

Then, based on the voice (text information) and image processing, a target object is identified from among the objects in the captured image, and its position is identified (S434). In this process, the position of the object is identified in map information (which may be obtained from an external source or may be held by the server 90) based on the shape, relative position, etc. of the object. For example, if a tower is shown in the captured image, the tower is identified on the map based on the position of the terminal device 1 and the shape of the tower.

Then, a route to the object is searched for (S436), and route information is obtained (S438). This process can be realized using a process similar to that used in well-known cloud-based navigation devices.

Then, a response for providing route guidance is generated (S440). In this process, too, a response similar to that provided by the navigation device may be generated.
When this process is completed, the guidance process is terminated. When the guidance process is performed while the user is moving, the message can be reproduced using the automatic conversation server process, with the reproduction condition being that the user reaches the point to which the guidance should be given.

[Effects of the thirteenth embodiment]
In the voice response system 100, when text information is input, the server 90 generates a response according to a captured image of the surroundings of the voice response system 100, and outputs the response by voice.

With this voice response system 100, a response can be output as voice in response to the captured image. This improves usability compared to a configuration in which a response is generated from text information only.

In addition, in the voice response system 100, the server 90 searches for the object contained in the text information in the captured image by image processing, identifies the position of the searched object, and provides guidance to the location of the object.

According to such a voice response system 100, it is possible to guide the user to an object in a captured image.
Furthermore, in the voice response system 100, when the server 90 provides guidance to a destination, it acquires route information to the destination, such as weather, temperature, humidity, traffic information, road surface conditions, etc., and outputs the route information by voice.

According to such a voice response system 100, the status (route information) to the destination can be notified to the user by voice.
[Modification of the thirteenth embodiment]
In addition to the above configuration, text information may be input to respond as to what has been recognized, and what (who) has been recognized from the captured image may be output by voice.

Furthermore, in the above voice response system 100, instead of the process of S48, the server 90 may acquire a moving image capturing an image of the shape of the user's mouth when inputting text information by voice. In this case, instead of the process of S52, the voice may be converted into text information, and unclear parts of the voice may be estimated based on the moving image to correct the text information.

According to such a voice response system 100, the contents of the speech can be estimated from the shape of the mouth, so that unclear parts of the speech can be estimated well.
[Fourteenth embodiment]
[Processing of the 14th embodiment]
Next, a voice response system according to a fourteenth embodiment will be described. In the voice response system according to the fourteenth embodiment, a user is requested to perform a predetermined action, and it is determined whether the user performs the action as requested. In this configuration, in the automatic conversation terminal process shown in FIG. 6 and the automatic conversation server process shown in FIG. 7, a movement request process 1 shown in FIG. 28 and a movement request process 2 shown in FIG. 29 are sequentially performed as details of the process of S56 described above.

When the process of S54 ends first, movement request process 1 starts, and in movement request process 1, as shown in FIG. 28, a response (message) instructing the user to move their line of sight or head to a specified position is output (S452). When this process ends, movement request process 1 ends.

Next, when the process of S54 is completed, movement request process 2 is started, and in movement request process 2, as shown in FIG. 29, it is determined whether the line of sight and head position have moved as instructed (S462). In this process, the image captured by the camera is processed, and the user's movements are detected using the detection results from various sensors of the terminal device 1. When detecting the line of sight through image processing, well-known line of sight recognition technology can be used.

If the gaze or head has not moved as instructed (S462: NO), the response generated in S452 is output again (S464). If the gaze or head has moved as instructed (S462: YES), another arbitrary response is generated (S466).

When this processing is completed, the movement request processing 2 is completed.
[Effects of the fourteenth embodiment]
In the voice response system 100, the line of sight of the user is detected, and if the user does not move his/her line of sight to a predetermined position in response to a call, a voice is output requesting the user to move his/her line of sight to the predetermined position.

According to the voice response system 100, the user can be made to look at a specific location, and therefore can reliably check for safety when driving a vehicle.
In the voice response system 100, the server 90 observes the positions of body parts and facial expressions, and if there is little change in response to the call, outputs a voice requesting the user to change the position of a body part or facial expression.

With this voice response system 100, it is possible to move the position of a part of the user's body to a specific position or induce the user to make a specific facial expression. The present invention can be used when driving a vehicle or during a physical examination, etc.

[Fifteenth embodiment]
[Processing of the 15th embodiment]
Next, a voice response system according to a fifteenth embodiment will be described. In the voice response system according to the fifteenth embodiment, when a user inputs a broadcast program or a piece of music as voice, a process of complementing the broadcast program or the piece of music is performed if the broadcast program or the piece of music is interrupted.

In this configuration, the broadcast music complement process shown in FIG. 30 is carried out as a detailed example of S56 described above. As shown in FIG. 30, the broadcast music complement process first determines whether the broadcast program or music (or the song, if sung by the user) has been interrupted (S482).

If there is an interruption (S482: YES), the synchronized broadcast program or song is set as the response content in the process of S492 described below (S484), and the broadcast song supplement process ends. If there is no interruption (S482: NO), if a broadcast program is being watched, the broadcast program is acquired (S486), and if a song is being played, the corresponding song is acquired (S488).

Here, in the karaoke DB 116, music pieces and lyrics are recorded in association with each other, and when a music piece is acquired in this process, a music piece with lyrics is acquired.
Next, the broadcast program or music piece that the user is viewing is specified (S490). Then, this broadcast program or music piece is acquired and prepared for playback in synchronization with the broadcast program or music piece that the user is viewing (S492), and the broadcast music supplementation process is terminated.

[Effects of the fifteenth embodiment]
In the voice response system 100, the server 90 acquires a broadcast program similar to the broadcast program that the user is watching, and when the broadcast program is interrupted, the server 90 outputs the broadcast program that it acquired to complete the interrupted broadcast program.

According to the voice response system 100, the broadcast program being viewed by the user can be compensated for without interruption.
In addition, in the voice response system 100, when a user adds lyrics to a song without lyrics and sings it, the server 90 compares the song with lyrics with the lyrics added by the user and outputs the lyrics by voice in the part where only the lyrics of the user are not included.

According to such a voice response system 100, a user of a so-called karaoke machine can make up for a portion that the user cannot sing (a portion where the lyrics are interrupted).
[Sixteenth embodiment]
[Processing of the 16th embodiment]
Next, a voice response system according to a sixteenth embodiment will be described. In the voice response system according to the sixteenth embodiment, when a captured image contains characters and a user asks the terminal device 1 how to read the characters, the system acquires information about the characters from an external source and outputs the reading of the characters contained in the information by voice.

In this configuration, as a detailed example of S56 described above, the character explanation process shown in FIG. 31 is performed. In the character explanation process, as shown in FIG. 31, it is first determined whether or not a question about the reading, such as "how to read" has been received (S502). If a question about the reading has been received (S502: YES), the reading of the image-recognized character is searched for from other servers connected via the Internet network 85 (S504), the obtained reading is set in the response (S506), and the character explanation process ends.

If the question is not about the reading (S502: NO), it is determined whether or not a question about the "meaning of a word" such as that found in a Japanese dictionary has been received (S508). If a question about meaning has been received, the meaning of the image-recognized character (word) is searched for from other servers connected via the Internet network 85 (S510), the obtained meaning is set as the response (S512), and the character explanation process ends.

[Effects of the sixteenth embodiment]
According to such a voice response system 100, the reading of characters recognized by image recognition is searched for from other servers, etc., and the obtained reading is set in the response, so that the user can be informed of how to read characters, the meaning of words, etc.

[Seventeenth embodiment]
[Processing of the 17th embodiment]
Next, a voice response system according to a seventeenth embodiment will be described. In the voice response system according to the seventeenth embodiment, a server 90 detects abnormal behavior or a state of a user of the terminal device 1 based on a sensor value detected by the terminal device 1, and performs a process of reporting an abnormality when the abnormality is detected.

In detail, the terminal device 1 performs the action response terminal processing shown in FIG. 32, and the server 90 performs the action response server processing. In the action response terminal processing, as shown in FIG. 32, first, the outputs from the various sensors mounted on the terminal device 1 are acquired (S522), and an image captured by the camera 41 is acquired (S524). The acquired outputs from the various sensors and the captured image are then transmitted as packets to the server 90 (S526), and the action response terminal processing ends.

Next, in the behavior response server process, as shown in FIG. 33, first, the processes of S42 to S44 described above are carried out. Next, behavior such as wandering is identified based on the location information of the terminal device 1 (detection results by the GPS receiver 27) (S532), and the user's environment is detected based on detection results by the temperature sensors 15, 19, etc. (S534). Then, an abnormality is detected (S536).

In this process, an abnormality is detected based on changes in location information and the environment. For example, if the user is motionless in a hot or cold location, or if the user is in a location that the user does not usually visit, an abnormality is detected (S536). Alternatively, the location information and the environment are converted into a score, and if this score falls below a reference value (outside the reference range), an abnormality is determined.

Then, it is determined whether or not an abnormality has been detected (S538). If no abnormality has been detected (S538: NO), the action response server processing is terminated. If an abnormality has been detected (S538: YES), a message indicating that an abnormality has occurred is generated (S540) and a notification is sent to a specified contact point (S542). Then, the above-mentioned processing of S62 to S68 (excluding S66) is performed, and the action response server processing is terminated.

[Effects of the seventeenth embodiment]
In the voice response system 100, the server 90 detects the user's behavior and the user's surrounding environment, and generates a message in accordance with the detected behavior and surrounding environment.

This voice response system 100 can notify users of dangerous locations and restricted areas. It can also detect abnormal behavior by the user.

Furthermore, in the voice response system 100, the server 90 judges the health condition of the user based on a captured image of the user, and generates a message in accordance with the health condition.
According to such a voice response system 100, the health condition of the user can be managed.

In addition, in the voice response system 100, the server 90 notifies a predetermined contact point when the health condition falls below a reference value.
According to the voice response system 100, when the health condition of the user is below a reference value, a notification can be issued, so that an abnormality can be notified to others at an earlier stage.

[Other embodiments]
The present invention is not limited to the above-described embodiment, and various other forms may be adopted as long as they fall within the technical scope of the present invention.

For example, the voice response system 100 may mediate communication between two or more parties. In detail, when vehicles need to give way to each other at an intersection, the terminal devices 1 may negotiate with each other as to which vehicle will enter the intersection first. In this case, each terminal device 1 transmits information on the direction of travel when approaching the intersection and the approach speed to the intersection to the server 90, and the server 90 sets priorities for each terminal device 1 according to the direction of travel and the approach speed, and generates and outputs voice messages such as "Stop" or "Enter" according to the priority.

In addition, when the terminal device 1 accepts an incoming call (incoming call) for communication that requires a real-time response, such as voice communication, the call may be accepted only when it is convenient for the user. Specifically, when the camera 41 captures an image of the user's face, this may be regarded as a time convenient for the user, and the call may be accepted.

Furthermore, some people become annoyed when the other party does not respond when they call during voice communication, etc. In order to suppress such feelings, the situation of the other party may be communicated to the user who is waiting for a response from the other party. For example, the terminal device 1 may manage the user's schedule, and if the user does not respond to an incoming call, it may search for what the user is doing or free time in the user's schedule and inform the user when the user will be able to respond.

In addition, if the user does not respond to an incoming call, the user's location may be communicated to the caller. For example, if the user is connected to the Internet via a smartphone or computer, it is possible to know which device is being operated. It is conceivable that the user's location can be identified from this information and communicated to the caller.

Furthermore, whether or not the user is able to respond to an incoming call may be determined using location information using GPS or the like. Based on the location information, it can be determined whether or not the user is in a car, at home, etc. For example, if the user is on the move or in bed, it can be determined that the user is in a public place or asleep and is therefore unable to respond to an incoming call. In this way, if the user is unable to respond to an incoming call, it is conceivable to inform the caller of what the user is doing, as described above.

In addition, a configuration using security cameras is also conceivable for obtaining location information. In recent years, security cameras have been installed in various places, and the location of the user can be recognized by using these security cameras and a configuration for identifying the user, such as facial recognition. Security cameras can also be used to determine the situation, such as what the user is doing (whether or not the user is in a position to answer the phone). Whether or not the user is able to answer an incoming call can also be determined based on conditions such as whether or not the user is using a different landline phone (incoming calls cannot be answered while the landline phone is in use).

Furthermore, when a user of terminal device 1 wants to have a conversation with someone, the results of learning about the user's personality may be used to call up a terminal device that is estimated to have a good compatibility between the users among an unspecified number of people. In such a case, the device may be configured to talk to the user about a topic that is likely to generate excitement (a topic that both users are interested in (extracted using the learning results)).

Also, when the voice response unit is not used for a long time (when the user does not speak for a reference time or longer), the voice response unit may prompt the user with some kind of message.
At this time, location information such as GPS may be used to select the words to be spoken.

[Relationship between each means described in the claims or the summary for solving the problem (the present invention) and the configuration in the embodiment]
The terminal device 1 and the server 90 in the above embodiment correspond to an example of a voice response unit of the present invention. The processes in S2 and S56 in the above embodiment correspond to an example of a response acquisition unit of the present invention.

Furthermore, the processes of S28, S60, and S64 in the above embodiment correspond to an example of the voice output means of the present invention. Furthermore, the processes of S2 and S6 in the above embodiment correspond to an example of the voice input means of the present invention.

Furthermore, the process of S14 in the above embodiment corresponds to an example of a voice transmission means of the present invention. Furthermore, the response candidate DB 105 in the above embodiment corresponds to an example of a response recording means of the present invention.

Furthermore, the process of S56 in the above embodiment corresponds to an example of a personality information acquisition means of the present invention. Furthermore, the processes of S22 and S56 in the above embodiment correspond to an example of a response acquisition means of the present invention.

Furthermore, the processes of S28, S60, and S64 in the above embodiment correspond to an example of a voice output means of the present invention. Furthermore, the processes of S254, S258, and S260 in the above embodiment correspond to an example of a first personality information generating means and a second personality information generating means of the present invention. Furthermore, the process of S56 in the above embodiment corresponds to an example of a personality information acquiring means of the present invention.

Furthermore, the processes of S22 and S56 in the above embodiment correspond to the response acquisition means of the present invention. Furthermore, the processes of S28, S60, and S64 in the above embodiment correspond to an example of the voice output means of the present invention.

Furthermore, the processes of S254, S258, and S260 in the above embodiment correspond to an example of a first personality information generating means and a second personality information generating means of the present invention.
Furthermore, the processes of S48 and S56 in the above embodiment correspond to an example of a response generating means of the present invention, and the processes of S28, S60, and S64 in the above embodiment correspond to an example of a voice output means of the present invention.

Moreover, in the above embodiment, the process of S48 corresponds to an example of the voice input video acquisition means of the present invention. Also, the process of S52 corresponds to an example of the character information conversion means of the present invention.

Furthermore, the preference information generation process in the above embodiment corresponds to an example of a preference information generation means of the present invention. Furthermore, the process of S56 in the above embodiment corresponds to an example of a response candidate acquisition means of the present invention.

Furthermore, the action character input process in the above embodiment corresponds to an example of the character information generating means of the present invention. Furthermore, the other device information acquisition means, the other terminal usage process in the above embodiment corresponds to an example of the transfer means of the present invention.

Furthermore, the process of S98 in the above embodiment corresponds to an example of a playback condition determination means of the present invention. Furthermore, the process of S100 in the above embodiment corresponds to an example of a message playback means of the present invention.

Furthermore, the process of S116 in the above embodiment corresponds to an example of a no-answer sending means of the present invention. Furthermore, the process of S372 in the above embodiment corresponds to an example of a speech accuracy detection means of the present invention.

Furthermore, the process of S374 in the above embodiment corresponds to an example of the accuracy output means of the present invention. Furthermore, the process of S204 in the above embodiment corresponds to an example of the connection control means of the present invention.

Furthermore, the process of S50 in the above embodiment corresponds to an example of the emotion determination means of the present invention. Furthermore, the process of S438 in the above embodiment corresponds to an example of the route information acquisition means of the present invention.

Furthermore, the process of S462 in the above embodiment corresponds to an example of the gaze detection means of the present invention. Furthermore, the process of S464 in the above embodiment corresponds to an example of the gaze movement request transmission means of the present invention.

Furthermore, the process of S464 in the above embodiment corresponds to an example of a change request sending means of the present invention. Furthermore, the process of S486 in the above embodiment corresponds to an example of a broadcast program acquisition means of the present invention.

Furthermore, the process of S484 in the above embodiment corresponds to an example of a broadcast program complementing means and a lyrics adding means of the present invention. Furthermore, the processes of S504 and S506 in the above embodiment correspond to an example of a pronunciation output means of the present invention. Furthermore, the processes of S522 and S524 in the above embodiment correspond to an example of a behavioral environment detection means of the present invention.

Furthermore, the processing of S538 in the above embodiment corresponds to an example of a health condition determination means of the present invention. Furthermore, the processing of S540 in the above embodiment corresponds to an example of a health message generation means of the present invention.

Furthermore, the processing of S542 in the above embodiment corresponds to an example of the notification means of the present invention.

1...Terminal device, 10...Behavior sensor unit, 11...3-dimensional acceleration sensor, 13...Axis gyro sensor, 15...Temperature sensor, 17...Humidity sensor, 19...Temperature sensor, 21...Humidity sensor, 23...Illuminance sensor, 25...Wetness sensor, 27...GPS receiver, 29...Wind speed sensor, 33...Electrocardiogram sensor, 35...Heart sound sensor, 37...Microphone, 39...Memory, 41...Camera, 50...Communication unit, 53...Wireless telephone unit, 55...Contact memory, 60...Alarm unit, 61...Display, 63...Illumination, 65...Speaker, 70...Operation unit, 71...Touch pad, 73...Confirmation button, 75...Fingerprint sensor, 7 7...rescue request lever, 80...communication base station, 85...internet network, 90...server, 100...voice response system, 101...calculation unit, 102...voice recognition DB, 103...predictive conversion DB, 104...voice DB, 105...response candidate DB, 106...personality DB, 107...learning DB, 108...preference DB, 109...news DB, 110...weather DB, 111...playback condition DB, 112...handwritten characters/sign language DB, 113...terminal information DB, 114...emotion determination DB, 115...health determination DB, 116...karaoke DB, 117...report destination DB, 118...sales DB, 119...client DB.

Claims (7)

An information providing system including a request source device that is a request source of a request, a providing source device that is an information providing source, and a server capable of communicating with the request source device and the providing source device,
The requesting device
a designation request transmission unit configured to transmit to the server a designation request including character information and designating the providing device;
The server,
a provision request sending unit configured to select a type of data according to character information included in the designation request, and to send a provision request, which is a request for providing information on the type of data, to a providing source device designated in the designation request;
a response sending unit configured to obtain provision information from the provision source device in response to the provision request and to provide information to the request source device based on the provision information;
Equipped with
The providing device is
a determination unit configured to determine whether or not a setting is made to permit information provision; and a provision unit configured to transmit information to be provided in response to the provision request to the server when the information provision is permitted, and to refuse the information provision when the information provision is not permitted;
An information provision system comprising: The information providing system according to claim 1,
the function of the request sending device is activated by an operation via an operation unit or by voice, and the designation request is input by voice.
An information provision system configured as follows. The information providing system according to claim 1 or 2,
The requesting device is an information providing system mounted in a vehicle. The information providing system according to any one of claims 1 to 3,
The providing device is
Further comprising a sensor for detecting an environment of the source device;
The information providing system is configured such that the providing unit transmits data obtained from the sensor as the provided information to the server. A terminal device configured to be movable together with a user,
The terminal device is a request source device constituting the information providing system according to any one of claims 1 to 4.
A terminal device configured as follows. A terminal device configured to be movable together with a user,
The terminal device is a source device constituting the information providing system according to any one of claims 1 to 4.
A terminal device configured as follows. A server configured to be able to communicate with a request source device that is a request source and a providing source device that is an information providing source,
an information acquisition unit configured, when receiving from the request source device a designation request that includes character information and designates the providing device , to select a type of data according to the character information included in the designation request, and to transmit a request for information provision regarding the type of data to the designated providing device that is configured to determine whether or not the designated providing device is set to permit information provision, and to acquire provision information provided by the providing device if the information provision is permitted;
a providing unit configured to provide, to the requesting device, provision information in response to a request from the requesting device;
A server comprising:

Images