JP2009037050A - Dialogue device and dialogue program - Google Patents

Abstract

The present invention provides a technology for realizing an interactive apparatus that does not bore users while establishing a natural conversation.
A dialogue database 30 stores a plurality of topics and information associated with each topic. The voice feature amount extraction program 24 extracts the feature amount of the user's voice. The motion feature amount extraction program 26 extracts the feature amount of the user's motion. The degree-of-interest estimation program 28 is set in a state space having voice and motion feature values as variables, stores a plurality of ranked regions, and uses the extracted voice feature values and motion feature values as elements. It is specified to which region the feature vector to be assigned belongs. The dialogue control program 22 switches the topic to be selected when the rank of the identified area is lower than the threshold value. Since the guidance robot 2 switches the topic when the user's interest in the information output regarding the current topic is low, the user is not bored.
[Selection] Figure 1

Description

  The present invention relates to an interactive apparatus for interacting with a user and an interactive program for causing a computer to function as the interactive apparatus.

An interactive device that responds to a user by outputting predetermined information from a speaker in response to a voice uttered by the user has been developed. Such an interactive apparatus is applied to, for example, a guide robot that explains exhibits at an exhibition hall. It is desirable that such an interactive device can respond as naturally as possible without asking each user a topic desired by the user or forcing the user to operate a button. If you are not explicitly asking what the topic you are interested in, or if you do not force a button operation, it is necessary to select a topic and provide information so that the user is not bored.
Patent Document 1 discloses an apparatus that provides information without boring the user by extracting a keyword included in a voice uttered by the user and outputting information related to the extracted keyword. .
The device described in Patent Literature 1 includes a plurality of topic control units that perform dialogue processing on different topics. This device extracts a keyword from the voice uttered by the user, calculates the similarity between the extracted keyword and the topic provided by each topic control unit, and provides the topic for which the highest similarity is calculated A control unit is selected, and interactive processing is executed.

JP 2005-301017 A

The technique of Patent Document 1 can provide information on a new topic so as not to bore the user if a keyword can be extracted from the voice uttered by the user, but does not function when the keyword cannot be extracted. In the technique of Patent Document 1, it is necessary to select a word having a semantic content related to a topic so that a user can estimate a topic of interest. However, it is difficult to assume that a user suddenly issues a keyword having a semantic content related to a topic different from the topic provided by the device so far, and the possibility that the next topic can be estimated from the voice emitted by the user is not Not expensive. If keywords related to a new topic cannot be extracted, the current topic will be continued, and the user will be bored. For example, abruptly outputting a question that induces a user to issue a keyword related to a topic of interest, such as “Please tell me a word of interest”, makes the conversation unnatural.
The present invention has been made in view of the above problems, and an object of the present invention is to realize an interactive apparatus that does not bore users while establishing a natural conversation as much as possible.

In the technique of Patent Document 1, in order to actively estimate a topic of interest to the user, a keyword to be selected must be limited to a word indicating meaning content related to a specific topic.
The inventors have thought that it is possible to realize an interactive device that does not bore the user by estimating the degree of interest of the user with respect to information provided by the device and switching the topic when the interest is low. In general, in a dialogue between people, it is known that the degree of interest of the speaker with respect to the current topic is indicated in so-called non-linguistic information such as the motion and expression of the speaker. For example, if a speaker utters `` Is that so? '', If the speaker utters while looking straight at the other party, it can be estimated that the speaker is very interested in the current topic, and utters while facing away. In this case, it can be estimated that the speaker is less interested in the topic. The word “yes” does not include semantic content related to a specific topic, but the nonverbal information that the speaker speaks with the word indicates the degree of interest of the speaker regarding the current topic. It is.
The present invention estimates the degree of interest of the user with respect to information output from the device based on the voice uttered by the user and the operation of the user, and switches the topic when the interest is low, thereby preventing the user from being bored. To realize.
Unlike the technique of Patent Document 1, the present invention does not actively estimate a topic that the user is interested in, and therefore it is not necessary to limit the keyword to a keyword having meaning content related to a specific topic. The dialogue apparatus according to the present invention also reacts to a keyword included in a voice uttered by a user, which is a keyword that is often issued in a normal dialogue but cannot be associated only with a specific topic. Since the topic is switched when the interest is estimated to be low, an unnatural question is not output. Do not bore users while conducting more natural conversations.

  The present invention is a device that interacts with a user, and includes a storage device, a control unit, a microphone, a camera, an audio feature amount extraction unit, an operation feature amount extraction unit, and a state estimation unit. The storage device stores a plurality of topics and information associated with each topic. The control means selects one topic from a plurality of topics stored in the storage device, and outputs information (for example, voice data) associated with the selected topic to the user. The microphone acquires the voice uttered by the user. The camera photographs the user. The voice feature amount extraction unit extracts the feature amount of the user's voice from the voice acquired by the microphone. The motion feature amount extraction means extracts a feature amount of the user's motion from the user's image captured by the camera. The state estimating means stores data of a plurality of areas set in a state space having the voice feature quantity and the motion feature quantity as variables, and ranked in relation to each other. In addition, the state estimation means specifies which of a plurality of regions set in the state space the feature amount vector having the extracted speech feature amount and motion feature amount as elements. The control unit switches the topic to be selected when the rank of the area specified by the state estimation unit is lower than a threshold value. Here, “a plurality of regions ranked mutually” is a region ranked according to the level of interest of the user. Specifically, the region that is estimated to have the highest degree of interest of the user is assigned the highest rank, and the region that is estimated to have the next highest interest is assigned the second rank. The region that is estimated to have the lowest interest is assigned the lowest rank. The specific range of each area and the ranking of each area are specified in advance by experiments or the like.

  The interactive apparatus according to the present invention can estimate the user's interest level in a region where a feature vector is located in a state space in which a voice feature and a motion feature are variables. When the ranking of the region where the feature vector is located is higher than the threshold (when the user's interest is estimated to be high), information on the currently provided topic is continuously output. On the other hand, when the ranking of the region where the feature vector is located is lower than the threshold (when it is estimated that the user's interest is low), the topic is switched. When the feature vector representing the degree of interest in the output information belongs to an area with a lower ranking than the threshold, information on a new topic is provided to the user, so that the user is not bored. When the feature vector representing the degree of interest of the user with respect to a new topic belongs to an area whose ranking is lower than the threshold value, information related to another topic is provided. By switching topics one after another when the user's interest is low, the user is not bored while performing a natural conversation. The information stored in the storage device is specifically audio data of content related to a topic. For example, when the topic is an exhibit, it is audio data describing the exhibit.

  The feature quantity vector includes the extracted speech feature quantity and motion feature quantity as elements. Even for sounds having the same feature amount (for example, in the case of the same word), the degree of interest of the user often varies depending on the user's action. For example, if the user utters “Yes” to the information output by the device, if the user utters it, it is estimated that the information output by the device is highly interested Can do. On the other hand, if the user says “Is that?” While looking away, it can be estimated that attention is distracted and interest is low with respect to the information output by the device. In the above state estimation means, the degree of interest of the user with respect to the information output by the apparatus can be well estimated by calculating the degree of interest based on the voice and the feature amount of the action.

Each topic may be associated with a plurality of pieces of information (specifically, audio data). In this case, the state estimation means includes a plurality of feature vectors acquired between the time when the control means ends output of one information and the time when output of the next information starts and ends. The likelihood for each region is calculated and stored, and when the output of the next one piece of information is completed, the likelihood average of a plurality of feature vectors for that region is calculated for each region in the state space, and the largest It is preferable to identify a region having a likelihood average.
A plurality of feature quantity vectors are obtained from a user's reaction (speech and action uttered by the user) to one information (the above-mentioned "next one information"), and the user is comprehensively based on the plurality of feature quantity vectors. The degree of interest can be estimated.
Further, when the output of one information is completed, that is, when the output of the next information is started, it is determined whether the current topic is to be continued or switched, so that a more natural dialogue is realized. .

The feature amount of the motion may be a temporal variation amount of the user's head movement including at least one of the fluctuation range and the fluctuation direction of the user's head movement. For example, the smaller the fluctuation range of the user's head movement, the higher the feature vector is located in a region showing a high degree of interest. This is because when the fluctuation range of the movement of the user's head is small, the user's attention is concentrated on the information output from the apparatus, and it can be estimated that the user is highly interested. In addition, the feature vector is located in a region that shows a lower degree of interest as the direction of movement of the user's head movement is closer to the horizontal direction. This is because it can be estimated that the closer the fluctuation direction of the movement of the user's head is to the horizontal direction, the less the user is looking away and less interest in the information output by the apparatus.
Alternatively, the feature amount of the action may be a feature amount that specifies the facial expression of the user. This is because if the user can be identified as smiling, the user is satisfied with the information output by the device, that is, it can be estimated that the user is highly interested in the information output by the device.
The dialogue apparatus of the present invention switches the topic to be selected when the rank of the area specified by the state estimation means is lower than the threshold value. When the rank of the identified area is higher than the threshold value, the output of information relating to the currently selected topic is continued.

  The voice feature amount is voice information such as the volume, pitch, energy, and frequency of the voice uttered by the user. Here, the feature amount of speech includes not only an element for specifying a word but also an element corresponding to a so-called phoneme of speech. In particular, the present invention focuses on elements corresponding to phonetic phonemes. In that sense, the feature value of the voice here can be translated into the feature value of the phoneme included in the voice.

More specifically, the phoneme feature amount can be expressed by the fluctuation range of the volume of the user's voice or the fluctuation range of the sound quality. Therefore, it is preferable that the voice feature amount includes at least one of the fluctuation range of the sound volume of the user and the fluctuation range of the sound quality.
The phoneme feature quantity represents the degree of interest and interest of the user with respect to the output information in the same manner as the action feature quantity. For example, the word “Is that?” Was issued with a large fluctuation range of the inflection (large fluctuation range of the volume) and with a small fluctuation range of the intonation (small fluctuation range of the volume) In some cases, the user's interest in the information output by the apparatus is different. The degree of interest of the user with respect to the information output from the apparatus can be accurately estimated from the feature amount vector having the extracted feature amount of the motion and the feature amount of the voice as elements.

It is preferable that the audio feature amount includes the maximum value of the rate of change of the sound volume of the user over time. The maximum rate of change over time of the volume of the sound produced by the user is also greater when the user is interested than when the user is not interested. Therefore, it is possible to accurately estimate the degree of interest of the user also by using the feature amount vector whose element is the maximum rate of change over time in the volume of the voice uttered by the user.
Note that the state estimation means may specify a region to which a feature quantity vector having a plurality of feature quantities of speech belongs. For example, a region to which a feature quantity vector having feature quantities of both the volume fluctuation range and the frequency fluctuation range as elements may be specified.

  The present invention embodies a computer that has a microphone, a camera, and a storage device that stores a plurality of topics and information associated with each topic as a dialogue device that interacts with a user. Can do. In this interactive program, a computer selects one topic from a plurality of topics stored in a storage device, and outputs audio associated with the selected topic to a user. From the above, voice feature quantity extracting means for extracting the feature quantity of the user's voice, action feature quantity extracting means for extracting the feature quantity of the user's motion from the user's image captured by the camera, and voice feature quantity And the data of multiple regions set in the state space with the motion feature variables as variables and ranked in relation to each other, and the extracted speech feature values and motion feature values are stored. It is made to function as state estimation means for specifying which of a plurality of regions set in the state space a feature quantity vector as an element belongs to. Further, the control means switches the topic to be selected when the rank of the area specified by the state estimation means is lower than the threshold value.

  When the interactive program of the present invention is used, the feature vector indicating the degree of interest of the user with respect to the information output by the computer is specified and specified as to which of the plurality of regions set in the state space belongs. If the rank of the selected area is higher than the threshold (when the user's interest is estimated to be high), information on the topic currently provided is continuously output, while the rank of the identified area Is lower than the threshold (when it is estimated that the user's interest is low), the topic is switched. By calculating the degree of interest based on the voice and the feature amount of the action, the degree of interest of the user with respect to the information output by the apparatus can be well estimated.

When a plurality of pieces of information are associated with each topic, the state estimating unit acquires the period from when the control unit finishes outputting one piece of information until it finishes outputting the next piece of information. The likelihood for each region in the state space of a plurality of feature quantity vectors is calculated and stored, and when the output of the “next one information” ends, a plurality of features for that region for each region in the state space It is preferable to calculate the likelihood average of the quantity vector and specify the region having the largest likelihood average.
A plurality of feature quantity vectors are obtained from a user's reaction (speech and action uttered by the user) to one information (the above-mentioned "next one information"), and the user is comprehensively based on the plurality of feature quantity vectors. The degree of interest can be estimated.
Further, when the output of one information is completed, that is, when the output of the next information is started, it is determined whether the current topic is to be continued or switched, so that a more natural dialogue is realized. .
Note that the feature amount of the user's voice and the feature amount of the action are as described above.

  In the present invention, the feature vector whose elements are the user's voice and the feature of the motion is lower than a threshold value among the plurality of regions set in the state space and mutually ranked. When it is determined that the position is in the ranking area, information on a new topic is provided to the user. Since the region in which the feature amount vector having the feature amount of the user's voice and the motion is located is specified, the degree of interest of the user with respect to the information output by the apparatus can be accurately estimated. Since the topic is switched according to the degree of interest estimated from the feature vector, the user is not bored while performing a natural conversation.

(First embodiment)
A first embodiment of the present invention will be described with reference to the drawings. The dialogue apparatus of the present embodiment is a guide robot 2 that guides visitors while having a dialogue at the exhibition hall. Specifically, the guidance robot 2 can move through the exhibition hall by wheels, and explains each exhibit while interacting with the visitors. Hereinafter, the visitors who are guided by the guidance robot 2 are referred to as “users”.
FIG. 1 is a block diagram illustrating a schematic configuration of the guide robot 2. As shown in FIG. 1, the guidance robot 2 includes a speaker 4, a microphone 6, a camera 8, and a control device 12. The control device 12 is connected to the speaker 4, the microphone 6, and the camera 8.
In addition, the guidance robot 2 includes a sensor for specifying its own position in the exhibition hall, a sensor for detecting the presence of a user, and the like (not shown). The guide robot 2 includes a moving wheel (not shown) in addition to the above, but the description of the moving wheel and its control is omitted. Also,

  The speaker 4 outputs the information stored in the dialogue database 30 as a sound in accordance with a command from the control device 12. The microphone 6 acquires the voice uttered by the user. The camera 8 photographs the user. The microphone 6 and the camera 8 may acquire the sound and the image only while the interactive process with the user is being executed, or may always acquire the sound and the image around the guidance robot 2.

  The control device 12 includes a CPU 14 and a storage device 20. Specifically, the storage device 20 is a ROM, a RAM, or a hard disk device, and includes a dialogue control program 22, an audio feature amount extraction program 24, an operation feature amount extraction program 26, and an interest level estimation program 28 (interest level index). Calculation means) and a dialogue database 30 are stored. Although not shown, the storage device 20 stores a main program for starting and stopping each of the above-described programs and controlling wheels for movement.

The dialogue database 30 is a database in which information on each topic is associated with a plurality of topics. The “topic” here specifically means each exhibition. The “topic information” is specifically text data describing the outline of each exhibit, text data describing the characteristics of each exhibit, and the like.
The dialogue database 30 also includes text data describing questions for asking the degree of interest of the user to the voice output from the guidance robot 2 (for example, “Did you not be interested in the exhibit you explained?” Etc.) are also included. Since these text data can be output as voice by the apparatus, it can also be referred to as “voice data”.

Next, programs executed by the CPU 14 of the control device 12 (various programs stored in the storage device 20) will be described with reference to the flowcharts of FIGS.
When a main program (not shown) of the guidance robot 2 detects a user (visitor) around it, the main program (visitor) identifies an exhibit near the user and activates the dialogue control program 22. Now, let the exhibit near the robot 2 be “exhibit A”.
The dialogue control program 22 selects “exhibit A” as a specific topic (one topic) from among a plurality of topics (exhibit types) stored in the dialogue database 30 (step S2). Next, the dialogue program 22 creates output data to be output as a voice to the user (step S4). “Output data” is information associated with the selected topic (exhibit A) (text data describing the description of the selected topic “exhibit A” stored in the dialogue database 30), or This is sentence data describing the above-mentioned question items. As will be described later, the dialogue program 22 continuously outputs information related to the current topic when the user is not interested in the current topic (exhibit A). The dialogue program 22 outputs “sentence data describing a question” after outputting some information on the current topic continuously.
Next, the dialogue control program 22 synthesizes the output data created in step S4 and outputs it from the speaker 4 (step S6).

  When the main program (not shown) activates the dialogue control program 22, it simultaneously activates the voice feature quantity extraction program 24, the action feature quantity extraction program 26, and the interest level estimation program 28. The voice feature quantity extraction program 24 acquires the voice uttered by the user in response to the voice output in step S6 from the microphone 6 (step S8), and the feature quantity of the voice uttered by the user from the acquired voice. Is extracted (step S12). Further, the motion feature amount extraction program 26 acquires a user image in response to the sound output in step S6 from the camera 8 (step S10), and acquires a feature amount of the user's motion from the acquired image (step S10). Step S14). In other words, the voice feature quantity extraction program 24 extracts the feature quantity of the voice uttered by the user within a predetermined time after the robot 2 outputs the information associated with the current topic by voice, and the motion feature quantity The extraction program 26 extracts a feature amount of the user's motion within a predetermined time after the robot 2 outputs information associated with the current topic as a voice. Hereinafter, the above-described predetermined time for extracting feature amounts of voice and action is referred to as “feature amount extraction time”.

  Here, the voice feature amount is a fluctuation range of the sound volume when the user issues a specific keyword within the feature amount extraction time. As the specific keyword, words that mean affirmation such as “Yes”, “I see”, “Yes” are selected in advance. The voice feature extraction program 24 extracts a specific keyword from the voice acquired from the microphone 6 within the feature extraction time. The voice feature amount extraction program 24 specifies the maximum volume value and the minimum volume value included in the extracted keyword voice. The audio feature quantity extraction program 24 extracts the difference between the maximum volume value and the minimum volume value (volume fluctuation range) as the audio feature quantity.

  The feature quantity of the motion is the fluctuation width in the vertical direction of the user's head within the feature quantity extraction time (the vertical movement width of the head). The motion feature amount extraction program 26 uses a user's image for each frame image (an image of each moving image obtained by the camera 8 over time is referred to as a frame image) from an image acquired from the camera 8 within the feature amount extraction time. Identify the head. As a method for identifying the head from the frame image, an appropriate algorithm is used from among generally known face recognition algorithms. The position of the head in each frame image is compared, and the difference between the highest head position and the lowest head position (the fluctuation range in the vertical direction of the head) is calculated and output. In other words, the motion feature amount extraction program 26 extracts the fluctuation range in the vertical direction of the user's head from the user's image captured by the camera 8 as the motion feature amount.

  Next, the degree-of-interest estimation program 28 generates a feature quantity vector having the extracted voice feature quantity and motion feature quantity as elements (step S16). The feature quantity vector is a two-dimensional vector whose elements are the fluctuation range of the volume that is the feature quantity of the sound and the fluctuation range of the head that is the feature quantity of the motion in the vertical direction. As will be described later, the feature vector is a three-dimensional vector or more including other audio feature values and motion feature values in addition to the volume fluctuation range and the vertical fluctuation range of the head. May be.

  The degree-of-interest estimation program 28 uses a two-dimensional plane (state space) whose state variables are the fluctuation range of the volume, which is a feature amount of speech, and the fluctuation range of the head, which is a feature amount of motion, as the degree of interest. An interest level map divided into a high level area, a medium level area, and a low level area is stored. The interest map will be described later. The interest level estimation program 28 specifies to which region of the interest level map the feature quantity vector generated in step S16 belongs (step S18).

If the specified interest level is not less than the middle level (medium level or high level) (step S20: YES), the process returns to step S4, and the next sentence data associated with the exhibit A is output. On the other hand, when the specified interest level is low (step S20: NO), the dialogue control program 22 has selected a plurality of topics (exhibit types) stored in the dialogue database 30 so far. Instead of the selected topic (exhibit A), another topic (exhibit other than exhibit A) is selected (step S22). That is, the dialogue control program 22 switches the topic selected from the dialogue database 30. Thereafter, the process returns to step S4. After the topic selected from the dialogue database 30 is switched in step S22, the output data created in step S4 is information associated with the newly selected topic (related to the selected new exhibit. Sentence data).
Thereafter, every time the dialogue control program 22 synthesizes and outputs the output data (step S6), the user's interest level is estimated. If the interest level is the medium level or higher, information on the same topic is obtained. If the level of interest is low, the topic is switched.

As described above, the guidance robot 2 outputs information associated with a specific topic to the user by voice, and estimates the degree of interest of the user for the output voice. Switch topics when the estimated interest level is low. The guidance robot 2 switches the provided topic when the user's interest in the currently provided topic is low. If the user's interest in the switched topic is low, the topic is further switched. The guidance robot 2 does not actively estimate the topic that the user is interested in, but switches the topic one after another when the degree of interest of the user with respect to the currently provided topic is low. By doing so, users are not bored.
Moreover, the guidance robot 2 does not use a keyword related only to a specific topic when estimating the user's degree of interest. The guidance robot 2 estimates the degree of interest of the user with respect to the current topic from the feature amount of the speech and the feature amount of the action uttered by the user within a predetermined time (within the feature extraction time) after outputting the sound. . Therefore, for example, the topic can be switched so as not to bore the user while establishing a natural conversation without suddenly outputting a question such as “Tell me about a topic you are interested in”.

Next, the audio feature amount, the motion feature amount, the interest level index, and the interest level map will be described in detail.
In this embodiment, when the user issues a specific keyword (a word meaning affirmation) within the feature amount extraction time, the fluctuation range of the volume of the keyword is extracted as the feature amount of the voice. Further, the fluctuation range in the vertical direction of the user's head within the feature amount extraction time is extracted as the feature amount of the operation. As a result of repeated studies, the inventors have found that in a conversation, if a word meaning affirmation is uttered with a large fluctuation range of the volume, the speaker has a strong tendency to show high interest in the conversation. It was. Similarly, when a speaker moves his / her head greatly up and down in a conversation, the speaker has a strong tendency to show high interest in the conversation. The inventors conducted an experiment in which many subjects talked about a specific topic, and the fluctuation range of the volume of a keyword (a word meaning affirmation) issued by the subject during the conversation and the fluctuation in the vertical direction of the subject's head The correlation between breadth and subject's level of interest was investigated. In the experiment, the level of interest of each subject was ascertained through interviews after the experiment. The correlation obtained as a result of the experiment is called an interest level map. The interest level map can represent a state space in which the variable range of the sound volume and the variable range of the head are the state variables as a space divided by region for each interest level. The interest level map is “a plurality of areas set in the state space with the voice feature quantity and the motion feature quantity as variables, and each area characterizes different levels of interest of the user. In other words, the data of a plurality of areas ranked in the order of the degree level.

An interest level map is shown in FIG. The vertical axis of the figure shows the magnitude of the fluctuation range of the volume, and the horizontal axis shows the fluctuation range of the head in the vertical direction. The area (a) in FIG. 4 is an area with a low level of interest, the area (b) is an area with an intermediate level of interest, and the area (c) indicates an area with a high level of interest. ing. In FIG. 4, a curve (d) indicates the boundary between the region (a) and the region (b), and a curve (e) indicates the boundary between the region (b) and the region (c).
As described above, the boundary (d) and the boundary (e) are set in advance based on the results of experiments on many subjects. The feature vector of the user is represented by a point in the interest level map shown in FIG.

FIG. 4 illustrates user feature vectors P1 to P4. As is clear from the feature quantity vectors P1 and P2 in FIG. 4, even if the fluctuation range of the volume is the same, the level of interest of the user may be different if the fluctuation range in the vertical direction of the head is different. Similarly, as is apparent from the feature amount vectors P3 and P4 in FIG. 4, even if the fluctuation range in the vertical direction of the head is the same, the level of interest of the user may be different if the fluctuation range of the volume is different. is there.
In this way, a state space is introduced in which the feature quantity of the user's voice and the feature quantity of the user's motion are used as state variables, and the region in which the user's feature quantity vector belongs is specified The user's level of interest can be well estimated.

  The interest level estimation program 28 stores the interest level map shown in FIG. The interest level estimation program 28 then calculates the fluctuation range of the volume (speech feature) extracted by the speech feature extraction program 24 and the vertical direction of the user's head extracted by the motion feature extraction program 26. A two-dimensional vector (feature quantity vector) having the fluctuation range (feature quantity of motion) as an element is used as an index indicating the degree of interest of the user. The dialogue control program 22 specifies to which region of the interest level map of FIG. 4 the obtained feature quantity vector belongs. When the obtained feature vector belongs to the region (a) in FIG. 4 (in other words, when the position of the obtained feature vector is lower than the threshold value represented by the boundary (d)), the dialogue Other exhibits (other topics) are selected from the database 24 (step S22 in FIG. 3).

In other words, the above processing of the interest level estimation program 28 for estimating the degree of interest of the user from the user's feature quantity vector and the interest level map can be expressed as follows.
The degree-of-interest estimation program 28 is a plurality of areas set in a state space in which the feature quantity of speech and the feature quantity of motion are variables, and each area characterizes a different level of interest of the user. The data of a plurality of areas ranked in order of the level of interest is stored. Note that the plurality of regions include the above-described region with a low interest level (region (a) in FIG. 4), a region with a medium interest level (region (b) in FIG. 4), and a region with a high interest level. (Region (c) in FIG. 4). These areas are set in advance by experiments with subjects.
The degree-of-interest estimation program 28 specifies to which of a plurality of regions in the state space the feature vector, whose elements are the extracted speech feature and motion feature.
Then, the dialogue control program 22 switches the topic to be selected when the rank of the region to which the feature quantity vector belongs is lower than the threshold value.
In this case, the interest level estimation program 28 corresponds to “state estimation means” recited in the claims.

(Second embodiment)
Next, a second embodiment of the present invention will be described. The dialogue apparatus of the second embodiment is a guidance robot that guides the exhibition hall in the same manner as the dialogue apparatus of the first embodiment. The interactive apparatus (guide robot) of the second embodiment differs from the interactive apparatus (guide robot 2) of the first embodiment in the process of specifying the interest level from the feature vector. More specifically, the process of the flowchart of FIG. 5 is executed instead of the process of the flowchart of FIG. In other words, the interactive apparatus of the second embodiment has the configuration shown in FIG. 1, and executes the processes shown in the flowcharts of FIGS. Hereinafter, processing different from the first embodiment (processing shown in the flowchart of FIG. 5) will be described. 5A and 5B show that the processing is connected to FIGS. 2A and 2B, respectively.

FIG. 6 is a schematic diagram of an interest level map included in the interest level estimation program 28 of the guidance robot of the second embodiment.
The interest map of the guidance robot of the second embodiment will be described. As described in the first embodiment, the inventors conducted an experiment in which many subjects talked about a specific topic, and the volume of a keyword (a word meaning affirmation) issued by the subject during the conversation varied. The width and the fluctuation range of the subject's head in the vertical direction were recorded. The level of interest of each subject was ascertained through interviews after the experiment. The white dots shown in FIG. 6 are feature amount vectors for each subject (two-dimensional vectors whose elements are the fluctuation range of the volume that is the feature amount of the sound and the fluctuation range of the head that is the feature amount of the motion). Represents. The tester's feature vector is referred to as a sample feature vector.
In this embodiment, the sample feature vectors are clustered in advance at the interest level of each sample feature vector heard. That is, a large number of sample feature amount vectors are grouped for each interest level. In FIG. 6, the group ω1 indicates a group with a high interest level, the group ω2 indicates a group with the interest level, and the group ω3 indicates a group with the low interest level. The guidance robot of this embodiment stores the sample feature vector in advance divided into three groups.

  The guidance robot according to the present embodiment generates a feature vector for the user (step S16 in FIG. 2), and then specifies which group the generated feature vector is closest to. At that time, the closest group is specified based on the likelihood between the calculated interest degree index and the sample interest degree index of each group. The interest level of the identified group is set as the interest level to which the feature vector of the user belongs. The “likelihood” is a statistical measure that mathematically represents the likelihood.

A process in which the guidance robot of the second embodiment specifies the level of interest of the user from the generated feature vector will be described with reference to FIGS. The guidance robot of this embodiment executes step S30 in FIG. 5 after step S16 in FIG. In step S16 of FIG. 2, a user feature vector is generated. A point P5 in FIG. 6 represents the feature vector P5 of the user generated in step S16.
In step S30, the likelihood between the feature vector P5 calculated in step S16 and each sample feature vector is calculated. Next, an average value of likelihood is calculated for each group (for each interest level) (step S32). For example, in FIG. 6, the average likelihood of the user's feature vector P5 and the sample feature vector belonging to the group ω1 is Pω1, and the likelihood between the feature vector P5 and the sample feature vector belonging to the group ω2. Is the average of Pω2, and the average likelihood of the feature vector P5 and the sample feature vector belonging to the group ω3 is Pω3. Next, the group (interest level) from which the largest likelihood average is calculated is specified (step S34). The “maximum likelihood average” in FIG. 6 corresponds to the likelihood average represented by a straight line with the shortest distance among the likelihood averages Pω1, Pω2, and Pω3 represented by straight lines. The interest level specified in this way is the interest level to which the feature vector of the user belongs.

When the specified interest level is equal to or higher than the medium level (medium level or high level) (step S36: YES), the process returns to step S4 in FIG. 2 to output the next sentence data associated with the exhibit A. On the other hand, when the specified interest level is low (step S36: NO), the topic (the type of the exhibit) selected so far from the plurality of topics (types of exhibits) stored in the dialogue database 30 ( Instead of the exhibit A), another topic (exhibit other than the exhibit A) is selected (step S38). That is, the conversation control program 22 switches the topic to be selected from the conversation database 30. Thereafter, the process returns to step S4 in FIG. After the topic selected from the dialogue database 30 is switched in step S38, the output data created in step S4 is information associated with the newly selected topic (related to the selected new exhibit. Sentence data).
Thereafter, every time the dialogue control program 22 synthesizes and outputs the output data (step S6), the user's interest level is estimated. If the interest level is the medium level or higher, information on the same topic is obtained. If the level of interest is low, the topic is switched.

The guidance robot of the second embodiment determines the interest level of the user's feature vector based on the likelihood of the sample feature vector and the user's feature vector whose relationship between the feature vector and the interest level is known. calculate. By using the likelihood, the user's level of interest can be estimated more accurately.
Note that the sample feature quantity vector grouped for each interest level is equivalent to representing a plurality of regions set in the state space with the speech feature quantity and the motion feature quantity as variables. Accordingly, the processing of the guidance robot of the second embodiment can be expressed as follows in the same manner as in the first embodiment.
The degree-of-interest estimation program 28 is a plurality of areas set in a state space in which the feature quantity of speech and the feature quantity of motion are variables, and each area characterizes a different level of interest of the user. The data of a plurality of regions ranked in order of the level of interest level (sample interest level index data in this embodiment) are stored. The plurality of areas are an area specified by a sample feature vector with a low level of interest (area with a low interest level) and an area specified by a sample feature vector with a medium level of interest. (Region with a medium interest level ranking) and a region (region with a high interest level ranking) specified by a sample feature vector having a high interest level. These areas are set in advance by experiments with subjects.

(Third embodiment)
Next, a third embodiment will be described. The dialogue apparatus of the third embodiment is a guidance robot that guides the exhibition hall in the same manner as the dialogue apparatus of the second embodiment. The interaction device (guide robot) of the third embodiment is different from the case of the interaction device (guide robot) of the second embodiment in the process of specifying the interest level from the feature vector. More specifically, the process of the flowchart of FIG. 7 is executed instead of the process of the flowchart of FIG. In other words, the interactive apparatus according to the third embodiment has the configuration shown in FIG. 1 and executes the processes shown in the flowcharts of FIGS. Hereinafter, processing different from that of the second embodiment (processing shown in the flowchart of FIG. 7) will be described. 7A and 7B show that the processing is connected to FIGS. 2A and 2B, respectively.

  In the guidance robot of this embodiment, a plurality of information (voice data group) associated with each of a plurality of topics is stored in the dialogue database 30 (see FIG. 1). One piece of information is composed of a series of sentence data groups including sentence data of at least one sentence related to an exhibit (topic) (in other words, content or voice data). Specifically, the information associated with a specific exhibit consists of three pieces of information, the first information is a text data group that explains the outline of the exhibit, and the second information is the information on the exhibit. It is a text data group explaining technical features, and the third information is a text data group explaining the application destination of the exhibit.

  The guidance robot of the third embodiment has the same interest level map as the interest level map of the guidance robot of the second embodiment described with reference to FIG. In order to simplify the description, description of each sample feature quantity vector is omitted. In this embodiment, as in FIG. 6, ω1 in FIG. 8 is a high interest level region ω1 indicating a region with the highest interest level, and ω2 is a medium interest level region ω2 indicating a region with a medium interest level. , Ω3 is expressed as a low interest level region ω3 indicating the region with the lowest interest level.

  The guide robot of this embodiment creates output data of one piece of information in step S4 of FIG. As will be described later, the guidance robot according to the present embodiment outputs the next one information (for example, the second information described above) after the output of the one information (for example, the first information described above) ends. The index (feature quantity vector) indicating the degree of interest of the user is continuously acquired from the start to the end of the output.

The guidance robot of this embodiment executes step S50 in FIG. 7 after step S16 in FIG. In step S16 of FIG. 2, a user feature vector is generated. Also in this embodiment, the point P5 in FIG. 8 is used as the user feature vector P5 calculated in step S16.
In step S50, the likelihood for each interest level (ω1, ω2, ω3) of the feature amount vector P5 calculated in step S16 is calculated. As described in the second embodiment, the likelihood of the feature vector P5 for the high interest level region ω1 (corresponding to the group ω1 in the second embodiment) is Pω51 (see FIG. 6). Similarly, the likelihood for the medium interest level region ω2 of the feature quantity vector P5 is Pω52, and the likelihood for the low level region ω3 is Pω53. The guidance robot stores the calculated likelihoods Pω51, Pω52, and Pω53 (step S52).

The guidance robot according to the present embodiment continues the process of acquiring the feature vector and calculating the likelihood for each region from the end of the output of one piece of information until the end of the output of the next piece of information. (Step S54: NO). That is, the likelihood for each region of a plurality of interest index (feature vector) acquired from the end of the output of one information until the end of the output of the next one information is calculated and stored. . For example, before the output of one piece of information is completed, a feature vector P5 is generated, and likelihoods Pω51, Pω52, and Pω53 for each interest level (ω1, ω2, ω3) of the feature vector P5 are calculated. Next, a feature quantity vector P6 is generated from the acquired user's voice and action feature quantity, and the likelihoods Pω61, Pω62, and Pω63 for each interest level are calculated and stored.
When the output of one piece of information is completed (step S54: YES), the likelihood average of a plurality of feature amount vectors P5 and P6 for each level (region) is calculated for each interest level (level region) (step S54: YES). Step S56). Next, an interest level (level region) having the largest likelihood average is specified (step S58). Next, it is determined whether or not the rank of the specified interest level (level region) (high level, medium level, or low level) is lower than the medium level (step S60). If the determination in step S60 is yes, another exhibit (other topic) is selected from the dialogue database (step S62). If the determination in step S60 is no, step S62 is skipped. Finally, the likelihood stored in step S52 is deleted (step S64), and the process returns to step S4 (see FIG. 2).

In this embodiment, a plurality of feature quantity vectors are acquired for one piece of information, and the interest level of the user is estimated based on the plurality of feature quantity vectors. Accordingly, it is possible to improve the accuracy of estimation of the user's interest level.
Also, when the output of one information is completed (when the determination in step S54 is YES), that is, when the output of the next information is started, it is determined whether to continue or switch the current topic. Therefore, a more natural dialogue is realized.

Specific examples of the present invention have been described in detail above, but these are merely examples and do not limit the scope of the claims. The technology described in the claims includes various modifications and changes of the specific examples illustrated above.
In the dialogue apparatus (guidance robot) of the embodiment, the feature quantity vector is represented by a two-dimensional vector having a volume fluctuation range (speech feature quantity) and a head fluctuation range (motion feature quantity) as elements. did. The feature quantity vector may be expressed as a multidimensional vector having other feature quantities of speech and other feature quantities of motion as elements. In this case, the state space using the voice and motion feature quantities as variables is a three-dimensional or more multidimensional space.
The other feature amount of the voice may be, for example, voice information such as pitch, energy, and frequency of the voice uttered by the user, MFCC (Mel-Frequency Cessential Coefficient), and the like. The MFCC is a coefficient obtained by performing discrete cosine transform processing on the spectrum of audio data acquired by the microphone 6.
Alternatively, the feature amount of speech may be the frequency at which a specific keyword is extracted within the feature amount extraction time. In this case, the feature vector is high because the user's interest in the current topic is high when there are many words that mean affirmation, such as “great”, “yes”, and “I see”. The feature vector is of low interest because the user is less interested in the current topic when it is specified that it is located in the area representing the degree of interest, and there are many words that mean negation, such as “Okay”. It is specified to be located in a region representing degrees.
The feature amount of the motion may be, for example, a fluctuation range of the user's eye movement or a variable (for example, a fluctuation range of the mouth size) specifying the user's facial expression.

  Also, during steps S20 and S22 in FIG. 3 or between steps S36 and S38 in FIG. 5, a question is given to the user as to whether or not to change the topic (for example, “Would you like to change the topic?”). When the user wishes to change the topic, it is also preferable to execute step S22 or step S38. If the determination in step S20 is NO, or if the determination in step S36 is NO, there is a high possibility that the user is bored with the current topic. In such a case, asking whether or not to change the topic does not impair the naturalness of the dialogue.

Further, the interest level map may be constructed by a learning algorithm from experimental data of a large number of subjects (a relationship between sample interest level indexes and their interest level). As the learning algorithm, for example, an algorithm such as a support vector machine, a k-nearest neighbor method, a decision tree method, a Bayesian network, or a neural network may be used.
Moreover, the interest level map corresponding to each interest level may be constructed individually. In this case, the likelihood of each feature level vector of the user for each interest level is calculated for each interest level map, and the interest level having the largest likelihood average is specified.

  In the above-described embodiment, the dialogue apparatus is a guidance robot that guides the exhibition hall. The interactive device is not limited to the robot that guides the exhibition hall, and may be embodied as a device that responds to visitors at a company reception, for example.

  The technical elements described in this specification or the drawings exhibit technical usefulness alone or in various combinations, and are not limited to the combinations described in the claims at the time of filing. In addition, the technology illustrated in the present specification or the drawings achieves a plurality of objects at the same time, and has technical utility by achieving one of the objects.

It is a block diagram which shows the structure of the dialogue apparatus (guide robot) of 1st Example. It is a flowchart figure of the process which a dialogue apparatus performs (1). It is a flowchart figure of the process which a dialogue apparatus performs (2). It is a figure explaining the state space (interesting degree map) which uses the feature quantity of an audio | voice and the feature-value of an action as a state variable. It is a flowchart figure of the process which the dialogue apparatus (guidance robot) of 2nd Example performs. It is a figure explaining the process which specifies the interest level by the interactive apparatus of 2nd Example. It is a flowchart figure of the process which the dialogue apparatus (guidance robot) of 3rd Example performs. It is a figure explaining the process which specifies the interest level by the dialogue apparatus of 3rd Example.

Explanation of symbols

2: Guide robot (dialogue device)
4: Speaker 6: Microphone 8: Camera 12: Control device 14: CPU
20: Storage device 22: Dialogue control program 24: Speech feature quantity extraction program 26: Motion feature quantity extraction program 28: Interest level estimation program 30: Dialogue database

Claims (8)

A device that interacts with a user,
A storage device storing a plurality of topics and information associated with each topic;
Control means for selecting one topic from a plurality of topics stored in the storage device, and outputting information associated with the selected topic to the user,
A microphone that captures the voice uttered by the user;
A camera that shoots the user,
A voice feature amount extraction means for extracting a feature amount of a user's voice from the voice acquired by the microphone;
Motion feature amount extraction means for extracting a feature amount of the user's motion from the user's image captured by the camera;
It stores the data of multiple regions that are set in the state space with the feature amount of speech and the feature amount of motion as variables, and that are ranked in relation to each other. State estimation means for specifying which of the plurality of regions set in the state space the feature amount vector having the feature amount of as an element, and
The said control means switches the topic selected when the order | rank of the area | region specified by the state estimation means is lower than a threshold value, The dialogue apparatus characterized by the above-mentioned. The storage device stores a plurality of information associated with each topic,
The state estimation means includes a plurality of feature vectors acquired between the time when the control means ends output of one information and the time when output of the next one information starts and ends. The likelihood for the region is calculated and stored, and when the output of the next one piece of information is finished, the likelihood average of a plurality of feature quantity vectors for the region is calculated for each region of the state space, and the largest likelihood is calculated. The interactive apparatus according to claim 1, wherein a region having a degree average is specified.   The interactive apparatus according to claim 1, wherein the feature amount of the action includes at least one of a fluctuation range and a fluctuation direction of the movement of the user's head.   4. The interactive apparatus according to claim 1, wherein the voice feature amount includes at least one of a fluctuation range of a sound volume of a user and a fluctuation range of a sound quality. A program that causes a computer having a microphone, a camera, and a storage device that stores a plurality of topics and information associated with each topic to function as an interactive device that interacts with a user.
A control means for selecting one topic from a plurality of topics stored in the storage device and outputting the information associated with the selected topic to the user by voice;
Voice feature extraction means for extracting the feature of the user's voice from the voice acquired by the microphone;
Motion feature amount extraction means for extracting a feature amount of the user's motion from the user's image captured by the camera;
It stores the data of multiple regions that are set in the state space with the feature amount of speech and the feature amount of motion as variables, and that are ranked in relation to each other. The feature quantity vector having the feature quantity as an element functions as state estimation means for specifying which of a plurality of areas set in the state space belongs,
An interactive program characterized in that the control means switches the topic to be selected when the rank of the area specified by the state estimation means is lower than a threshold value. The storage device stores a plurality of information associated with each topic,
The state estimation unit is configured to estimate a plurality of feature quantity vectors acquired between the end of the output of one piece of information and the end of the output of the next piece of information for each region of the state space. The degree of likelihood is calculated and stored, and when the output of the next one piece of information is finished, the likelihood average of a plurality of feature vectors for the area is calculated for each area of the state space, and the largest likelihood average is calculated. The interactive program according to claim 5, wherein the area is specified.   The interactive program according to claim 5 or 6, wherein the feature amount of the action includes at least one of a fluctuation range and a fluctuation direction of the movement of the user's head.   The interactive program according to any one of claims 5 to 7, wherein the voice feature amount includes at least one of a fluctuation range of sound volume and a fluctuation range of sound quality.

Images