JP2021117580A - Information processing equipment and programs - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide an information processing device and a program capable of providing a conversational sentence according to a situation of each user.
SOLUTION: In a process of acquiring event specific information that identifies a user's event associated with date and time information and speaking at a date and time after the date and time represented by the date and time information, a related conversational sentence related to the event specific information. It is an information processing device that generates.
[Selection diagram] Fig. 1

Description

The present invention relates to an information processing device and a program.

In recent years, various tools for managing user schedules have been developed. Among such tools, there is known a technique for registering and managing schedules based on predetermined keywords included in conversations from users (see, for example, Non-Patent Document 1 and the like).

"IPhone User Guide", [online], Apple Computer, [Searched on December 16, 1st year of Reiwa], Internet <URL: https://support.apple.com/ja-jp/guide/iphone/iph3d110f84/ios >

However, the above-mentioned conventional tools cannot be utilized such as providing conversational sentences related to the future schedule only by managing the future schedule while communicating with the user by conversational sentences. Insufficient information is provided according to the circumstances of each user.

The present invention has been made in view of the above circumstances, and one of its purposes is to provide an information processing device and a program capable of providing conversational sentences according to the circumstances of each user.

One aspect of the present invention that solves the problems of the above-mentioned conventional example is an information processing device, in which an acquisition means for acquiring event specific information that identifies a user's event associated with date and time information and the date and time information are used. In the process of uttering at a date and time after the represented date and time, it is provided with a conversation sentence generation means for generating a related conversation sentence related to the event specific information.

According to the present invention, it is possible to provide a conversational sentence according to the circumstances of each user.

It is a block diagram which shows the structural example of the information processing system which concerns on embodiment of this invention. It is a block diagram which shows the structural example of the terminal apparatus which concerns on embodiment of this invention. It is a functional block diagram which shows the example of the server which concerns on embodiment of this invention. It is explanatory drawing which shows the example of the conversational sentence queue used in the information processing system which concerns on embodiment of this invention. It is explanatory drawing which shows the content example of the action database used in the information processing system which concerns on embodiment of this invention. It is a functional block diagram which shows the example of the terminal apparatus which concerns on embodiment of this invention. It is explanatory drawing which shows the example of the setting information used in the information processing system which concerns on embodiment of this invention. It is a flow chart which shows the operation example of the information processing system which concerns on embodiment of this invention. It is another flow chart which shows the operation example of the information processing system which concerns on embodiment of this invention. It is a flowchart which shows the example of the process of selecting the conversational sentence of the information processing system which concerns on embodiment of this invention.

Embodiments of the present invention will be described with reference to the drawings. As illustrated in FIG. 1, the information processing system 1 according to the embodiment of the present invention includes a server 10 as an information processing device and a terminal communicably connected to the server 10 via a communication means such as a network. It is configured to include the device 20.

Here, as shown in FIG. 1, the server 10 includes a control unit 11, a storage unit 12, and a communication unit 13. Further, the terminal device 20 is a robot, and as illustrated in FIG. 2, includes at least a leg portion 21 and a main body portion 22, and the main body portion 22 includes a control unit 31, a storage unit 32, and a sensor unit 33. The display unit 34, the audio output unit 35, the communication unit 36, and the drive unit 37 are housed. Further, the leg portion 21 and the main body portion 22 are connected via an actuator that can rotate at least one axis, and the direction of the main body portion 22 can be rotated with respect to the leg portion 21.

The control unit 11 of the server 10 is a program control device such as a CPU, and operates according to a program stored in the storage unit 12. In the present embodiment, the control unit 11 receives request information from the terminal device 20. Further, the control unit 11 executes a process based on the received request information. For example, as one of the processes, the control unit 11 includes action information including an action instruction for instructing an action to be executed by the terminal device 20 and character string information representing the content of the voice uttered by the terminal device 20. Is transmitted to the terminal device 20 which is the source of the request information.

Further, in an example of the present embodiment, the control unit 11 acquires information representing an event such as "date" of the user (hereinafter, referred to as event specific information) associated with the date and time information representing the date and time. , In the process of executing the process of sending the voice uttered by the terminal device 20 to the terminal device 20 at a date and time after the date and time represented by the date and time information, the related conversational sentence related to the event specific information acquired above is transmitted. Generate. The details of the processing of the control unit 11 will be described later.

The storage unit 12 is a disk device or a memory device, and holds a program executed by the control unit 11. The storage unit 12 also operates as a work memory of the control unit 11. In an example of the present embodiment, the storage unit 12 may store information for generating an instruction to the terminal device 20. For example, the storage unit 12 holds a conversation sentence queue in which candidates for conversation sentences to be spoken by the terminal device 20 are accumulated. The contents of this conversational sentence queue will be described later.

The communication unit 13 is a network interface or the like, and sends various information to the terminal device 20 via the network according to an instruction input from the control unit 11. Further, the communication unit 13 outputs the information received via the network to the control unit 11.

The control unit 31 of the terminal device 20 is a program control device such as a CPU, and operates according to a program stored in the storage unit 32. In the present embodiment, the control unit 31 sends request information to the server 10 at a predetermined timing together with the device identification information unique to the terminal device 20, which will be described later.

In an example of the present embodiment, when the sensor unit 33 described later accepts the input of the user's voice, the control unit 31 of the terminal device 20 converts the input voice into character string information. For this process, a widely known voice recognition process can be used, and the control unit 31 sends, for example, voice information input to a voice recognition server that executes voice recognition processing, and receives the recognized character string information. This process may be executed by doing so.

Further, the control unit 31 sends request information to the server 10 when a voice is input by the user (trigger). This request information includes information for identifying a trigger (for example, information indicating that a voice has been input by the user) and information necessary for processing on the server 10, for example, here, a recognition result of the voice input by the user. Contains character string information.

That is, when the control unit 31 determines that a predetermined trigger has occurred, it collects information necessary for processing on the server 10, and together with the information for identifying the trigger, the server includes the request information including the collected information. It will be sent to 10. This trigger can be arbitrarily set, such as when a voice is input by the user as in the previous example, or when a predetermined time has come. The details of the operation of the control unit 31 will also be described later.

The storage unit 32 is a memory device or the like, and holds a program executed by the control unit 31. The storage unit 32 also operates as a work memory of the control unit 31.

In the present embodiment, the storage unit 32 may store setting information in which the trigger is associated with information that specifies information to be sent to the server 10. This setting information will be described later with a specific example. Further, the storage unit 32 stores device identification information uniquely set for the terminal device 20.

Further, in an example of the present embodiment, the storage unit 32 stores the image data of the animation to be displayed on the display unit 34. Specifically, the storage unit 32 contains image data of eye animation such as smiling eye image data, tear-flowing animated eye image data, etc., image data of a closed mouth in a laughing state, and a crying state. A plurality of image data of each part representing the facial expression of the character may be stored, such as the image data of the closed mouth of the character, the image data of the animation of the mouth during vocalization, and the image data of the animation of the mouth.

The sensor unit 33 includes at least a microphone that is a voice sensor. Further, the sensor unit 33 may include a touch sensor, an acceleration sensor, and the like. The sensor unit 33 outputs the audio signal detected by each sensor, information indicating the position touched by the user, acceleration information, and the like to the control unit 31.

The display unit 34 is a liquid crystal display or the like, and displays image data according to an instruction input from the control unit 31. In an example of this embodiment, the display unit 34 displays the facial expression of the character by using the image data of the eyes and the mouth. The voice output unit 35 is a speaker or the like, and sounds a voice according to a voice signal input from the control unit 31.

The communication unit 36 includes a network interface. The communication unit 36 transmits / receives information to / from the server 10 via a network wirelessly or by wire. Specifically, the communication unit 36 sends request information and the like to the server 10 according to the instruction input from the control unit 31. Further, the communication unit 36 outputs the information received from the server 10 to the control unit 31.

The drive unit 37 drives the actuator so as to rotate the main body 22 with respect to the leg 21 according to the instruction input from the control unit 31.

Next, the operation of the control unit 11 of the server 10 of the present embodiment will be described. In the present embodiment, the control unit 11 of the server 10 includes a reception unit 41, an event management unit 42, an event information acquisition unit 43, a conversation sentence generation unit 44, and a conversation history, as illustrated in FIG. It includes a management unit 45, an action information generation unit 46, and an instruction transmission unit 47.

The receiving unit 41 receives the request information and the device identification information from the terminal device 20. This request information includes information that identifies the cause (trigger) of the request for the action to be executed by the terminal device 20. The type of trigger will be described later, and an example thereof is voice input by a user. The request information based on the trigger that there is a voice input by the user may include information representing the content of the voice input by the user together with the information that identifies the trigger. Here, the information representing the content of the voice input by the user may be character string information which is the result of recognizing the voice.

The receiving unit 41 receives the information for specifying the trigger, the character string information representing the content of the voice input by the user, and the like included in the request information received here, as well as the event information acquisition unit 43 and the action information generation unit 46. Output to.

The event management unit 42 stores and manages event information for each user in the storage unit 12. Here, the event information is the schedule of the user, and the date and time information is associated with the event specific information that identifies the event.

The event information acquisition unit 43 receives the input of the character string information representing the content of the voice input by the user from the reception unit 41. Then, the event information acquisition unit 43 determines whether or not the received character string information is information related to the event. Specifically, the event information acquisition unit 43 of the present embodiment compares the character string information with a pattern of information related to a predetermined event.

Here, the pattern is the part that matches the predetermined date and time words (date and time related words) such as "tomorrow", "the day after tomorrow", and "next week", and the event such as "date", "work", and "beauty salon". Includes parts that match words related to (event-related words). This pattern can be represented, for example, by a regular expression string, for example, "[Tomorrow | The day after tomorrow | Weekend | Next week] [D + day] [W + day] [Sun] [ha | ni] W + [ga] [is | What] * ”is set.

Here, in the above example, "*" is a regular expression that matches any character including spaces, and "+" is a regular expression that matches the type of character specified by the character immediately before one or more characters. Yes, [X | Y | ...] is a regular expression that matches any of X, Y ... Further, [Z] means a pattern that may or may not exist, and "D" corresponds to a number and "W" corresponds to a character. Therefore, the above string is
"I have a date tomorrow,"
"Next Monday is the exam."
Matches character string information such as. And in the above pattern
"[Tomorrow | The day after tomorrow | Weekend | Next week] [D + day] [W + day] [Sun]"
Part is a date and time related word,
Of "W + [ga] [aru | what]", the part of "W +" (matching any character) at the beginning corresponds to each event-related word.

Further, this pattern is not limited to one, and may be plural. For example, in addition to the above pattern, a pattern such as "W + [ga] [is | what] * [tomorrow | the day after tomorrow | weekend | next week] [D + day] [W + day] [day]" may be set. .. This example matches an example that does not match the pattern illustrated above, such as "There is a live, tomorrow", and the event information acquisition unit 43 represents the content of the voice input by the user. When the character string information matches the pattern of the information related to the event (if there are a plurality of them, one of them) as in the above example, it is determined that the accepted character string information is the information related to the event.

That is, when the event information acquisition unit 43 determines that the character string information representing the content of the voice input by the user matches the pattern of the information related to the event as in the above example and is the information related to the event. , Extract the date and time information and the event specific information that specifies the event from the character string information. Specifically, when the character string information is "I have a date tomorrow", the event information acquisition unit 43 obtains the date and time information of the day following the date and time when the process is being executed.

Here, the date and time information is information representing the date and time when the event occurs, which is estimated based on the date and time related words found by the above pattern. As for the process of estimating the date and time information from the date and time related words, a widely known technique can be adopted, so detailed description here will be omitted.

For example, when the processing is performed on December 15, the event information acquisition unit 43 acquires the date and time information of December 16, which is "tomorrow" from that date. Further, as the event specific information for specifying the event, the event information acquisition unit 43 extracts "date" from the character string information. A widely known method can be adopted as the method for extracting the event specific information.

The event information acquisition unit 43 outputs the date and time information extracted here and the event specific information that identifies the event to the conversation sentence generation unit 44. Further, the event information acquisition unit 43 interrupts the process when the character string information representing the content of the voice input by the user does not match the pattern of the information related to the event (if there are a plurality of them, none of them).

The conversation sentence generation unit 44 generates a conversation sentence by receiving the date and time information output by the event information acquisition unit 43 and the event specific information. Specifically, the conversation sentence generation unit 44 determines a date and time (utterance period) after the date and time output by the event information acquisition unit 43.

The utterance period is determined, for example, as follows. In an example of this embodiment, the server 10 uses a predetermined number of days candidate such as "1 day", "2 days", and "1 week" as the number of days candidate information, and a utterance period determined for each number of days candidate. It is assumed that the storage unit 12 stores a number of days data table in which the number of days from the first day to the last day (the length of the utterance period) and the word that specifies the occurrence date of the event (called a date and time demonstrative) are associated with each other. ..

The conversation sentence generation unit 44 randomly selects and adds one of the number of days candidates included in the number of days data table to the date and time output by the event information acquisition unit 43, and determines the first day of the utterance period. Further, the conversation sentence generation unit 44 determines the last day of the utterance period by adding the utterance period length associated with the selected number of days candidate to the first day of the utterance period.

As an example, when the date and time output by the event information acquisition unit 43 is "December 15", "2 days" (assuming the associated utterance period length is "0 days") is selected from the candidate days. Then, the conversation sentence generation unit 44 sets the first day of the utterance period to "December 17" and the last day to "December 17".

Further, the conversation sentence generation unit 44 generates a conversation sentence by using the date and time demonstrative associated with the selected number of days candidate and the event specific information output by the event information acquisition unit 43. For example, the conversation sentence generation unit 44 is a candidate for a predetermined conversation sentence pattern.
"By the way, how was the <event specific information> of the <date and time demonstrative>?"
"Ah. The <date and time demonstrative> was <event specific information>, right?"
...
One of the candidates such as, etc. is randomly selected, and the date and time instruction and the event specific information are inserted into the selected conversation sentence pattern.

As a result, the conversation sentence generation unit 44 can be, for example,
"By the way, how was your date the day before yesterday?"
Generate a conversational sentence like.

The conversation sentence generation unit 44 associates the conversation sentence generated here with the utterance period determined above as an utterance condition, and stores the conversation sentence in the conversation sentence queue stored in the storage unit 12. That is, in the example here, the conversational sentence queue stored in the storage unit 12 of the server 10 is stored in association with the utterance condition (C) and the conversational sentence (D), as illustrated in FIG. Become.

In addition to the above-mentioned utterance period, the utterance conditions may include conditions related to the utterance time, conditions such as the temperature at the time of utterance, and the like. Furthermore, the utterance conditions are not always necessary and need not be set. When the utterance condition is not defined (when there is no utterance condition), the conversational sentence without the utterance condition is a conversational sentence that can be spoken at all times.

Further, in the present embodiment, in addition to the conversational sentence generated by the conversational sentence generation unit 44, the conversational sentence may be registered in advance in this conversational sentence queue. In this way, the pre-registered conversational sentence is, for example,
・ In relation to the utterance condition that the maximum temperature was 35 degrees or higher and the time was after 18:00, "It was hot today."
・ In relation to the utterance condition that the time is between 2:00 am and 4:00 am, "Why don't you go to bed soon?"
There can be examples such as.

The conversation history management unit 45 is uttered for each user by the content of the voice input by the user received by the reception unit 41 and the terminal device 20 of the user instructed by the action information generation unit 46 to be described later. Record the contents of the conversational sentences in sequence. That is, the conversation history management unit 45 records the history of conversations between the user and the terminal device 20.

The action information generation unit 46 determines an action to be executed by the terminal device 20 that has sent the request based on the information input from the reception unit 41, and the information (action instruction) for instructing the action and the execution of the action. Action information including the information required for the above (hereinafter referred to as parameter information) is generated and output to the instruction transmission unit 47.

In an example of this embodiment, the storage unit 12 of the server 10 contains information (T) for specifying a trigger and user as information for generating an instruction to the terminal device 20 as illustrated in FIG. Information to be compared with the information representing the content of the voice input by (V, hereinafter referred to as comparison character string information. However, this comparison character string information may not be necessary depending on the type of trigger) and action information generation. It is assumed that an action database including at least one record (R) in which information (S) representing a process executed by the server 10 is associated with each other is stored.

The action information generation unit 46 is executed by the server 10 for generating the comparison character string information (V, if any) and the action information associated with the information (T) that identifies the trigger input from the reception unit 41. Get information that represents the processing to be done.

Then, if the comparison character string information is acquired (if the comparison character string information is associated with the information that identifies the trigger), the action information generation unit 46 transfers the character string information output by the reception unit 41 and the comparison character string. Compare with information. Then, when the action information generation unit 46 determines that the character string information output by the reception unit 41 matches the comparison character string information, the action information generation unit 46 executes the process represented by the acquired information to generate the action information.

If the comparison character string information has not been acquired, the action information generation unit 46 executes the process represented by the acquired information to generate the action information.

As a specific example, here, in the action database, the information that identifies the trigger that "there was a voice input by the user" and the comparison character string information that should be compared with the information that represents the content of the voice input by the user. In addition to the character string information such as "* news [o | ha] * [not | tell | read aloud] *", "news character string information is obtained from a predetermined web server on the Internet, and the relevant information is obtained. It is assumed that a record associated with the information "Instruct to read out the character string information" is recorded.

It is assumed that this comparison character string information is also represented by a regular expression. Therefore, the above string is
"Tell me about today's news"
"Are there any news?"
It will match the character string information such as.

For example, the action information generation unit 46 specifies information that identifies a trigger that "there was a voice input by the user" from the reception unit 41, and "is there any news?" As information that represents the content of the voice input by the user. When the input with the character string information such as is accepted, the record containing the information specifying the trigger is searched from the action database.

Here, the action information generation unit 46 finds the above record from the action database, and compares the comparison character string information included in the record with the accepted character string information. In the above example, it is judged that the accepted character string information "Are there any news?" Matches the comparison character string information "* News [| ha] * [No | Tell | Read aloud] *". Therefore, the action information generation unit 46 obtains information including the above-mentioned record found in the search, which represents the process to be executed by the server 10, for example, "(step 1) news character string information, on a predetermined web on the Internet. Get from the server,
(Step 2) Generate an instruction to read out the character string information (Step 3) Generate an instruction to display the animation information to be played at the time of reading out ”, and execute the process according to this information.

That is, the action information generation unit 46 acquires news character string information from a predetermined web server on the Internet according to the read information. Further, the action information generation unit 46 should display the image data of the animation to be displayed at the start of execution of the action processing performed in parallel, the image data of the animation to be displayed during execution, and the image data of the animation to be displayed at the end of execution. Generates a display instruction of animation information including information for identifying each image data of animation (the file name of each image data may be used).

Then, in this example, the action information generation unit 46 generates action information including the action instruction and the parameter information and outputs the action information to the instruction transmission unit 47. Here, the action instruction includes an instruction to read out the character string information and an instruction to display the animation information. Further, the parameter information includes the acquired character string information and information for specifying the image data of the animation.

Further, here, the information representing the process to be executed by the server 10 may include the instruction of "selection of conversational sentence". When such an instruction is included, the action information generation unit 46 selects a conversational sentence by, for example, the following method according to the instruction.

The action information generation unit 46 extracts a conversation sentence that satisfies the utterance condition associated with the conversation sentence from the conversation sentences stored in the conversation sentence queue. Here, various information necessary for determining whether or not the utterance condition is satisfied, for example, information on the current date and time (date and time when processing is being executed), weather information, etc., is stored in an NTP (Network Time Protocol) server via a network. Or, it may be obtained from a predetermined web server.

The action information generation unit 46 randomly selects, for example, one of the conversational sentences extracted as satisfying the utterance condition. Further, the action information generation unit 46 reads out the conversation history of the user of the terminal device 20 that sent the request, which is recorded by the conversation history management unit 45. Then, the action information generation unit 46 determines whether or not the conversation becomes natural when the selected conversation sentence is spoken, following the currently recorded conversation history.

Whether or not this judgment is appropriate as a sentence following the currently recorded conversation history using, for example, a neural network obtained by machine learning the text of a conversation made between humans, that is, continuity in the conversation. It can be realized by letting you judge whether or not there is. Such a process is a process known as so-called Next Sentence Prediction (NSP). As a neural network for predicting the next sentence, for example, a model known as BERT (https://arxiv.org/pdf/1706.03762.pdf) can be used. The learning data for machine learning for predicting the next sentence is a pair of conversational sentences (referred to as a first conversational sentence and a second conversational sentence) and the first and second conversational sentences. It is possible to adopt widely known learning data such as a method of using information representing the continuity of the data and a machine learning processing method using the same.

In this way, if the selected conversation sentence is valid or not, that is, the continuity of the conversation is judged as the sentence following the conversation history currently recorded using the neural network, the output of the neural network is , The validity is expressed numerically. Therefore, the action information generation unit 46 determines that the conversation is natural (determines that there is continuity) when the numerical value exceeds a predetermined threshold value.

Then, when the action information generation unit 46 determines that there is continuity as described above, the action information generation unit 46 generates an action instruction including an instruction to read out the character string information and an instruction to display the animation information. Further, the action information generation unit 46 generates parameter information including the character string information of the selected conversation sentence and the information for specifying the image data of the animation, and uses the generated action instruction and parameter information as action information. Instruct the instruction transmission unit 47 to send the data to the terminal device 20.

The instruction transmission unit 47 transmits the action information generated by the action information generation unit 46 to the terminal device 20 which is the transmission source of the request information received by the reception unit 41.

Next, the operation of the control unit 31 of the terminal device 20 will be described. In the present embodiment, as illustrated in FIG. 6, the control unit 31 functionally includes a request sending unit 51, an action information receiving unit 52, a voice synthesis unit 53, and an action processing execution unit 54. It is composed.

When the request sending unit 51 determines that a predetermined trigger has occurred, it collects information necessary for processing on the server 10, and together with the information for identifying the trigger, the server includes the request information including the collected information. Send to 10. Specifically, here, triggers such as voice input by the user or a predetermined time are listed in advance, included in the setting information, and stored in the storage unit 32.

As an example, as illustrated in FIG. 7, the setting information is required for the generation condition (C) for each name (trigger name: N) and the processing on the server 10 in relation to the trigger, as illustrated in FIG. It is assumed that the information (P) for specifying the information, the interval time (T), and the like are listed and recorded in association with the information related to the trigger processing.

When the request sending unit 51 determines that the generation condition is satisfied by referring to this setting information, it is assumed that a trigger satisfying the generation condition has occurred, and it is necessary for processing on the server 10 in relation to the trigger. Refer to information (P).

Then, the request sending unit 51 collects information necessary for processing on the server 10 specified by the referenced information, and collects the collected information and information (which may be a trigger name) that identifies the triggered trigger. The included request information is sent to the server 10.

The action information receiving unit 52 receives the action information from the server 10 and outputs the received action information to the action processing execution unit 54.

The voice synthesis unit 53 synthesizes voice data based on the character string information input from the action processing execution unit 54, which will be described later. The voice synthesis unit 53 outputs the voice data obtained by synthesis to the action processing execution unit 54.

The action processing execution unit 54 extracts the action instruction and the parameter information from the action information sent by the server 10, and executes the process according to the action instruction. Specifically, as in the above example, an instruction to read out the acquired character string information, an action instruction including a display instruction of the animation information, the acquired character string information, and animation image data. A case where the action information receiving unit 52 receives the action information including the parameter information including the above will be described.

In this example, the action processing execution unit 54 outputs the acquired character string information to the voice synthesis unit 53 to acquire voice data. Further, the action processing execution unit 54 reads out the image data of the animation specified by the information included in the action information from the storage unit 22. Then, the action processing execution unit 54 outputs the voice data output by the voice synthesis unit 53 to the voice output unit 35 to ring the voice, and outputs the image data of the read animation to the display unit 34. Play the image data of the animation.

[motion]
The information processing system 1 of the present embodiment has the above configuration and operates as in the following example. In the following example, the storage unit 12 of the server 10 is stored as an action database in which information representing the processing executed by the server 10 for generating action information is associated with each trigger that is the cause of the action request. It is assumed that it has been done.

In the following example, as an example of the information contained in this action database,
-Information that identifies the trigger (T): The user is having a conversation-Process to be executed:
(Step 1) Select the character string information of the conversation sentence (Step 2) Generate an instruction to read out the character string information (Step 3) Generate an instruction to display the animation information to be played at the time of reading out. And.

Further, as setting information, the storage unit 32 of the terminal device 20 specifies the generation condition (C) and the information required for processing on the server 10 in relation to the trigger for each trigger, as illustrated in FIG. Information (P) to be stored is associated and stored.

In the following example, this setting information is
-Information that identifies the trigger (trigger name N): There was a voice input by the user-Occurrence condition (C): The user uttered a predetermined wake word-Information that identifies the information required for processing on the server 10 ( P):
Character string information of the content spoken by the user ...
Such information shall be included. Here, the wake word is a word such as "Hey listen" or "Get up" that should be recognized as the start of voice input when the user utters the word, and is predetermined. And. The terminal device 20 may remove the character string portion corresponding to this wake word from the character string information of the content uttered by the user required for processing on the server 10.

Hereinafter, the operation of the server 10 that holds the information of such settings and the terminal device 20 will be described with reference to FIGS. 8 and 9.

When the user utters to the terminal device 20, "Hey, listen. It's a date tomorrow ..." (S11 in FIG. 8), the terminal device 20 executes a process of recognizing the user's voice. (S12), the character string information corresponding to the voice spoken by the user is acquired. As described above, the voice recognition process may be performed by accessing the voice recognition process service via the network without performing the voice recognition process by the terminal device 20 itself.

The terminal device 20 refers to the setting information and checks whether any of the trigger generation conditions is satisfied (S13). Here, since the user is uttering the wake word "Hey, listen", it is assumed that a trigger "there was a voice input by the user" has occurred (S13: Yes), and the user follows the setting information. Collects the character string information of the content spoken by. If it is determined in step S13 that none of the trigger generation conditions is satisfied (S13: No), the terminal device 20 ends the process.

Here, since the content uttered by the user has already acquired the character string information of the content uttered by the user in step S12, the terminal device 20 has the character string information and information for identifying the generated trigger ( Request information including the trigger name “there was a voice input by the user”) is sent to the server 10 (S14).

The server 10 receives the request information from the terminal device 20. Then, the server 10 compares a predetermined pattern of information related to the event with the character string information included in the request information to determine whether or not the character string information included in the request information is information related to the event. Judgment is made by (S15).

Here, assuming that the pattern is "* [Tomorrow | The day after tomorrow | Weekend | Next week] [D + day] [W + day] [Sun] [ha | ni] W + [ga] [is | what] *", the above was accepted. Since the character string information is "Hey, listen. It's a date tomorrow ...", the server 10 determines that it matches the pattern. That is, this character string information is determined to be information related to the event (S15: Yes).

When it is determined that the information is not related to the event (S15: No), the server 10 executes another process, that is, a process obtained by referring to the action database.

Therefore, the server 10 extracts the date and time information and the event specific information from the character string information (S16). Since "tomorrow" is used here, the server 10 obtains date and time information of "December 16", which is the day following the date and time when this process is executed (for example, December 15). Further, the server 10 extracts "date" from the above character string information as event specific information.

Next, the server 10 generates a conversational sentence. First, the server 10 randomly acquires one number of days from the number of days candidates predetermined in the number of days data table as the utterance period of the conversation sentence (S17: determination of the utterance period). Here, it is assumed that "7 days" is acquired as a candidate for the number of days. Further, in the process of step S16, the server 10 has the utterance period length (here, “7 days”) recorded in the number of days data table in association with the acquired number of days candidate, and the date and time demonstrative (here, “during this period”). The utterance period is set to "from December 23 to December 30" by using the date and time information (December 16) extracted in step S16, the acquired number of days candidates, and the utterance period length. ".

In the process of determining the utterance period, one day is randomly determined from the number of days candidates predetermined in the number of days data table, but the present embodiment is not limited to this, and the server 10 is a step. A predetermined number of days associated with the event specific information acquired in S16 may be acquired as the utterance period.

In this example, for example, if the event specific information is a "date" (an event that can occur relatively frequently and the topic lifetime is considered to be relatively short), the relatively short number of days is "4 days". In the case of "travel" (an event that is considered to have a relatively long topical lifetime), a data table predetermined as "14 days" as a relatively long number of days is stored in the storage unit 12. The server 10 sets the speech period with reference to this data table.

In another example, the utterance period may be determined by referring to a data table in which a plurality of day number candidates are stored in association with each of the event specific information acquired in step S16.

In this example, if the event-specific information is "dating" (an event that can occur relatively frequently and is considered to have a relatively short topical lifetime), then "a candidate for the number of days with a relatively short number of days" "1 day, 2 days, 3 days, 4 days", and in the case of "travel" (event that is considered to have a relatively long topical lifetime), "1 day, 2 days" including relatively long days , ..., 13th, 14th ”and so on are stored in the storage unit 12.

Then, the server 10 randomly selects one number of days from the number of days candidates predetermined in the data table in association with the event specific information acquired in step S16, and sets the selected number of days as the utterance period.

In this example, when the event specified by the event specific information acquired in step S16 is "date", the utterance period of 1 to 4 days is set, and the event is specified by the event specific information acquired in step S16. When the event to be held is "travel", the utterance period of 1 to 14 days will be set.

The server 10 generates a conversation sentence by using the date and time demonstrative associated with the selected number of days candidate, the event specific information output by the event information acquisition unit 43, and the information of the candidate of the conversation sentence pattern determined in advance. (S18). Here, the server 10 selects one of a plurality of predetermined conversational sentence pattern candidates, and inserts the date and time demonstrative (“during this time”) and the event specific information (“date”). So, conversational sentences, for example,
"By the way, how was your date during this time?"
To generate a conversational sentence such as.

The server 10 associates the conversation sentence generated in the process of step S18 with the utterance period determined in the process of step S17 as a utterance condition, and stores the conversation sentence in the conversation sentence queue stored in the storage unit 12 (S19).

By this process, the conversation sentence queue is a preset conversation sentence.
(1) Speaking conditions: The maximum temperature was 35 degrees or higher, and the time was after 18:00. Conversation: "It was hot today."
(2) Speaking condition: The time is between 2:00 am and 4:00 am Conversational sentence "Why don't you go to bed soon?"
...
And so on
(N) Speaking conditions: During the utterance period from December 23 to December 30 Conversation: "By the way, how was your date during this time?"
Conversational sentences with are accumulated.

During this operation, the server 10 sequentially records the content of the voice input by the user and the content of the conversation sentence uttered by the terminal device 20 of the user, and generates the information of the conversation history. ing. Here, for example, when the following conversation is held with the user, the information of the conversation history is retained.

That is, on the server 10,
User: "Hey, listen. I'm going on a date tomorrow."
Utterance of terminal device 20: "Oh, that's right."
User: "How is the weather?"
Utterance of terminal device 20: "It looks like it will be fine tomorrow."
Information on the conversation history such as is recorded.

The server 10 may execute another process, that is, a process obtained by referring to the action database, in parallel with the processes of steps S16 to S19.

Then, for example, on December 24, when the user utters "Get up. Is there any news?" To the terminal device 20 (S21 in FIG. 9), the terminal device 20 recognizes the user's voice. (S22), the character string information corresponding to the voice spoken by the user is acquired.

Further, the terminal device 20 refers to the setting information and checks whether any of the trigger generation conditions is satisfied (S23). Here, since the user is uttering a wake word, it is assumed that a trigger "there was a voice input by the user" has occurred (S23: Yes), and the character string information of the content uttered by the user is determined according to the setting information. Collect "Get up. Any news?" Then, the terminal device 20 sends out the request information including the character string information and the information specifying the generated trigger (trigger name “there was a voice input by the user”) to the server 10 (S24).

In the server 10, the terminal device is provided with the comparison character string information "* news [| ha] * [not | tell | read aloud] *" associated with the trigger "there was a voice input by the user". It is determined whether or not the character string information “Get up. Is there any news?” Sent by 20 matches or not (S25).

Here, since the character string information "Get up. Is there any news?" Received from the terminal device 20 by the server 10 matches the comparison character string information, the server 10 receives the comparison character string information in step S25. It is determined that the character string information matches, and the process specified by the information associated with the trigger "there was a voice input by the user" and the comparison character string information is started.

For example, the server 10 acquires news character string information from a predetermined web server on the Internet, and sends it to the terminal device 20 as action information together with an instruction to read out the character string information (S26).

The terminal device 20 executes a process of reading out news information according to this instruction (S27). Here, for example, news such as "Warm days continue from last week, but cold waves are expected to hit at the end of the year" is read aloud.

The server 10 records the character string information sent and received to and from the terminal device 20 up to this point as a conversation history. In the above example,
User: "Get up. Any news?"
Utterance of terminal device 20: "'It's been a warm day since last week, but it's expected that a cold wave will hit at the end of the year.'"
The history of conversations such as is recorded.

In the processing up to this point, the server 10 includes in the predetermined event-related information pattern and the request information whether or not the character string information included in the request information is information related to the event. Judging by comparing with the character string information, the character string information of "Get up. Is there any news?" Is the pattern "* [Tomorrow | Tomorrow's day | Weekend | Next week] [D + day] [W + day] Since it does not match [day] [ha | ni] W + [ga] [is | what] * ", it is not regarded as information related to the event and the corresponding processing is not performed.

After that, when the user further utters to the terminal device 20 such as "Oh, it was certainly warm last week" (S28), the terminal device 20 acquires the character string information corresponding to the voice spoken by the user. .. Then, the terminal device 20 assumes that, for example, a trigger to the effect that "the user is having a conversation" has occurred, and according to the setting information, the character string information of the content spoken by the user "Oh. Certainly it was warm last week." Request information including information for identifying the generated trigger is sent to the server 10 (S29).

The server 10 compares whether or not the character string information included in the request information is information related to the event by comparing the predetermined pattern of the information related to the event with the character string information included in the request information. Although it is determined, since this character string information also does not match the above pattern, it is not regarded as information related to the event, and the corresponding processing is not performed.

On the other hand, the server 10 executes a process specified by the information associated with the information that identifies the trigger that "the user is having a conversation", selects the character string information of the conversation sentence, and selects the character. An instruction to utter the column information is sent to the terminal device 20 (S30).

That is, as illustrated in FIG. 10, the server 10 extracts a conversational sentence that satisfies the utterance condition associated with the conversational sentence from the conversational sentences stored in the conversational sentence queue in the process of step S30. (S41). Further, the server 10 randomly selects, for example, one of the unselected conversational sentences from the conversational sentences (S42).

On the other hand, the server 10 refers to the conversation history of the user of the terminal device 20 that sent the request, and naturally when the conversation sentence selected in step S42 is spoken following the currently recorded conversation history. It is determined whether or not the conversation will be successful (S43).

Specifically, the server 10 uses a BERT model that machine-learns the text of a conversation between humans to perform next sentence prediction processing, so that the selected conversation sentence follows the currently recorded conversation history. (The more reasonable the value, the larger the value) is obtained. Then, the server 10 determines that the conversation will be natural (with continuity) when the acquired value exceeds a predetermined threshold value, and will not be a natural conversation (without continuity) otherwise. Judge.

In step S43, when the server 10 determines that the conversation will be natural when the selected conversation sentence is spoken (S43: Yes), the server 10 includes an instruction to read out the character string information of the selected conversation sentence. Generate information. Then, the server 10 sends the action information to the terminal device 20 (S44).

If it is determined in step S43 that the conversation does not become a natural conversation when the selected conversation sentence is spoken (S43: No), the server 10 returns to the process S42 and continues the process. If there is no unselected conversation sentence in the process S42, a predetermined process such as generating action information including an instruction to read out the character string information of the predetermined conversation sentence and sending it to the terminal device 20 is performed. (S45: Execution of default processing).

Returning to FIG. 9, the terminal device 20 executes a process of reading out the character string information according to the instruction of the action information (S31).

As a specific example, in the process of step S43,
User: "Get up. Any news?"
Utterance of terminal device 20: "'It's been a warm day since last week, but it's expected that a cold wave will hit at the end of the year.'"
User: "Oh, it was warm last week."
Following the conversation history such as, as one of the conversation sentences that the utterance conditions are satisfied, it is judged whether or not the conversation sentence "By the way, how was the date during this time?" Is naturally continuous. Will be done.

If it is determined that the conversation is continuous, the server 10 instructs the terminal device 20 to utter this conversation sentence, so that the conversation as a whole is completed.
User: "Get up. Any news?"
Utterance of terminal device 20: "'It's been a warm day since last week, but it's expected that a cold wave will hit at the end of the year.'"
User: "Oh, it was warm last week."
Utterance of terminal device 20: "By the way, how was your date during this time?"
And so on.

As described above, in the present embodiment, the conversation about the past schedule appears, so that the conversation of the content corresponding to the user can be realized and the feeling of familiarity can be aroused.

[In the case of a single terminal device]
In the above description, the terminal device 20 recognizes the voice emitted by the user and sends the character string information obtained to the server 10. Then, the server 10 generates and provides the character string information (conversational sentence) that is the source of the voice data to be uttered by the terminal device 20.

However, in one aspect of the embodiment of the present invention, the terminal device 20 may function as an information processing device. In this case, the server 10 is not always necessary.

In this example, the control unit 31 of the terminal device 20 realizes the functions of the request transmission unit 51, the action information reception unit 52, the voice synthesis unit 53, and the action processing execution unit 54, and the reception unit 41 and the reception unit 41. It also operates as an event management unit 42, an event information acquisition unit 43, a conversation sentence generation unit 44, a conversation history management unit 45, and an action information generation unit 46.

Further, in this case, all the data stored in the storage unit 12 in the above description, such as the action database and the conversation sentence queue, are stored in the storage unit 32 of the terminal device 20.

Then, in this example, the request sending unit 51 outputs the request information to the receiving unit 41 realized by the control unit 31 itself. Further, the receiving unit 41 receives the output request information and performs processing. Further, the action information generation unit 46 outputs the generated action information to the action information receiving unit 52 realized by the control unit 31 itself.

The output and acceptance of this information may be performed via the network (communication unit 36) through a local loopback, or may be stored and read in the storage unit 32 without going through the network. ..

[Stop utterance]
Further, the terminal device 20 voices when the user performs a predetermined operation on the terminal device 20 during playback of voice data (including voice data synthesized based on character string information received from the server 10). The playback of the data may be interrupted.

This predetermined operation may be performed by touching the touch sensor, for example, when the sensor unit 33 includes the touch sensor. In this example, when the terminal device 20 detects that the user touches the touch sensor during the reproduction of the audio data, the reproduction of the audio data is interrupted.

In yet another example, this predetermined operation may be that the user utters a predetermined word. For example, when the terminal device 20 determines that the user has issued a word requesting to be quiet during the reproduction of the voice data, the terminal device 20 interrupts the reproduction of the voice data.

In yet another example, the information processing system 1 may be controlled so as not to utter a predetermined conversational sentence even when the user does not perform the operation.

Specifically, the sensor unit 33 of the terminal device 20 may include a motion sensor capable of detecting the number of people (since it is a widely known device, detailed description thereof will be omitted). Then, the information on the number of people detected by the motion sensor around the terminal device 20 may be used for selecting a conversational sentence.

In this example, when the server 10 or the terminal device 20 that executes the function as the conversation sentence generation unit 44 generates a conversation sentence based on the information output by the event information acquisition unit 43 and registers it in the conversation sentence queue, the utterance is spoken. In addition to the information on the utterance period, the condition includes the condition related to the information on the number of people detected by the terminal device 20. As an example, the condition relating to this number of people is that the number of people is "1" (singular).

When the server 10 selects a conversational sentence, the terminal device 20 sends the request information to be sent to the server 10 including the number of people information indicating the number of people detected in the surroundings.

Then, the server 10 or the terminal device 20 that executes the function as the action information generation unit 46 extracts the conversational sentence that satisfies the speech condition associated with the conversational sentence from the conversational sentences stored in the conversational sentence queue. At that time, select a conversational sentence that satisfies the conditions using this number of people information.

As in the above example, if the condition that the number of people is "1" is included as the utterance condition of the conversation sentence related to the past event information, when a plurality of people exist around the terminal device 20. Is in a state where conversational sentences related to past event information are not selected, so that the chance of leaking the user's private information can be reduced.

[Expression of emotion]
In an example of the present embodiment, the action information may further include information related to the emotion of the character displayed on the display unit 34 of the terminal device 20. This emotional information may be, for example, information that identifies joy (Joy), sadness (Sad), or the like. Further, the emotional information may include information (flat) indicating no emotion.

The terminal device 20 selects the image data of the eyes to be displayed on the display unit 34 with reference to the emotional information included in the received action information. As an example, when the emotional information is "Sad", the terminal device 20 selects and displays the image data of the animation of the eyes that shed tears from the image data of the eyes.

Further, in an example of the present embodiment, when the emotional information to be included in the action information is determined, the server 10 or the terminal device 20 that generates the character string information to be uttered is a word included in the character string information. Emotions may be determined based on. Such processing can be performed, for example, as processing for determining the polarity of a sentence represented by character string information based on a word included in character string information. Specifically, when the character string information of news is uttered, the character string information of the news is included in the action information together with the information for identifying the feeling of sadness when a word representing a tragic event is included.

[Delete information]
Further, in the present embodiment, the server 10 or the terminal device 20 that stores the conversational sentence queue is associated with the already elapsed utterance period (the finished utterance period) from the conversational sentence queue at predetermined timing intervals. You may delete the conversational sentence.

Further, among the information stored in the conversation sentence queue, the conversation sentence stored by the conversation sentence generation unit 44 may be deleted from the conversation sentence queue when the conversation sentence is uttered. This prevents the same conversational sentence from being played over and over again.

[Get from schedule information]
Further, in the description of the present embodiment so far, in the server 10 or the terminal device 20 that functions as the event information acquisition unit 43, the character string information representing the content of the voice input by the user matches the pattern of the information related to the event. When doing so, the date and time information and the event specific information were extracted from the character string information. However, this embodiment is not limited to this.

In an example of the present embodiment, the event information acquisition unit 43 may acquire the predetermined schedule information and extract the date and time information and the event specific information in place of the above processing or in addition to the above processing. good. Specifically, the server 10 or the terminal device 20 that functions as the event information acquisition unit 43 receives in advance from the user the access right to the web service in which the user registers the schedule information. Then, the terminal device 20 acquires the schedule information from the web service according to the setting.

When the schedule information acquired here includes date and time information and event specific information (for example, "date" or "work") that identifies an event in which the user is involved in the date and time, the event information acquisition unit 43 The date and time information and the event specific information are taken out and output to the conversation sentence generation unit 44.

According to this example, the conversation can be provided based on the user's past schedule, even if it does not appear in the conversation with the user.

[When taking a wide utterance period]
Further, in the explanation so far, the conversation sentence generation unit 44 has assumed that the entire conversation sentence is generated in advance and stored in the conversation sentence queue. Therefore, the conversation sentence generation unit 44 includes in advance relative demonstrative words (“yesterday”, “the day before yesterday”, etc.) up to the day when the past event, which is the subject of the conversation, occurs according to the utterance period.

However, the present embodiment is not limited to this, and the conversation sentence generation unit 44 generates a conversation sentence template showing only the position of the word representing the day when the past event occurred in the conversation sentence, and puts it in the conversation sentence queue. You may accumulate it. In this case, in the process of the action information generation unit 46 after the conversation sentence is selected, when the instruction to utter the selected conversation sentence is generated, the day when the target past event occurred from that point is set. You may generate a word to represent it.

In this example, the event information acquisition unit 43 causes the event management unit 42 to record the acquired event information and issue an event identifier unique to the event that identifies the recorded event.

Then, the conversation sentence generation unit 44 only positions the words representing the day when the past event occurred at the time of generating the conversation sentence related to the event information (including the date and time information and the event specific information) acquired by the event information acquisition unit 43. Generates a conversational sentence template that represents, associates it with the corresponding event identifier, and stores it in the conversational sentence queue. Here, the template of the conversation sentence includes a code to be replaced with a word that specifies the date and time, such as "How was the date of <date and time>?".

After that, when the action information generation unit 46 selects the conversational sentence associated with this event identifier as the target of speech, the action information generation unit 46 refers to the event identifier associated with the selected conversational sentence and causes an event. The event information (including at least the date and time information) corresponding to the event identifier is acquired from the management unit 42.

Then, when viewed from the current date and time, a word that specifies the date and time is generated, and the selected conversational sentence is replaced with a code that should be replaced with the word that specifies the date and time. This word is generated in advance based on the interval of days, if it is 1 day, "yesterday", if it is 2 days, "the day before yesterday", if it is 3 to 6 days, "during this time", or if it is 7 to 13 days. For example, if it is set in advance as "last week", etc., it can be obtained based on the difference (interval between days) between the current date and time and the date and time when the target event occurred.

Then, the action information generation unit 46 generates action information including an instruction to utter a conversational sentence including a word that specifies the date and time.

[Characteristics of Embodiment]
The present embodiment is also characterized by the following. That is, one aspect of the present embodiment is an information processing device, which is an acquisition means for acquiring event specific information such as "date", which identifies a user's event associated with date and time information, and the date and time information. In the process of speaking at a date and time after the date and time represented by, a conversation sentence generation means for generating a related conversation sentence which is a conversation sentence related to the event specific information such as "How was the date last week" is provided.

This information processing device may be implemented as a terminal device provided for each user, or may be implemented as a server that is communicably connected to the terminal device and realizes each of the above means.

According to this example, the information processing device provides a conversation sentence about the past schedule, and the conversation about the past event is realized, so that the conversation with more private content can be realized for the user and the feeling of familiarity can be aroused. ..

In addition, here, when the conversation sentence input from the user includes an event-related word such as "date" related to a predetermined event and a date-related word such as "next weekend" that specifies the date and time. The storage means for accumulating the event specific information related to the event-related word and the date and time information represented by the date-time-related word in association with each other is provided.

In this example, information about the event can be accumulated only by inputting it as a conversational sentence without the user registering it as schedule information or the like.

Further, the conversation sentence generation means records the related conversation sentence such as "How was your date last week?" In association with information indicating the period for uttering the related conversation sentence, and records the current date and time in the period. Is included, it is determined whether or not the conversational sentence input by the user and the related conversational sentence have continuity, and when it is determined that the conversational sentence has continuity, the utterance processing of the related conversational sentence is performed. It may include an execution means for executing the above.

According to this example, the conversation about the past schedule appears at the timing when it is judged to be continuous naturally, so that the conversation becomes natural.

Here, the execution means is machine-learned using learning data in which the first conversation sentence, the second conversation sentence, and the information representing the continuity of the first and second conversation sentences are associated with each other. It is also possible to determine whether or not the accepted conversational sentence and the related conversational sentence have continuity by using the neural network.

Further, while the related conversational sentence is being spoken, the user may stop the utterance when a predetermined input such as an operation of forcibly stopping the utterance is performed. This makes it possible to stop the utterance when the utterance is not appropriate.

Further, the reproduction device for uttering the generated related conversational sentence and a means for detecting a person located in the vicinity of the reproduction device are further included, and only when there is a single person in the vicinity of the reproduction device, the above-mentioned The playback device may be used to utter a related conversational sentence. As a result, it is possible to suppress utterances in inappropriate situations.

1 Information system, 10 servers, 11 control units, 12 storage units, 13 communication units, 20 terminal devices, 21 legs, 22 main units, 31 control units, 32 storage units, 33 sensor units, 34 display units, 35 voices Output unit, 36 communication unit, 37 drive unit, 41 receiver unit, 42 event management unit, 43 event information acquisition unit, 44 conversation sentence generation unit, 45 conversation history management unit, 46 action information generation unit, 47 instruction transmission unit, 51 Request sending unit, 52 action information receiving unit, 53 voice synthesis unit, 54 action processing execution unit.

Claims (7)

An acquisition method for acquiring event-specific information that identifies a user's event associated with date and time information,
A conversation sentence generation means for generating a related conversation sentence related to the event specific information in a process of uttering at a date and time after the date and time represented by the date and time information.
Information processing device equipped with. The information processing device according to claim 1.
When the conversation sentence input from the user includes an event-related word related to a predetermined event and a date-time-related word that specifies a date and time, the event-specific information related to the event-related word and the date and time are specified. An information processing device including a storage means for accommodating and accumulating the date and time information represented by a related word. The information processing device according to claim 1 or 2.
The conversational sentence generation means records the related conversational sentence in association with information indicating a period during which the related conversational sentence is spoken.
When the current date and time is included in the period, the conversational sentence input from the user and the execution means for executing the utterance processing of the related conversational sentence when the related conversational sentence has continuity. Information processing equipment including. The information processing device according to claim 3.
The execution means is a machine-learned neural network using learning data in which a first conversation sentence, a second conversation sentence, and information representing the continuity of the first and second conversation sentences are associated with each other. An information processing device that determines whether or not the input conversation sentence and the related conversation sentence have continuity by using. The information processing device according to any one of claims 1 to 4.
An information processing device that stops the utterance when the user makes a predetermined input while the related conversational sentence is being uttered. The information processing device according to any one of claims 1 to 5.
A playback device that utters the generated related conversational sentence, and
Further including means for detecting a person located in the vicinity of the playback device, including
An information processing device that causes the playback device to utter the related conversational sentence only when there is only one person located in the vicinity of the playback device. Computer,
An acquisition method for acquiring event-specific information that identifies a user's event associated with date and time information,
A conversation sentence generation means for generating a related conversation sentence related to the event specific information in a process of uttering at a date and time after the date and time represented by the date and time information.
A program that functions as.

Images