JP2020014085A - Operation assistance device, operation assistance method, and operation assistance system - Google Patents

Abstract

PROBLEM TO BE SOLVED: To facilitate remote control of an electric device which meets the demand of a person near the electric device. An operation assisting device 1 has a text information acquisition unit 131 that acquires text information created based on a voice message input to a voice processing device 4 having a microphone, and a building in which the voice message is issued. After the device identification unit 132 that identifies a plurality of installed electric devices and the text information acquisition unit 131 acquire the text information, one or more electric devices are selected from the plurality of electric devices specified by the device identification unit 132. Display control for displaying text information and one or more icons used for operating one or more electric devices selected by the selection unit 133 on the selection unit 133 and the communication terminal 3 associated with the voice processing device 4. It has a part 134 and. [Selection diagram] Fig. 2

Description

  The present invention relates to an operation assisting device, an operation assisting method, and an operation assisting system for assisting an operation of an electric device.

  2. Description of the Related Art Conventionally, a system for displaying a message for remotely controlling an electric device on a communication terminal has been known (for example, see Patent Document 1).

Japanese Patent No. 5710570

  In a conventional system, a message determined based on a situation around a place where an electric device is installed is transmitted to a communication terminal. However, when the message is determined based only on the situation around the place where the electric device is installed as in the conventional system, a message that does not meet the request of a person near the electric device is transmitted. There was a problem that sometimes.

  Therefore, the present invention has been made in view of these points, and an object of the present invention is to make it easy to remotely control an electric device that meets the needs of a person near the electric device.

  An operation assisting device according to a first aspect of the present invention includes a text information obtaining unit that obtains text information created based on a voice message input to a voice processing device having a microphone, and a building from which the voice message is issued. A device specifying unit that specifies a plurality of electric devices installed in the device, and after the text information obtaining unit obtains the text information, one or more electric devices from the plurality of electric devices specified by the device specifying unit A selection unit for selecting the text information and one or more icons used to operate the one or more electrical devices selected by the selection unit are displayed on a communication terminal associated with the voice processing device. And a display control unit for causing the

  The selection unit selects the one or more electric devices based on, for example, a word included in the text information and an electric device stored in a storage unit in association with the word. The selection unit may select the one or more electric devices based on a result of specifying an input person of the voice message corresponding to the text information.

  The display control unit may arrange and display the one or more icons in an order determined based on the likelihood of the process in which the selection unit has selected the one or more electric devices.

  The display control unit, in association with the communication terminal, storing in the storage unit history information indicating an icon operated by the user of the communication terminal after displaying the one or more icons in the past in the communication terminal, The selection unit may select the one or more electric devices based on the history information stored in the storage unit.

  The display control unit may be configured to communicate at least one of a first icon used to remotely operate the electric device and a second icon used to display a method of operating the electric device on the communication terminal. It may be displayed on a terminal.

  The display control unit may determine a type of the one or more icons to be displayed on the communication terminal based on a result of specifying an input person of the voice message corresponding to the text information.

  The display control unit, based on the result of specifying the input person of the voice message corresponding to the text information, to a communication terminal selected from the plurality of communication terminals associated with the voice processing device, the text information, The one or more icons may be displayed.

  A terminal information acquisition unit that acquires selected icon information for identifying an icon selected from the one or more icons in the communication terminal from the communication terminal, and an operation content message corresponding to the icon indicated by the selected icon information, A message transmitting unit that transmits the voice message to the voice processing device to which the voice message has been input.

  After the message transmitting unit transmits the operation content message to the voice processing device, when the text information acquisition unit acquires the predetermined text information, an operation instruction corresponding to the icon indicated by the selected icon information, The apparatus may further include an instruction transmitting unit that transmits the instruction to a control device that controls the plurality of electric devices.

  The instruction transmission unit, after the message transmission unit transmits the operation content message to the voice processing device, after the text information acquisition unit acquires the text information including information specifying the electrical device, the text The operation instruction corresponding to the electric device specified by the information may be transmitted.

  The display control unit, after the message transmission unit transmits the operation content message to the voice processing device, after the text information acquisition unit acquires the text information including information specifying an electrical device, the text The one or more icons used to operate the electric device specified by the information may be displayed on the communication terminal.

  The text information obtaining unit further obtains device status information indicating a status of the plurality of electrical devices at the time when the voice message is input, and the display control unit indicates the text information and the device status information indicate. The states of the plurality of electric devices may be displayed on the communication terminal.

  The text information obtaining unit further obtains device status information indicating a status of the plurality of electrical devices at the time when the voice message is input, and the display control unit determines that the device status information satisfies a predetermined condition. Then, the one or more icons may be displayed on the communication terminal.

  The selection unit may select the one or more electric devices installed in a room where a voice processing device to which the voice message is input is installed.

  An operation assisting method according to a second aspect of the present invention includes the steps of obtaining text information created based on a voice message input to a voice processing device having a microphone, and installing the text information in a building where the voice message is issued. Specifying a plurality of electrical devices that have been identified; selecting one or more electrical devices from the plurality of specified electrical devices after receiving the text information; and a communication terminal associated with the voice processing device. Displaying the text information and one or more icons used to operate the selected one or more electrical devices.

  An operation assisting system according to a third aspect of the present invention includes a voice processing device having a microphone, a communication terminal associated with the voice processing device, and the voice processing based on a voice message input to the voice processing device. An operation assisting device that assists in operating the electrical equipment in the building where the device is installed. The operation assisting device includes: a text information acquisition unit that acquires text information created based on the voice message; and a device specification that specifies a plurality of the electric devices installed in the building where the voice message is issued. Unit, after the text information obtaining unit obtains the text information, the selecting unit for selecting one or more electrical devices from the plurality of electrical devices specified by the device specifying unit, and associated with the audio processing device The communication terminal includes a display control unit configured to display the text information and one or more icons used for operating the one or more electric devices selected by the selection unit. The communication terminal transmits an operation instruction for operating an electric device corresponding to one icon selected from the one or more icons.

  ADVANTAGE OF THE INVENTION According to this invention, there exists an effect that the remote control of the electric equipment which responded to the request of the person who is near the electric equipment can be performed easily.

It is a figure for explaining an outline of an operation auxiliary system. FIG. 2 is a diagram illustrating a functional configuration of the operation assisting device according to the first embodiment. It is a figure showing an example of an electric equipment selection database. FIG. 5 is a diagram illustrating an example of a user information database in which information on users is collected. FIG. 7 is a diagram illustrating an example of a screen (message screen) displayed on the communication terminal. It is a figure showing the example of the screen (remote control screen) displayed on a communication terminal. It is a figure showing an example of a processing sequence in an operation auxiliary system. It is a figure showing the functional composition of the operation auxiliary device concerning a 2nd embodiment.

[Overview of Operation Assistance System S]
FIG. 1 is a diagram for explaining an outline of the operation assisting system S. The operation assisting system S includes an operation assisting device 1, an audio conversion server 2, a communication terminal 3, an audio processing device 4, and a gateway 5. The operation auxiliary device 1, the voice conversion server 2, the communication terminal 3, the voice processing device 4, and the gateway 5 are connected to a network N. The network N includes a communication line capable of transmitting and receiving digital data, such as the Internet and a mobile phone network.

  The operation assist device 1 is a device that assists remote control of various electric devices (for example, the television 21, the lighting 22, the air conditioner 23, and the key 24) installed in the building B. The operation assisting device 1 is a server managed by a company that provides an operation assisting service using the operation assisting system S, for example.

  The voice conversion server 2 converts a voice message emitted in the building B into text information. The voice conversion server 2 transmits the generated text information to the operation assisting device 1.

  The communication terminal 3 is a terminal used by a user who has a right to remotely control an electric device installed in the building B, and is, for example, a smartphone or a tablet. The communication terminal 3 displays, on the display, an icon for remotely controlling an electric device installed in the building B, together with text information based on a voice message issued in the building B based on the control of the operation assisting device 1. The communication terminal 3 displays, for example, an icon corresponding to “air conditioner”, and when the user operating the communication terminal 3 selects the icon, displays a remote control screen for adjusting the temperature of the air conditioner. Details of the screen displayed by the communication terminal 3 will be described later.

  The audio processing device 4 has a microphone and a speaker. The voice processing device 4 converts the voice message detected by the microphone into a digital signal, and transmits the converted digital signal to the voice conversion server 2 via the network N. The voice processing device 4 can also obtain a result of analyzing a voice message detected by the microphone from the voice conversion server 2 and issue a voice message according to the result of the analysis.

  The gateway 5 is a control device that controls a plurality of electric devices installed in the building B. The gateway 5 controls an electric device corresponding to electric device identification information (hereinafter, referred to as an electric device ID) included in the operation instruction based on, for example, the operation instruction received from the communication terminal 3. The electric device ID is information for identifying the electric device.

[Overview of Process Flow in Operation Assistance System S]
Hereinafter, a typical processing flow in the operation assisting system S will be described with reference to FIG. First, when the person H in the building B issues a voice message, the voice processing device 4 acquires the voice message ((1) in FIG. 1). The voice processing device 4 converts the obtained voice message into a digital signal, and transmits the converted digital signal to the voice conversion server 2 ((2) in FIG. 1).

  The voice conversion server 2 phoneme analyzes the digital signal of the received voice message and converts it into text information ((3) in FIG. 1). The voice conversion server 2 transmits the generated text information to the operation assisting device 1 ((4) in FIG. 1).

  Subsequently, after obtaining the text information from the voice conversion server 2, the operation assisting device 1 performs, based on the content of the text information corresponding to the voice message, or based on who issued the voice message, An electric device presumed to be desirably operated by remote control is specified ((5) in FIG. 1). The operation assisting device 1 transmits the text information and the icon information to the communication terminal 3 in order to display the icon for remotely operating the specified electric device on the communication terminal 3 together with the text information received from the voice conversion server 2 ( (6) in FIG. 1). The icon information is, for example, image information or text information indicating the type of the specified electric device or the model name of the electric device.

  When receiving the text information and the icon information from the operation assisting device 1, the communication terminal 3 displays the received text information in time series and displays an icon corresponding to the icon information ((7) in FIG. 1). The icon may be an image or a text as long as the icon is switched to a screen for remote operation by selection by the user of the communication terminal 3.

  When the user selects an icon by, for example, touching the screen, the communication terminal 3 displays a remote control screen for remotely operating an electric device corresponding to the selected icon. The communication terminal 3 transmits operation instruction information corresponding to the user's operation content on the remote control screen to the gateway 5 ((8) in FIG. 1). The operation instruction information includes information for specifying the electric device and information indicating the operation content. The gateway 5 controls the electric device specified in the received operation instruction information based on the operation content indicated by the operation instruction information ((9) in FIG. 1). Note that the communication terminal 3 may transmit the operation instruction information to the gateway 5 without passing through the operation assisting device 1, or may transmit the operation instruction information to the gateway 5 via the operation assisting device 1.

<First embodiment>
[Functional Configuration of Operation Auxiliary Device 1]
FIG. 2 is a diagram illustrating a functional configuration of the operation assisting device 1 according to the first embodiment. The operation assisting device 1 includes a communication unit 11, a storage unit 12, and a control unit 13. The control unit 13 includes a text information acquisition unit 131, a device identification unit 132, a selection unit 133, and a display control unit 134.

  The communication unit 11 is a communication interface for accessing the network N, and has, for example, a LAN (Local Area Network) controller. The communication unit 11 receives text information from the voice conversion server 2 via the network N and transmits icon information to the communication terminal 3 via the network N.

  The storage unit 12 includes a storage medium such as a read only memory (ROM), a random access memory (RAM), and a hard disk. The storage unit 12 stores a program executed by the control unit 13. In addition, the storage unit 12 stores various data necessary for the control unit 13 to execute a process of transmitting icon information to the communication terminal 3.

  FIG. 3 and FIG. 4 are diagrams illustrating an example of data stored in the storage unit 12. FIG. 3 illustrates an example of an electric device selection database used by the selection unit 133 to select one or more electric devices. In the electrical equipment selection database, the electrical equipment name is associated with a related word that is a word related to the electrical equipment. As will be described later, the selection unit 133 refers to, for example, the electric device selection database shown in FIG. 3 and determines, as an electric device to be remotely operated by the user of the communication terminal 3, the text received from the voice conversion server 2. Select the electrical device associated with the word contained in the information.

  FIG. 4 shows an example of a user information database in which information on users is collected. In the user information database, a device ID for identifying the voice processing device 4, a name of an electrical device installed in the building where the voice processing device 4 is installed, and a name of a user who can remotely control the electrical device. The communication terminal 3 is associated with a unique communication terminal ID. The communication terminal ID is, for example, a mobile phone number assigned to the user of the communication terminal 3 by a mobile phone carrier.

  Returning to FIG. 2, the configuration of each unit of the control unit 13 will be described. The control unit 13 is, for example, a CPU (Central Processing Unit), and functions as a text information acquisition unit 131, a device identification unit 132, a selection unit 133, and a display control unit 134 by executing a program stored in the storage unit 12. I do.

  The text information acquisition unit 131 acquires text information created based on the voice message input to the voice processing device 4. Specifically, the text information acquisition unit 131 acquires, via the communication unit 11, text information generated by the voice conversion server 2 converting a voice message. Further, the text information acquisition unit 131 acquires the device ID of the voice processing device 4 to which the voice message has been input, in association with the text information. The text information acquisition unit 131 inputs the acquired device ID to the device identification unit 132, and inputs the acquired text information to the selection unit 133.

  The device specifying unit 132 specifies a plurality of electric devices installed at the location where the voice message has been input. The device specifying unit 132 specifies, for example, the electric device name associated with the device ID input from the text information obtaining unit 131 by referring to the user information database illustrated in FIG. When the text information acquisition unit 131 acquires S10001 as the device ID in the example shown in FIG. 4, it is determined that two air conditioners, a television, and two lights are installed in the building where the voice message was issued. Identify.

  After the text information obtaining unit 131 obtains the text information, the selection unit 133 selects one or more electrical devices whose icons are displayed on the communication terminal 3. The selection unit 133 selects one or more electric devices based on the words included in the text information acquired by the text information acquisition unit 131, for example, with reference to the electric device selection database shown in FIG. The selection unit 133 extracts one or more words included in the text information and, when the extracted one or more words match or is similar to any of the related words in the electrical device selection database, matches or is similar. Select the electrical device name associated with the related term. In the case of the example shown in FIG. 3, for example, when the text information is “this room is very hot” and the word “hot” is included, the selection unit 133 selects the air conditioner.

  When one or more words included in the text information are associated with a plurality of electric devices in the electric device selection database, the selection unit 133 selects the plurality of electric devices. If the text information includes a plurality of words, the selection unit 133 selects a plurality of electric devices associated with each of the plurality of words.

  The selection unit 133 may select the electric device based on the result of specifying the input person of the voice message corresponding to the text information instead of or together with the word included in the text information. For example, when the subject included in the text information is “I”, the selection unit 133 specifies that the person who issued the voice message is a child, and when the subject included in the text information is “I”, Identify the person who sent the voice message as your mother. Then, when specifying that the person who issued the voice message is a child, the selection unit 133 selects an electric device (for example, an air conditioner and a television) stored in the storage unit 12 in advance in association with the child.

  When specifying that the person who issued the voice message is the mother, the selection unit 133 selects an electric device (for example, an air conditioner and a key) stored in the storage unit 12 in advance in association with the mother. As described above, the selection unit 133 selects the electric device based on the input person of the voice message, thereby increasing the probability that the input user of the voice message can select the electric device that the user wants to remotely control.

  Note that the selection unit 133 may obtain information for specifying the input person of the voice message from the voice conversion server 2 and select an electric device based on the obtained information. In this case, the voice conversion server 2 performs frequency analysis of the digital signal received from the voice processing device 4 to specify the age and gender of the input person of the voice message, and determines the specified result to the input person of the voice message. The operation assisting device 1 is notified as information for specifying.

  Further, the electric device selection database includes information indicating whether or not each electric device is installed in the same room as the audio processing device 4, and the selecting unit 133 outputs a voice message based on the information. One or more electrical devices installed in the room where the input voice processing device 4 is installed may be selected. By doing so, the selection unit 133 can select the electrical device in the room where the person who has input the voice message is located, as the electrical device whose icon is to be displayed on the communication terminal 3. The probability that the user can remotely control the electrical device in the room where the person who input the voice message is located increases.

  The display control unit 134 controls the communication terminal associated with the voice processing device 4 to operate the text information based on the voice message input to the voice processing device 4 and the one or more electric devices selected by the selection unit 133. One or more icons to be used are displayed. Specifically, the display control unit 134 transmits to the communication terminal 3 the text information transmitted from the voice conversion server 2 and the icon information including the information for specifying the electric device selected by the selection unit 133. Thus, the text information and the icon based on the voice message are displayed on the communication terminal 3. The icon information may include information indicating the type name of the electric device together with the type of the electric device selected by the selection unit 133.

  FIG. 5 and FIG. 6 are diagrams illustrating examples of screens displayed on the communication terminal 3. FIG. 5 shows a message screen displayed on the communication terminal 3 after the child in the building B issues a voice message saying "I've arrived at home" or "Hot!" Since the word "hot" is included in the voice message, the screen shown in FIG. 5 is displayed when the selecting unit 133 selects "air conditioner A", "air conditioner B", and "fan".

  FIG. 6 is a diagram showing an example of a remote control screen displayed when the user of communication terminal 3 selects "air conditioner A" on the screen shown in FIG. The communication terminal 3 stores, for example, screen data of a remote control screen that differs for each type of electric device, and when an icon is selected, corresponds to the type of electric device indicated by the icon information transmitted from the operation assisting device 1 Display the remote control screen. The communication terminal 3 stores screen data of a remote control screen that differs for each model name of an electric device. When an icon is selected, the communication terminal 3 corresponds to the model name of the electric device indicated by the icon information transmitted from the operation assisting device 1. May be displayed. When a button in the remote control screen is pressed while the remote control screen is displayed, the communication terminal 3 transmits operation instruction information corresponding to the pressed button to the gateway 5.

  The communication terminal 3 does not store the screen data of the remote control screen, and uses address information for acquiring the screen data (for example, for accessing an area where the screen data is stored in the storage unit 12 of the operation assisting device 1). (URL information). In this case, the communication terminal 3 acquires the screen data using the address information associated with the remote control screen data of the electric device corresponding to the selected icon, and displays the remote control screen based on the acquired screen data.

  The display control unit 134 may arrange and display one or more icons in an order determined based on the likelihood of the process in which the selection unit 133 has selected one or more electric devices. For example, in the electrical device selection database, a weighting factor is assigned to each related word, and the display control unit 134 determines that the likelihood increases as the value of the weighting factor of the related word included in the text information increases. Then, the icons are arranged and displayed in descending order of the likelihood. By doing so, it is possible to increase the probability that the user of the communication terminal 3 can perform a remote operation according to the intention of the person who issued the voice message.

  The display control unit 134 may cause the storage unit 12 to store history information indicating an icon selected by the user of the communication terminal 3 in association with the communication terminal 3. To do so, when an icon is selected by the user, the communication terminal 3 transmits information indicating the selected icon to the operation assisting device 1. When the history information is stored in the storage unit 12 as described above, the selection unit 133 can select one or more electrical devices based on the history information stored in the storage unit 12, and thus issues a voice message. It is possible to increase the probability that a person who wants to operate the electronic device wants to operate.

  The display control unit 134 may store the word included in the text information based on the voice message and the history information indicating the icon selected by the user of the communication terminal 3 in the storage unit 12 in association with the communication terminal 3. Good. In this case, since the selection unit 133 can select one or more electric devices according to the content of the voice message, there is a high probability that the user will select the communication terminal 3 when a specific voice message is issued. Icons are easier to display. The display control unit 134 may cause the communication terminal 3 to display a plurality of icons arranged in the order determined based on the history information.

  The display control unit 134 causes the communication terminal 3 to display at least one of a first icon used to remotely control the electric device and a second icon used to display a method of operating the electric device on the communication terminal 3. It may be displayed. The first icon is an icon for displaying a remote control screen when selected by the user, as shown in FIG. The second icon is an icon for displaying an operation manual of the electric device corresponding to the second icon when selected by the user. The communication terminal 3 may store the electronic data of the operation manual in association with the model name of the electric device, or may obtain the electronic data of the operation manual from the operation auxiliary device 1 or another device.

  The display control unit 134 may cause the communication terminal 3 to display a third icon for notifying the in-home device that cannot be directly operated from the communication terminal 3 to a predetermined state, instead of the first icon. Good. The display control unit 134 acquires information indicating the state of the electric device associated with the received message, for example, when the text information acquisition unit 131 receives a predetermined message, and when the acquired information satisfies a predetermined condition. Causes the communication terminal 3 to display the third icon.

  Specifically, when the text information obtaining unit 131 obtains text information corresponding to a voice message “Now” by a child who has returned home, the display control unit 134 is a window that is an electrical device corresponding to the text information. Detects the state of the open / close sensor. When the open / closed state of the window is “open”, the display control unit 134 causes the communication terminal 3 to display a third icon “close the window”. Further, the display control unit 134 transmits a voice message “Please close the window” to the voice processing device 4. With such a configuration, a child who hears the voice message "Please close the window" issued by the voice processing device 4 can close the window.

  The display control unit 134 may determine the type of one or more icons to be displayed on the communication terminal 3 based on the result of specifying the input person of the voice message corresponding to the text information. The display control unit 134 displays the first icon so that the user (for example, father) of the communication terminal 3 can remotely control the electric device when the input person is a child, and performs communication when the input person is not a child. The second icon is displayed so that the user of the terminal 3 can explain the operation method. By doing so, the user of the communication terminal 3 can take appropriate action for the input person of the voice message.

  The display control unit 134 selects and selects one communication terminal 3 from the plurality of communication terminals 3 associated with the voice processing device 4 based on the result of specifying the input person of the voice message corresponding to the text information. The communication terminal 3 may display text information and one or more icons. In this case, the storage unit 12 associates the person who issued the voice message among the members of the family associated with the voice processing device 4 with the person whose text information and icons are to be displayed on the communication terminal 3. Remember. For example, when the voice message input person is a child, storage unit 12 stores mother's communication terminal 3 as communication terminal 3 that displays text information and icons. When the input person of the voice message is the mother, the storage unit 12 stores the communication terminal 3 of the father as the communication terminal 3 displaying the text information and the icon.

  The display control unit 134 displays the text information and the icon on the communication terminal 3 of the person corresponding to the input person of the voice message by referring to the information stored in the storage unit 12. For example, when the input person of the voice message is a child, the display control unit 134 causes the communication terminal 3 of the mother to display the text information and the icon. When the input person of the voice message is the mother, the display control unit 134 causes the communication terminal 3 of the father to display the text information and the icon. With the display control unit 134 configured as described above, a person who is suitable for the skill level at which the person who inputs the voice message can operate the electric device can provide remote control or advice on the operation method.

[Processing Sequence in Operation Assistance System S]
FIG. 7 is a diagram illustrating an example of a processing sequence in the operation assisting system S. When receiving the input of the voice message, the voice processing device 4 transmits a digital signal of the voice message to the voice conversion server 2. The voice conversion server 2 converts the received voice message into text information (S1). The voice conversion server 2 transmits the generated text information to the operation assisting device 1.

  After receiving the text information, the operation assisting device 1 selects an electric device for which an icon for remote operation is displayed on the communication terminal 3 (S2). Subsequently, the operation assisting device 1 transmits to the communication terminal 3 icon information for displaying the icon corresponding to the selected electric device on the communication terminal 3 together with the text information. The communication terminal 3 displays the icon indicated by the icon information together with the received text information (S3).

  In this state, the communication terminal 3 monitors whether an icon has been selected (S4). The communication terminal 3 waits while displaying the text information and the icon until the icon is selected. When the icon is selected (YES in S4), the communication terminal 3 remotely operates the electric device corresponding to the selected icon. A remote control screen is displayed (S5). The communication terminal 3 transmits operation instruction information including the content operated on the remote control screen to the gateway 5. The gateway 5 controls the electric device specified in the received operation instruction information (S6).

[Effects of Operation Assistance Device 1 According to First Embodiment]
As described above, the selection unit 133 of the operation assisting device 1 selects one or more electrical devices after acquiring the text information created based on the voice message input to the voice processing device 4. Then, the display control unit 134 displays the text information and one or more icons used to operate the one or more electric devices selected by the selection unit 133 on the communication terminal 3 associated with the audio processing device 4. Let it. Since the operation assisting device 1 has such a configuration, even if a person who inputs a voice message to the voice processing device 4 cannot operate the electric device, the user of the communication terminal 3 can It becomes possible for the person who inputs the voice message to remotely control the electric device as desired.

<Second embodiment>
FIG. 8 is a diagram illustrating a functional configuration of an operation assisting device 1A according to the second embodiment. The operation assisting device 1A according to the second embodiment is different from the operation assisting device 1 according to the first embodiment in further including a terminal information acquisition unit 135, a message transmission unit 136, and an instruction transmission unit 137. The same in other respects.

  The terminal information acquisition unit 135 acquires selected icon information for specifying an icon selected from one or more icons in the communication terminal 3 from the communication terminal 3. The terminal information acquisition unit 135 notifies the message transmission unit 136 of the acquired selected icon information.

  The message transmitting unit 136 transmits an operation content message corresponding to the icon indicated by the selected icon information to the voice processing device 4 to which the voice message has been input. For example, when the user of the communication terminal 3 selects an icon for remotely controlling the air conditioner, the message transmitting unit 136 outputs an operation content message "Remotely control the air conditioner, is it OK?" Send to The message transmission unit 136 transmits an operation content message via the voice conversion server 2, for example. The voice processing device 4 may output the received operation content message as voice from a built-in speaker, or may display the message on a display provided in the voice processing device 4.

  The instruction transmitting unit 137 transmits an operation instruction for controlling the electric device to the gateway 5. For example, when the text information acquisition unit 131 acquires predetermined text information after the message transmission unit 136 transmits the operation content message to the voice processing device 4, the instruction transmission unit 137 corresponds to the icon indicated by the selected icon information. To the gateway 5. The predetermined text information is text information indicating that the electric device indicated by the operation content message is approved to be remotely operated, and is a positive word such as “Good” or “Thank you”.

  When the text information acquisition unit 131 acquires text information including a word indicating that remote operation is to be stopped, the instruction transmission unit 137 does not transmit the operation instruction information to the gateway 5. As described above, the instruction transmitting unit 137 transmits the operation instruction information to the gateway 5 on condition that the predetermined text information is obtained, thereby performing the remote operation contrary to the intention of the voice message input person. Can be prevented.

  When the text information acquisition unit 131 acquires text information including a word indicating that the remote operation is to be stopped, the display control unit 134 stops displaying the icon for the remote operation on the communication terminal 3. Alternatively, the display on the remote control screen may be stopped. Since the message transmitting unit 136 transmits the operation content message in this way, the input person of the voice message recognizes that the electric device is remotely controlled, and can cope with the recognition result. It is possible to prevent a remote operation that does not meet the intention of the input person from being performed.

  In addition, after the message transmitting unit 136 transmits the operation content message to the voice processing device 4 and the text information acquiring unit 131 acquires the text information including the information specifying the electric device, the instruction transmitting unit 137 transmits the text. An operation instruction corresponding to the electric device specified by the information may be transmitted. For example, when the message information transmission unit 136 transmits the operation content message indicating that the air conditioner is turned on and then the text information acquisition unit 131 acquires the text information “I am in the living room”, An operation instruction for turning on an air conditioner installed in a room is not transmitted, and an operation instruction for turning on an air conditioner installed in another room is not transmitted. By doing so, the instruction transmitting unit 137 can realize remote operation that meets the needs of the voice message inputter, and thus the satisfaction of the voice message inputter is improved.

  When the text information obtaining unit 131 obtains text information including information specifying the electrical device after the message transmitting unit 136 transmits the operation content message to the voice processing device 4, the display control unit 134 performs the processing based on the text information. One or more icons used to operate the specified electrical device may be displayed on the communication terminal 3. For example, when the message information transmission unit 136 transmits the operation content message indicating that the air conditioner is turned on and then the text information acquisition unit 131 acquires the text information “I am in the living room”, Icon for remotely operating the air conditioner installed in the communication terminal 3 is displayed. By doing so, the communication terminal 3 can realize remote control that meets the needs of the voice message inputter, and thus the satisfaction of the voice message inputter is improved.

[Effects of Operation Auxiliary Device 1A According to Second Embodiment]
As described above, in the operation assisting device 1A according to the second embodiment, the terminal information acquisition unit 135 specifies which icon has been selected in the communication terminal 3, and the message transmission unit 136 has the voice processing device Via 4, the user of the voice message is notified of which electrical device is to be remotely controlled. Then, the instruction transmitting unit 137 transmits operation instruction information for controlling the electric device to the gateway 5 on condition that the input person of the voice message inputs a predetermined voice message. When the operation assisting device 1A has such a configuration, the probability of performing a remote operation against the intention of the person who inputs the voice message can be reduced, so that the satisfaction of the person who inputs the voice message improves.

<Third embodiment>
In the operation assisting device 1 according to the first embodiment and the operation assisting device 1A according to the second embodiment, it is assumed that the user of the communication terminal 3 remotely controls the electric device without checking the state of the electric device. I was On the other hand, the operation assisting device 1 or the operation assisting device 1A according to the third embodiment is different from the first and second embodiments in that the user of the communication terminal 3 can grasp the state of the electric device. Is different from the embodiment.

  The text information acquisition unit 131 of the present embodiment further acquires device status information indicating the status of a plurality of electrical devices at the time when a voice message is input. The display control unit 134 causes the communication terminal 3 to display text information and states of a plurality of electric devices indicated by the device state information. Since the text information acquisition unit 131 and the display control unit 134 are configured as described above, the user of the communication terminal 3 can remotely operate the electric device after recognizing the state of the electric device. Operation probability can be reduced. For example, when the setting of the air conditioner is set to 25 ° C., the user of the communication terminal 3 can determine that the set temperature of the air conditioner is not further reduced.

  Further, the display control unit 134 may cause the communication terminal 3 to display one or more icons when the device status information satisfies a predetermined condition. For example, even when the text information acquired by the text information acquisition unit 131 includes the word "hot", if the set temperature of the air conditioner is 25 ° C, it is not possible to further lower the set temperature. Is appropriate. Therefore, when the device state information indicates that the set temperature of the air conditioner is equal to or lower than the threshold, the display control unit 134 does not cause the communication terminal 3 to display an icon for remotely operating the air conditioner. With this configuration, the display control unit 134 prevents the user of the communication terminal 3 from erroneously remotely operating the electric device when the request of the person who inputs the voice message is not appropriate. be able to.

<Fourth embodiment>
In each of the above embodiments, after the operation assisting device 1 or the operation assisting device 1A causes the communication terminal 3 to display an icon for remotely controlling the electric device, the electric device is operated in accordance with the operation of the user of the communication terminal 3. Remotely controlled. On the other hand, the operation assisting device 1B according to the fourth embodiment controls the electric device automatically based on the content of the text information based on the voice message before the icon is displayed on the communication terminal 3, and then performs the communication. This is different from the other embodiments in that an icon can be displayed on the terminal 3.

  The operation assisting device 1B includes an instruction transmitting unit 137 included in the operation assisting device 1A according to the second embodiment, in addition to the configuration of the operation assisting device 1 according to the first embodiment. When the text information based on the voice message acquired by the text information acquisition unit 131 includes a predetermined word, the instruction transmission unit 137 transmits operation instruction information for controlling an electric device associated with the predetermined word. Send to gateway 5. The predetermined word is, for example, a related word included in the electric device selection database shown in FIG.

  The instruction transmitting unit 137 transmits operation instruction information for controlling the electric device so as to improve the state indicated by the related word. For example, when the text information includes a related word “hot”, the instruction transmitting unit 137 transmits operation instruction information for instructing to lower the temperature of the air conditioner.

  After the instruction transmitting unit 137 transmits the operation instruction information to the gateway 5, the display control unit 134 transmits to the communication terminal 3 text information indicating that the electric device has been remotely operated. The display control unit 134 sequentially transmits, for example, the communication terminal 3 such that the text information based on the voice message and the text information indicating that the electric device has been remotely operated are displayed in a time-series manner. . The communication terminal 3 controls the text information based on the voice message and the remote operation of the operation assisting device 1B so that the user can recognize that the operation assisting device 1B has automatically remotely operated the electric device after the voice message is issued. And text information indicating that the operation has been performed.

  As described above, since the operation assisting device 1B can automatically control the electric device based on the content of the voice message, for example, the user of the communication terminal 3 can issue a voice message saying “It seems hot” Can prevent a problem that a child suffers from heat stroke when remote control is not possible.

  Note that the necessity of the automatic remote control of the operation assisting device 1B differs depending on the electric device. Therefore, the storage unit 12 may store information indicating the necessity of automatic remote control by the operation assisting device 1B in association with the electric device. Then, the instruction transmitting unit 137 includes, in the text information based on the voice message, a related word corresponding to the electric device in which information indicating that automatic remote control is necessary is stored in the storage unit 12. On the condition that the display control unit 134 transmits the operation instruction information to the gateway 5 before transmitting the icon information to the communication terminal 3.

  As described above, the present invention has been described using the embodiment, but the technical scope of the present invention is not limited to the scope described in the above embodiment, and various modifications and changes are possible within the scope of the gist. is there. For example, the specific embodiment of the dispersion / integration of the apparatus is not limited to the above-described embodiment, and all or a part of the apparatus may be functionally or physically dispersed / integrated in an arbitrary unit. Can be. Further, new embodiments that are generated by arbitrary combinations of the plurality of embodiments are also included in the embodiments of the present invention. The effect of the new embodiment caused by the combination has the effect of the original embodiment.

Reference Signs List 1 operation assisting device 2 voice conversion server 3 communication terminal 4 voice processing device 5 gateway 11 communication unit 12 storage unit 13 control unit 21 television 22 lighting 23 air conditioner 24 key 131 text information acquisition unit 132 device identification unit 133 selection unit 134 display control unit 135 terminal information acquisition unit 136 message transmission unit 137 instruction transmission unit

Claims (17)

A text information acquisition unit that acquires text information created based on a voice message input to a voice processing device having a microphone,
A device identification unit that identifies a plurality of electrical devices installed in the building where the voice message was issued,
After the text information acquisition unit acquires the text information, a selection unit that selects one or more electrical devices from the plurality of electrical devices identified by the device identification unit,
The communication terminal associated with the audio processing device, the text information, a display control unit that displays one or more icons used to operate the one or more electrical devices selected by the selection unit,
Operation assisting device having The selection unit, based on the words included in the text information and the electrical device stored in the storage unit in association with the word, to select the one or more electrical devices,
The operation assisting device according to claim 1. The selection unit is configured to select the one or more electric devices based on a result of specifying an input person of the voice message corresponding to the text information,
The operation assisting device according to claim 1. The display control unit displays the one or more icons in an order determined based on the likelihood of the process in which the selection unit has selected the one or more electrical devices,
The operation assisting device according to any one of claims 1 to 3. The display control unit, in association with the communication terminal, after storing in the storage unit history information indicating an icon operated by the user of the communication terminal after displaying the one or more icons in the communication terminal in the past,
The selection unit selects the one or more electrical devices based on the history information stored in the storage unit,
The operation assisting device according to claim 1. The display control unit may be configured to communicate at least one of a first icon used to remotely operate the electric device and a second icon used to display a method of operating the electric device on the communication terminal. To display on the terminal,
The operation assisting device according to claim 1. The display control unit determines a type of the one or more icons to be displayed on the communication terminal based on a result of specifying an input person of the voice message corresponding to the text information,
The operation assistance device according to claim 6. The display control unit, based on the result of specifying the input person of the voice message corresponding to the text information, to a communication terminal selected from the plurality of communication terminals associated with the voice processing device, the text information, Displaying the one or more icons,
The operation assisting device according to any one of claims 1 to 7. A terminal information acquisition unit that acquires selected icon information from the communication terminal for specifying an icon selected from the one or more icons in the communication terminal.
A message transmitting unit that transmits an operation content message corresponding to the icon indicated by the selected icon information to a voice processing device to which the voice message has been input,
Further having
An operation assisting device according to any one of claims 1 to 8. After the message transmitting unit transmits the operation content message to the voice processing device, when the text information acquisition unit acquires the predetermined text information, an operation instruction corresponding to the icon indicated by the selected icon information, Further comprising an instruction transmission unit for transmitting to a control device that controls the plurality of electrical devices,
The operation assisting device according to claim 9. The instruction transmission unit, after the message transmission unit transmits the operation content message to the voice processing device, after the text information acquisition unit acquires the text information including information specifying the electrical device, the text Transmitting the operation instruction corresponding to the electrical device specified by the information,
The operation assisting device according to claim 10. The display control unit, after the message transmission unit transmits the operation content message to the voice processing device, after the text information acquisition unit acquires the text information including information specifying an electrical device, the text Causing the communication terminal to display the one or more icons used to operate the electrical device specified by the information,
The operation assisting device according to claim 10. The text information obtaining unit further obtains device state information indicating a state of the plurality of electric devices at the time when the voice message is input,
The display control unit causes the communication terminal to display the text information and the states of the plurality of electric devices indicated by the device state information,
The operation assisting device according to claim 1. The text information obtaining unit further obtains device state information indicating a state of the plurality of electric devices at the time when the voice message is input,
The display control unit, when the device state information satisfies a predetermined condition, causes the communication terminal to display the one or more icons,
An operation assisting device according to any one of claims 1 to 13. The selection unit selects the one or more electrical devices installed in the room where the voice processing device in which the voice message has been input is installed,
The operation assisting device according to any one of claims 1 to 14. Obtaining text information created based on a voice message input to a voice processing device having a microphone;
Identifying a plurality of electrical devices installed in the building where the voice message was issued,
After receiving the text information, selecting one or more electrical devices from the plurality of specified electrical devices,
A step of displaying the text information and one or more icons used to operate the selected one or more electric devices on a communication terminal associated with the voice processing device,
Operation assisting method having Based on a voice processing device having a microphone, a communication terminal associated with the voice processing device, and a voice message input to the voice processing device, operate an electrical device in a building where the voice processing device is installed. And an operation assisting device that assists for
The operation assist device includes:
A text information acquisition unit that acquires text information created based on the voice message,
A device identification unit that identifies a plurality of the electrical devices installed in the building where the voice message was issued,
After the text information acquisition unit acquires the text information, a selection unit that selects one or more electrical devices from the plurality of electrical devices identified by the device identification unit,
The communication terminal associated with the voice processing device, the text information, a display control unit that displays one or more icons used to operate the one or more electrical devices selected by the selection unit,
Has,
The communication terminal transmits an operation instruction for operating an electric device corresponding to one icon selected from the one or more icons,
Operation assistance system.

Images