WO2012008553A1 - Robot system - Google Patents

Abstract

A robot which is caused by a user to make a predetermined action through the action etc. of the user is provided. An operation device is provided with an action association and storage means which associates and stores action sensing information with robot control information and an action determination means which inputs measured values from a sensor for measuring the action of the user, retrieves the action association and storage means for the action sensing information matching with the measured values, and outputs to the robot a control instruction for making the action described in the robot control information corresponding to the retrieved action sensing information.

Description

Robot system

The present invention relates to a robot system.

In order to facilitate communication between users at remote locations and people at the site, use the movement of the line of sight of the robot acting on behalf of the user at the site, and tell the people at the site what the remote user sees at the site. It is important to communicate. In order to achieve this, a remote user must control the posture of the mechanism on which the robot camera is mounted so that the robot's line of sight is aligned with a desired object at the site.
For example, in the robot apparatus described in Patent Document 1, a user makes a robot attend a remote conference hall, and uses an operation terminal for operating the robot from a remote location to operate the head of the robot equipped with the camera. Can do. By moving the head, the user can control the direction of the camera at a remote location and acquire an image in the direction desired by the user. In addition, the participant in the conference hall with the direction of the head of the robot can understand what the user is looking at.

JP 2002-46088 A

In the system described in Patent Document 1, a remote user can use the head operation unit of the operation terminal to operate the head of a robot equipped with a camera in the field. Therefore, the user can view the video captured by the head camera on the operation terminal. In actual use scenes such as meetings, there are cases where materials are displayed on a display that can be viewed by all participants on the site, and participants discuss while viewing the screen. At this time, the user at the remote location does not look at the display on which the material at the site is displayed via the operation terminal, but displays the same material on another display installed at the remote location and discusses it while referring to it. May participate in. When using in such a way, in order for the user at the remote site to understand that the user is viewing the display material, the remote head is operated after the robot head is directed to the display at the site. It is necessary to go through the procedure of reading the material displayed on the local display.
Therefore, the robot described in Patent Document 1 has a problem that it is troublesome to operate the robot on site. This invention is providing the robot system for solving the said subject.

The operation device according to an embodiment of the present invention includes a behavior association storage unit that stores behavior sensing information and robot control information in association with each other, and inputs a measurement value from a sensor that measures a user's behavior, and the behavior association An action determining means for retrieving action sensing information matching the measured value from a storage means and outputting a control command for taking an action described in the robot control information corresponding to the retrieved action sensing information to the robot; .
An operation program according to an embodiment of the present invention inputs a measurement value from a sensor that measures a user's behavior and a storage process in which behavior sensing information and robot control information are associated with each other and stored in the behavior association storage unit. And searching for behavior sensing information that matches the measured value from the behavior association storage means, and outputting to the robot a control command for taking an action described by the robot control information corresponding to the searched behavior sensing information. The operation determination process is executed.
In an operation method according to an embodiment of the present invention, behavior sensing information and robot control information are associated with each other and stored in a behavior association storage unit, and a measurement value is input from a sensor that measures a user's behavior, and the behavior association is performed. The behavior sensing information matching the measured value is retrieved from the storage means, and a control command for taking the behavior described by the robot control information corresponding to the retrieved behavior sensing information is output to the robot.

The robot system of the present invention makes it easy for a user to cause a robot to perform a predetermined operation through his / her own operation.

It is a figure which shows the structure of the robot system 90 of 1st Embodiment. It is a figure which shows the external appearance of the robot 20 of 1st Embodiment. It is a figure which shows the external appearance of the operating device 10 of 1st Embodiment. It is a figure which shows the structure of the operation input part 13 and action recognition part 14 of 1st Embodiment. It is a flowchart which shows the process of the operation | movement determination part 16 of 1st Embodiment. It is a figure which shows the example of data stored in action matching DB15 of 1st Embodiment. It is a figure which shows the structure of the operation input part 13 and action recognition part 14 of 2nd Embodiment. It is a figure which shows the structure of the operating device 10 of 3rd Embodiment.

Next, the form for implementing the robot system 90 which concerns on invention is demonstrated in detail with reference to drawings.
FIG. 1 is a diagram illustrating an example of a first embodiment of a robot system 90 according to the present invention. The robot system 90 according to the first embodiment includes an operation device 10 and a robot 20. The operation device 10 is often installed in a remote place of the robot 20, but may be installed in a nearby place such as an adjacent room. Hereinafter, it is assumed that the operating device 10 is installed in a remote place of the robot 20. The operating device 10 and the robot 20 are connected via a network. The network may be either wireless or wired. In the case of wireless, the network is a wireless local area network (LAN), a mobile phone network, or the like. In the case of wired communication, the network is serial communication, parallel communication, LAN, or the like. The network may be another alternative communication method.
The robot 20 includes an audio input unit 21 such as a microphone and a video input unit 22 such as a camera. The voice input by the voice input unit 21 and the image input by the video input unit 22 are transmitted to the voice presentation unit 11 such as a speaker of the operation device 10 and the video presentation unit 12 such as a display device via a network, and are transmitted to a remote place. Is presented to the user of the operating device 10. In addition, the controller device 10 includes a voice input unit 17. The voice input by the voice input unit 17 is transmitted to the voice presentation unit 25 of the robot 20 via the network, and presented to people in the field such as a conference room.
The operation device 10 includes an operation input unit 13. When the user of the operation device 10 wants to move the head of the robot 20 or the like, the user operates the joystick 31 or the like of the operation input unit 13. When a signal indicating the operation is input from the operation input unit 13, the operation determination unit 16 transmits a control command to the operation control unit 24 of the robot 20 via the network to operate the operation mechanism 23.
The operating device 10 includes an action recognition unit 14. The behavior recognition unit 14 is a sensor that is mounted on the user's head, for example, and measures the user's motion and posture. When recognizing that a remote user has performed a certain action from the input measurement value, the action determination unit 16 generates action sensing information corresponding to the action. Thereafter, the same part searches the behavior sensing information 53 recorded in the behavior association DB 15 (Data Base) for a match with the information. The same unit transmits a control command to the operation control unit 24 of the robot 20 via the network so as to execute the operation described by the robot control information 54 associated with the searched behavior sensing information 53.
The operation determination unit 16 is configured by hardware such as a logic circuit. The operation determination unit 16 may be realized by a processor (not shown) of the operation device 10 that is also a computer reading and executing a program stored in a memory. The program is stored in, for example, a hard disk device connected to the operation device 10 and is read into the memory when the operation device 10 is initially set.
FIG. 6 shows an example of data recorded in the action association DB 15. The action association DB 15 includes data that associates the actions of the user and the robot 20. The action determination unit 16 refers to the data and recognizes that the user's line of sight at the remote place has seen the object (for example, the material A at hand) at the remote place. Then, the same unit installs the line of sight of the robot 20 in the field (for example, a conference room) and corresponds to the object (for example, the document A displayed on the screen) associated with what the user sees on the remote side. Perform the action of pointing the camera in the direction.
The behavior association DB 15 is a table including one or more records (rows) that associate the behavior sensing information 53 and the robot control information 54. Each record of the action association DB 15 may include an ID 51 and an action recognition type 52.
The behavior sensing information 53 is data describing a user's behavior (posture, motion, etc.). The behavior sensing information 53 is, for example, a predetermined position centered on one or more virtual rotation axes of the user's head measured by the behavior recognition unit 14 (angle sensor 41) mounted on the user's head. Or the rotation angle from the current position. The behavior sensing information 53 includes a single angle, a change from one angle within a predetermined time to another angle, a holding time of the same angle, and the like.
The robot control information 54 is data describing the behavior of the robot 20. The robot control information 54 is, for example, a rotation angle from a predetermined position or a current position around one or more operation rotation axes of the head of the robot 20. The robot control information 54 includes a single angle, a change from one angle within a predetermined time to another angle, a holding time of the same angle, and the like.
The action recognition type 52 stores text indicating the type of action.
Each data stored in the action association DB 15 may be stored in advance before the robot system 90 is activated, or may be stored by the user using the operation device 10 as described later.
The action determination unit 16 inputs one or more measurement values from the action recognition unit 14 (angle sensor 41), and based on the measurement value itself, a change in the measurement value, a holding time of the same value, or the like Generate sensing information. When the generated action sensing information matches the value of the action sensing information 53 stored in the action association DB 15, the action determining unit 16 takes the action described by the robot control information 54 of the matched record. The robot 20 is controlled.
As a specific example, the record of ID = 1 in FIG. 6 stores the text “I watched television” as the action recognition type 52. The record includes data such as “value of angle sensor 41, upper 10 degrees, left 100 degrees” as behavior sensing information 53, and data “head up 20 degrees, right 90 degrees” as robot control information 54. .
If it is determined that the user's action is “angle sensor value, upper 10 degrees, left 100 degrees” from the measurement value of the angle sensor 41 that measures the angle of the user's head, the action determination unit 16 sets the action association DB 15 By searching, the record of ID = 1 in FIG. 6 is found. Subsequently, based on the robot control information 54 of the same record, the motion determination unit 16 transmits a control command for “head up 20 degrees, right 90 degrees” to the motion control unit 24 of the robot 20.
In addition, the action recognition unit 14, the action determination unit 16, the action control unit 24, and the like, as the action sensing information 53, “moved the head downward by 5 deg within 1 sec and then moved upward by 5 deg”, and the robot control information 54 as “ The head may be moved 10 deg downward in 0.5 sec and moved 10 deg upward in 0.5 sec ”.
Note that the voice presentation unit 11, the video presentation unit 12, the operation input unit 13, the action recognition unit 14, and the voice input unit 17 may be provided in another device outside the operation device 10.
Further, the behavior recognition unit 14 may monitor a user's behavior from a video image of a camera installed in the vicinity of the user and detect a head, hand position, eyelid movement, mouth movement, and the like. In this case, the behavior sensing information 53 may include data such as pointing a finger at the object, moving the object, or using a function of the object.
The robot control information 54 may describe the behavior of the robot 20 such as moving an arm or self-running. In this case, it is assumed that the robot 20 can take the described action.
FIG. 2 is a view showing the appearance of the robot 20. The robot 20 has a video input unit 22 that captures video on the robot 20 side, a voice input unit 21 that records audio, and a remote site side in order to realize a dialogue between a remote user and a person on the robot 20 side. You may provide the audio | voice presentation part 25 which outputs a user's audio | voice. The robot 20 includes an operation mechanism 23 for transmitting a user's movement at a remote place. The motion mechanism 23 is moved by an actuator and operates in the vertical direction as a pitch axis and the horizontal direction as a yaw axis as an operation rotation axis.
The operation mechanism 23 may have an operation axis in the direction of the neck that is a roll axis. Furthermore, the operation mechanism 23 may have an actuator of any one of a pitch axis, a yaw axis, and a roll axis. Further, the operation mechanism 23 may include an actuator capable of translational operation in the front / rear, up / down, and left / right directions. The robot 20 may further include a moving mechanism that moves the robot 20 itself in addition to the head as the operation mechanism 23. The moving mechanism may be either a wheel mechanism that can move in all directions or a wheel mechanism that includes two parallel driving wheels.
The video input unit 22 may calculate distance information using a plurality of cameras and present the distance information to the user on the controller device 10 side. Further, the video input unit 22 may have a zoom mechanism, and may capture a video with a zoom magnification desired by the user on the operation device 10 side. The video input unit 22 may be equipped with a plurality of cameras having different shooting directions so that the user on the operation device 10 side can select a video. The video input unit 22 may be equipped with a camera that captures the entire periphery of the robot 20, a super wide-angle camera, and a fisheye camera.
The audio input unit 21 is a monaural microphone or a stereo microphone. Moreover, the voice input unit 21 may measure the direction of sound using a plurality of microphones and present it to the operation device 10 side user.
The robot 20 may include a line-of-sight display unit 28 that enables a person observing the robot 20 to determine the line of sight of the robot 20. The line-of-sight display unit 28 is, for example, unevenness that represents the eyes and nose of the robot 20. When the robot 20 includes the video input unit 22 having an appearance like an eye, the video input unit 22 also functions as the line-of-sight display unit 28.
FIG. 3 is a diagram illustrating an appearance of the operation device 10. The operation device 10 is a device for a remote user to remotely operate the robot 20. The operation device 10 includes an operation input unit 13 for inputting the movement of the robot 20, a video presentation unit 12 that presents a video shot on the robot 20 side, and a voice presentation that presents voice input on the robot 20 side. Unit 11 and a voice input unit 17 for inputting a voice of a remote user. A user who uses the operation device 10 interacts with a person (site person) on the robot 20 side through voice, video, and movement of the robot 20.
FIG. 4 is a diagram illustrating the configuration of the operation input unit 13 and the action recognition unit 14 of the present embodiment. The operation input unit 13 includes, for example, a two-axis joystick 31 for controlling the upper and lower and left and right axes of the head of the robot 20. The operation input unit 13 includes a line-of-sight recording button 32 for recording the line of sight of the robot 20. Moreover, the action recognition part 14 is the angle sensor 41 which is fixed to the head of the user of a remote place, and measures the attitude | position of the said head, for example.
The operation input unit 13 is configured to rotate the pitch axis, the yaw axis, and the roll axis of the robot 20 head in accordance with the operation rotation axis of the motion mechanism 23 of the head of the robot 20 and any of the front / back, up / down, and left / right translation operations. A plurality of joysticks 31 for inputting such an operation may be provided.
FIG. 5 is a flowchart for explaining the processing flow of the operation determination unit 16. The motion determination unit 16 controls the motion mechanism 23 so that the robot 20 sees the same kind of object at the site when the user at the remote location views the object at the remote location. For this reason, the action determination unit 16 executes a process of associating the action of the user at the remote place with the action of the robot 20 and the process of controlling the action of the robot 20 based on the recognition result of the action of the user at the remote place. .
The operation determination unit 16 determines whether or not there is an operation input of the joystick 31 (S1). When the input of the joystick 31 is detected (Y in S1), the operation determination unit 16 outputs a control command for instructing the action to the operation control unit 24 of the robot 20 in accordance with the input of the joystick 31 (S2). Specifically, when the joystick 31 is input in the upward direction, the motion determining unit 16 transmits a control command for moving the head of the robot 20 upward.
The motion determination unit 16 may change the motion speed of the head of the robot 20 according to the tilt amount of the joystick 31. Further, the motion determination unit 16 may use the tilt amount of the joystick 31 as the motion angle of the head of the robot 20.
As the next process, the operation determination unit 16 determines whether the line-of-sight recording button 32 has been pressed (S3). When the line-of-sight recording button 32 is pressed (Y in S3), the operation determination unit 16 generates behavior sensing information based on the measurement value input from the behavior recognition unit 14 at the timing of pressing. At the same time, the same unit reads the posture data of the motion mechanism 23 of the robot 20 (S4), associates both, and records them in the action association DB 15 (S5).
The action determination unit 16 changes the measurement value of the angle sensor 41 at the time when the line-of-sight recording button 32 is pressed, the change of the measurement value within a predetermined time (for example, several seconds) before the time point, or the holding time of the same measurement value. Action sensing information based on the above is generated.
An example of how to use the line-of-sight recording button 32 is as follows. As a first step, the user operates the joystick 31 or the like in advance so that the line-of-sight display means 28 on the head of the robot 20 faces a television screen or the like (second object) in the field in advance. When the robot 20 includes the video input unit 22, the user can easily perform this operation while viewing the image captured by the video input unit 22 through the video presentation unit 12 of the operation device 10.
As a second step, a user at a remote location presses the line-of-sight recording button 32 with his head facing the direction of a television or the like (first object) installed near the user. As described above, when the user presses the line-of-sight recording button 32, the posture data (robot control information 54) as an action of the robot 20 watching the TV screen at the site and the action of the remote user watching the TV screen at the remote place. The measured values of the head angle sensor 41 and the like (behavior sensing information 53) are associated with each other and recorded in the behavior association DB 15.
Next, the operation determination unit 16 inputs the current measurement value of the angle sensor 41 and the like, and compares whether the behavior sensing information based on the measurement value matches any of the behavior sensing information 53 registered in the behavior association DB 15. (S6). At this time, the motion determination unit 16 may determine a complete match, or whether the current behavior sensing information obtained from the angle sensor 41 is closer to the value registered in the behavior association DB 15 than the predetermined value. The approximate match may be determined by By this process, the operation determination unit 16 determines that the user at the remote location has taken the action of looking at the first target object at the remote location.
When there is matching action sensing information 53 (Y in S6), the action determination unit 16 takes action described by the robot control information 54 registered in the action association DB 15 corresponding to the action sensing information 53. A control command for output to the operation control unit 24.
With this process, the robot 20 performs an action of viewing the second object on the spot that is associated with the first object at the remote place viewed by the user at the remote place.
If the action association DB 15 in which the action sensing information 53 and the robot control information 54 are already stored is prepared in advance, the line-of-sight recording button 32 may not be provided. Furthermore, the operation determination unit 16 does not have to implement the process (S3 to S5) when the line-of-sight recording button 32 is pressed.
Note that the behavior of the robot 20 (description content of the robot control information 54) need not be the same as the user's behavior (description content of the behavior sensing information 53). That is, the moving direction of the user's head (up / down / left / right, etc.) and the moving direction of the head of the robot 20 may be different, or the moving angle may be different.
The robot system 90 of the present invention makes it easy for the user to cause the robot 20 to perform a predetermined operation through his / her own operation. The reason is that when the controller device 10 recognizes that the user has performed the behavior described by the behavior sensing information 53, the robot 20 causes the robot 20 to take the behavior described by the robot control device 54. The user can operate the robot 20 only by performing an action according to his / her purpose and desire.
Furthermore, the robot system 90 of the present invention facilitates communication between people in the field such as a conference room and users in other places such as remote locations.
For example, a display that displays the document A at the user's hand without the remote user pointing the head of the robot 20 at the site to the screen of the site displaying the document A (while keeping another direction) , The person at the site is not notified that the user at the remote location is viewing the document A. As a result, there is a problem that discussion between the user and the people on site cannot be smoothly conducted. The robot system 90 of the present invention prevents such a situation from occurring.
The reason is that the action association DB 15 uses the posture description data of the user's head facing the first object (such as a display at hand) as action sensing information 53 and the second object (site screen). This is because the posture description data of the robot 20 whose line of sight is directed to the robot control information 54 can be stored correspondingly.
<Second Embodiment>
FIG. 7 is a diagram illustrating a configuration of the operation input unit 13 and the action recognition unit 14 of the second embodiment. The action recognition unit 14 detects and recognizes a barcode from a camera 42 fixed to the user's head so that a user at a remote location can take a picture, and an image of the imaging result of the camera 42. The barcode recognition unit 43 outputs the reading result to the operation determination unit 16.
If a barcode is printed in advance on the materials distributed to the attendees of the meeting, the action recognition unit 14 recognizes the behavior that the user at the remote location has seen the material by reading the barcode. Is possible. In addition, if you paste objects such as a sheet with a barcode printed in advance on a remote object, such as a television or a whiteboard, the action that a remote user saw those objects, The operation determination unit 16 can determine by reading the barcode.
What is attached to the object may be not only a barcode sheet but also an infrared transmitter, a radio transmitter, a QR code, or the like. The action recognition unit 14 may be configured to recognize each tag. For example, the action recognition unit 14 may include an IC tag reader instead of the camera 42.
In addition, the robot 20 may be mounted with a camera on the head and detect a tag attached to the site side. When the robot 20 detects a specific tag with the action recognition unit 14 due to the action of a user at a remote location, the motion determination unit 16 can detect a predetermined tag with the camera 42 on the head of the robot 20. May be controlled. With this configuration, the action of the user or the robot 20 to see a certain object is realized by an operation of reading a tag attached to the object.
The action association DB 15 of the present embodiment may store, as the action sensing information 53, the value stored in the barcode or tag read by the action recognition unit 14 (see the record of ID = 3 in FIG. 6). ). The action association DB 15 may further store a barcode or tag value to be read by the robot 20 as the robot control information 54.
The operation device 10 according to the present embodiment expands the degree of freedom of user operation and the degree of freedom of action setting of the robot 20. The reason is that the camera 42 or the IC tag reader provided in the operation terminal 10 reads the barcode or tag to recognize the user's action. In addition, the IC tag reader or the like included in the robot 20 reads the barcode or tag to determine the behavior of the robot 20.
<Third Embodiment>
FIG. 8 is a diagram illustrating a configuration of the operation device 10 according to the third embodiment. The operating device 10 includes an action association DB 15 and an action determination unit 16. The behavior association DB 15 stores behavior sensing information 53 and robot control information 54 in association with each other.
The action determination unit 16 inputs a measurement value from a sensor that measures the user's action, searches the action association DB 15 for action sensing information 53 that matches the measurement value, and performs robot control corresponding to the searched action sensing information 53. A control command for taking an action described by the information 54 is output to the robot 20.
The robot system 90 of the present invention makes it easy for the user to cause the robot 20 to perform a predetermined operation through his / her own operation. The reason is that when the controller device 10 recognizes that the user has performed the behavior described by the behavior sensing information 53, the robot 20 causes the robot 20 to take the behavior described by the robot control device 54.
Each unit of the robot system 90 described above may be realized by hardware such as a logic circuit, or may be realized by a computer and software executed on the computer. The present invention is not limited to the above embodiment. Various changes that can be understood by those skilled in the art can be made to the configuration and details of the present invention within the scope of the present invention.
This application claims the priority on the basis of Japanese application Japanese Patent Application No. 2010-160440 for which it applied on July 15, 2010, and takes in those the indications of all here.

DESCRIPTION OF SYMBOLS 10 Operation apparatus 11, 25 Audio | voice presentation part 12 Image | video presentation part 13 Operation input part 14 Action recognition part 15 Action matching DB
DESCRIPTION OF SYMBOLS 16 Operation | movement determination part 17, 21 Voice input part 20 Robot 22 Image | video input part 23 Operation mechanism 28 Gaze display part 24 Operation control part 31 Joystick 32 Gaze recording button 41 Angle sensor 42 Camera 43 Bar code recognition part 51 ID
52 Action Recognition Type 53 Action Sensing Information 54 Robot Control Information 90 Robot System

Claims (10)

Action association storage means for storing action sensing information and robot control information in association with each other;
A measurement value is input from a sensor that measures a user's behavior, behavior sensing information that matches the measurement value is retrieved from the behavior association storage unit, and is described by robot control information corresponding to the retrieved behavior sensing information. An operation device comprising an operation determining means for outputting a control command for taking an action to the robot. The action association storage means uses a measured value of the head posture of the user whose face is directed to the first object as the action sensing information, and directs the line of sight to the second object. The operation device according to claim 1, wherein information describing the posture of the robot provided with is stored as the robot control information. The operation determination means inputs robot control information from the robot and a measurement value from the sensor at a predetermined timing, and stores the input measurement value and the robot control information in association with each other in the action association storage means. The operating device according to claim 1 or 2. The control command is input from the robot operation device connected via a network, and the head is rotated around one or more motion rotation axes according to the control command, and the motion rotation shafts are centered. Outputting the robot control information including one or more rotation angles, and
At the timing when the user performs a predetermined operation from the operation input means for instructing the operation of the robot, the user is fixed to the user's head and centered on one or more virtual motion rotation axes of the user's head A robot system comprising: the operation device according to claim 3, further comprising: the operation determining unit that inputs a measurement value from the sensor that measures one or more rotation angles and robot control information from the robot. The control command is input from the robot operation device connected via a network, and the head is rotated around one or more motion rotation axes according to the control command, and the motion rotation shafts are centered. Outputting the robot control information including one or more rotation angles, and
At the timing when the reader fixed in front of the user's head reads the data,
A robot system including the operation device according to claim 3, wherein the operation determining unit is configured to input the read data as a measurement value and input robot control information from the robot. On the computer,
A storage process in which the action sensing information and the robot control information are associated with each other and stored in the action association storage unit;
A measurement value is input from a sensor that measures a user's behavior, behavior sensing information that matches the measurement value is retrieved from the behavior association storage unit, and is described by robot control information corresponding to the retrieved behavior sensing information. An operation program for executing an action determination process for outputting a control command for taking an action to the robot. In the computer,
A line-of-sight display unit that directs the line of sight to the second object with the measured value of the head posture of the user facing the first object as the action sensing information. The operation program according to claim 6, wherein the storage process of storing information describing the posture of the robot including the robot control information is executed. In the computer,
At a predetermined timing, the robot control information is input from the robot, the measurement value is input from the sensor, and the action determination process is performed in which the input measurement value and the robot control information are associated with each other and stored in the action association storage unit. The operation program according to claim 6 or 7. Corresponding behavior sensing information and robot control information are stored in the behavior association storage means,
A measurement value is input from a sensor that measures a user's behavior, behavior sensing information that matches the measurement value is retrieved from the behavior association storage unit, and is described by robot control information corresponding to the retrieved behavior sensing information. An operation method that outputs control commands to the robot to take actions. A line-of-sight display unit that directs the line of sight to the second object with the measured value of the head posture of the user facing the first object as the action sensing information. The operation method according to claim 9, wherein information describing a posture of the robot provided with is stored as the robot control information.

Images