JP2025060080A - Training data generation device, training data generation method, and program - Google Patents

Abstract

To provide a learning data generation device, a learning data generation method, and a program, capable of improving the accuracy of movement of a robot even by the learning based on operation of an unskilled operator.SOLUTION: A learning data generation device 40 generates learning data for imitative learning of a robot 30. The learning data generation device 40 comprises an acquisition part 43 which acquires movement information when a first user operates the robot 30, and a reaction when a second user recognizes the movement information. The learning data generation device 40 further comprises a generation part 46 which generates learning data on the basis of the reaction of the second user and the movement information.SELECTED DRAWING: Figure 1

Description

The present disclosure relates to a training data generation device, a training data generation method, and a program.

Conventionally, there is known a device that learns how to operate a robot based on operation information that indicates the state of the robot and non-operation information that does not affect the state of the robot (see Patent Document 1).

International Publication No. 2016/189924

The accuracy of operations by an unskilled operator is lower than that of an experienced operator. Even if non-operation information is referenced, if the operator is not skilled in an operation, it is difficult to improve the accuracy of the robot's movements by learning based on that operation.

The objective of the present disclosure is to provide a learning data generation device, a learning data generation method, and a program that can improve the accuracy of a robot's movements even through learning based on the operations of an unskilled operator.

A training data generation device according to an embodiment of the present disclosure includes:
A learning data generation device for generating learning data for imitation learning of a robot, comprising:
an acquisition unit that acquires motion information when a first user operates the robot and a reaction of a second user when the second user recognizes the motion information;
A generation unit generates the learning data based on a reaction of the second user and the motion information.

A learning data generating method according to an embodiment of the present disclosure includes:
A learning data generation method for generating learning data for imitation learning of a robot, comprising the steps of:
acquiring operation information when a first user operates the robot and a reaction of a second user when the second user recognizes the operation information;
The learning data is generated based on a reaction of the second user and the motion information.

A program according to an embodiment of the present disclosure includes:
A program for operating a learning data generation device for learning a robot's movement, comprising:
causing the learning data generation device to acquire motion information when a first user operates the robot and a reaction of a second user when the second user recognizes the motion information;
The learning data is generated based on the reaction of the second user and the motion information.

According to a learning data generation device, a learning data generation method, and a program according to an embodiment of the present disclosure, the accuracy of a robot's movements can be improved even through learning based on the operations of an unskilled operator.

1 is a block diagram showing an example of the configuration of a robot control system according to an embodiment. FIG. 11 is a diagram showing an example of operation information of a robot. 2B is a diagram showing an example in which the operation information in FIG. 2A is corrected to the planned operation information by deleting the operation error portion. FIG. FIG. 2B is a diagram showing an example in which a part of the motion information in FIG. 2A that does not match the motion instructed by an expert is deleted and replaced with reference data. 2B is a diagram showing an example in which a portion of the motion information in FIG. 2A that does not match the motion instructed by an expert is deleted and interpolated. FIG. FIG. 1 is a diagram showing an example of a difference between the gaze of an expert and that of an unexpert. 1 is a flowchart illustrating an example of a procedure of a training data generating method according to an embodiment.

(Configuration example of robot control system 1)
As shown in FIG. 1, a robot control system 1 according to an embodiment includes a robot 30, a robot controller 20, a teaching data generating device 10, and a learning data generating device 40. At least one component of the robot control system 1 may be communicatively connected via a network, or may be communicatively connected without a network. At least one component of the robot control system 1 may be communicatively connected by wire or wirelessly. At least one component of the robot control system 1 may be communicatively connected via a dedicated line. At least one component of the robot control system 1 may be communicatively connected to each other in various other forms, without being limited to these examples. Each component of the robot control system 1 will be specifically described below. A configuration including the robot 30 and the robot controller 20 is also referred to as a robot system.

<Teaching Data Generating Device 10>
The teaching data generation device 10 includes a control unit 11, a storage unit 12, and a communication unit 13. The control unit 11 includes a learning unit 14. The teaching data generation device 10 acquires learning data from the learning data generation device 40 via the communication unit 13. The teaching data generation device 10 generates teaching data by performing learning using the learning data via the learning unit 14. The teaching data is information that causes the robot controller 20 to automatically control the robot 30 to perform a task.

The control unit 11 or the learning unit 14 may be configured to include at least one processor or dedicated circuit. The processor may execute a program that realizes various functions of the teaching data generating device 10. The processor may be realized as a single integrated circuit. The integrated circuit is also called an IC (Integrated Circuit). The processor may be realized as multiple communicatively connected integrated circuits and discrete circuits. The processor may be configured to include a CPU (Central Processing Unit). The processor may be configured to include a DSP (Digital Signal Processor) or a GPU (Graphics Processing Unit). The processor may be realized based on various other known technologies. The dedicated circuit may include, for example, an FPGA (Field-Programmable Gate Array) or an ASIC (Application Specific Integrated Circuit), etc.

The storage unit 12 may be configured to include an electromagnetic storage medium such as a magnetic disk, or may be configured to include a memory such as a semiconductor memory or a magnetic memory. The storage unit 12 may be configured as a HDD (Hard Disk Drive) or an SSD (Solid State Drive). The storage unit 12 stores various information and programs executed by the processor. The storage unit 12 may function as a work memory for the processor or a dedicated circuit. At least a part of the storage unit 12 may be included in the processor or the dedicated circuit. At least a part of the storage unit 12 may be configured as a storage device separate from the teaching data generation device 10.

The communication unit 13 may be configured to include a communication device configured to be able to communicate with other devices in a wired or wireless manner. The communication device may be configured to be able to communicate using a communication method based on various communication standards. Furthermore, the teaching data generation device 10 is communicatively connected to at least one component of the robot control system 1 via a line, whether wired or wireless. The teaching data generation device 10 and at least one component of the robot control system 1 are provided with an interface that uses a mutual standard protocol, and can communicate bidirectionally.

The teaching data generating device 10 may be configured to include one or more servers. The teaching data generating device 10 may be configured to cause multiple servers to execute parallel processing. The teaching data generating device 10 does not need to be configured to include a physical housing, and may be configured based on virtualization technology such as a virtual machine or a container orchestration system. The teaching data generating device 10 may be configured using a cloud service. When the teaching data generating device 10 is configured using a cloud service, it may be configured by combining managed services. In other words, the functions of the teaching data generating device 10 may be realized as a cloud service.

The teaching data generating device 10 may include at least one server and at least one database. The server functions as the control unit 11 or the learning unit 14. The database functions as the storage unit 12. The number of servers may be one or two or more. When the number of servers is one, the functions realized by one server include functions realized by multiple servers. The servers are connected to each other so as to be able to communicate with each other via wired or wireless communication. The number of databases may be one or two or more. The number of databases may be appropriately increased or decreased based on the capacity of data managed by the teaching data generating device 10 and the availability requirements required for the teaching data generating device 10. The databases are connected to each server so as to be able to communicate with each other via wired or wireless communication. The teaching data generating device 10 may be connected to an external database. A teaching data generating system may be configured that includes the teaching data generating device 10 and an external database.

Although the teaching data generating device 10 is shown as one configuration in FIG. 1, multiple configurations can be operated as one system as necessary. In other words, the teaching data generating device 10 is configured as a platform with variable capacity. By using multiple configurations as the teaching data generating device 10, even if one configuration becomes inoperable due to an unforeseen event such as a natural disaster, the operation of the system can continue using the other configurations. In this case, each of the multiple configurations is connected by a line, whether wired or wireless, and is configured to be able to communicate with each other. The multiple configurations may be constructed across cloud services and on-premise environments.

<Robot Controller 20>
The robot controller 20 acquires teaching data from the teaching data generation device 10 and controls the robot 30 based on the teaching data, thereby causing the robot 30 to perform a task. When the teaching data generation device 10 generates only scenario data as teaching data, the robot controller 20 may generate source code capable of controlling the robot 30 based on the scenario data.

The robot controller 20 may, for example, cause the robot 30 to perform the task of reading a meter installed in a facility such as a factory. The robot controller 20 may cause the robot 30 to perform various tasks, including but not limited to this. The robot controller 20 may or may not be connected to a cloud computing environment. When the robot controller 20 is not connected to a cloud computing environment, the operation of the robot controller 20 is completed in an on-premise environment. When the operation of the robot controller 20 is completed in an on-premise environment, the operation of the teaching data generation device 10 may be executed by the robot controller 20.

The robot controller 20 has a control unit 21, a memory unit 22, and an input unit 23. The control unit 21 can store teaching data and the like acquired from the teaching data generating device 10 in the memory unit 22, and can control the robot 30 based on the acquired teaching data. The control unit 21 can also control each function of the robot controller 20. The memory unit 22 can store information necessary for controlling the robot 30 or the robot controller 20 itself.

The input unit 23 accepts input of a command to manually operate the robot 30. When a command to manually operate the robot 30 is input to the input unit 23, the control unit 21 controls the robot 30 based on the input command. As a result of controlling the robot 30, the control unit 21 generates information indicating how the robot 30 operated, and outputs the information to the learning data generation device 40. The information indicating how the robot 30 operated is also referred to as operation information.

The control unit 21 may be configured to include at least one processor or dedicated circuit. The processor may execute a program that realizes various functions of the teaching data generating device 10. The processor may be realized as a single integrated circuit. The integrated circuit is also called an IC (Integrated Circuit). The processor may be realized as multiple communicatively connected integrated circuits and discrete circuits. The processor may be configured to include a CPU (Central Processing Unit). The processor may be configured to include a DSP (Digital Signal Processor) or a GPU (Graphics Processing Unit). The processor may be realized based on various other known technologies. The dedicated circuit may include, for example, an FPGA (Field-Programmable Gate Array) or an ASIC (Application Specific Integrated Circuit), etc.

The storage unit 22 may be configured to include an electromagnetic storage medium such as a magnetic disk, or may be configured to include a memory such as a semiconductor memory or a magnetic memory. The storage unit 22 may be configured as a HDD (Hard Disk Drive) or an SSD (Solid State Drive). The storage unit 22 stores various information and programs executed by the processor. The storage unit 22 may function as a work memory for the processor or a dedicated circuit. At least a part of the storage unit 22 may be included in the processor or the dedicated circuit. At least a part of the storage unit 22 may be configured as a storage device separate from the robot controller 20.

The input unit 23 may include an input device. The input device may include, for example, a joystick. The input device may include, for example, a touch panel or a touch sensor, or a pointing device such as a mouse. The input device may include physical keys. The input device may include a voice input device such as a microphone. The input device is not limited to these examples and may include various other devices.

The input unit 23 may be realized as an input terminal separate from the robot controller 20. The number of input terminals is not limited to one, and may be two or more. The input terminal may be configured as a tablet terminal, or as a PC terminal such as a desktop PC (Personal Computer) or a notebook PC. The input terminal is not limited to these examples, and may be configured as various devices.

The robot controller 20 may be equipped with a communication device that acquires teaching data from the teaching data generating device 10. The communication device of the robot controller 20 may be configured in the same manner as or similarly to the communication device of the teaching data generating device 10.

In the configuration illustrated in FIG. 1, one robot controller 20 is connected to one robot 30. One robot controller 20 may be connected to two or more robots 30. One robot controller 20 may control only one robot 30, or may control two or more robots 30. The number of robot controllers 20 and robots 30 is not limited to one, and may be two or more.

<Robot 30>
The robot 30 may be configured as a robot arm having an arm. The arm may be configured as, for example, a 6-axis or 7-axis vertical articulated robot. The arm may be configured as a 3-axis or 4-axis horizontal articulated robot or a SCARA robot. The arm may be configured as a 2-axis or 3-axis Cartesian robot. The arm may be configured as a parallel link robot or the like. The number of axes constituting the arm is not limited to those exemplified.

The robot 30 may include an end effector attached to the arm. The end effector may include, for example, a gripping hand configured to grip a work object. The gripping hand may have multiple fingers. The number of fingers on the gripping hand may be two or more. The fingers on the gripping hand may have one or more joints. The end effector may include a suction hand configured to pick up the work object. The end effector may include a scooping hand configured to scoop up the work object. The end effector may include a tool such as a drill and be configured to perform various processes such as drilling holes in the work object. The end effector is not limited to these examples and may be configured to perform various other operations.

The robot 30 can control the position of the end effector by operating the arm. The end effector may have an axis that is a reference for the direction in which it acts on the workpiece. If the end effector has an axis, the robot 30 can control the direction of the axis of the end effector by operating the arm. The robot 30 controls the start and end of the operation of the end effector acting on the workpiece. The robot 30 can move or process the workpiece by controlling the operation of the end effector while controlling the position of the end effector or the direction of the axis of the end effector.

The robot 30 may include, for example, an automated guided vehicle (AGV), an autonomous mobile robot (AMR), or a drone. The robot 30 is not limited to these, and may be configured in various other forms, such as a vehicle, electronic device, or controlled machine.

The robot 30 may further include a sensor that detects the state of at least one component of the robot 30. The sensor may detect information about the actual position or posture of at least one component of the robot 30, or the speed or acceleration of at least one component of the robot 30. The sensor may detect a force acting on at least one component of the robot 30. The sensor may detect a current flowing through a motor that drives at least one component of the robot 30, or a torque of the motor. The sensor can detect information obtained as a result of the actual operation of the robot 30. The robot controller 20 can grasp the result of the actual operation of the robot 30 by acquiring the detection result of the sensor.

<Learning Data Generating Device 40>
The learning data generating device 40 includes a control unit 41, a storage unit 42, an acquisition unit 43, and an output unit 44. The control unit 41 includes an analysis unit 45 and a generation unit 46. The learning data generating device 40 acquires, via the acquisition unit 43, from the robot controller 20, operation information of the robot 30 when an operator manually operates the robot 30. The learning data generating device 40 outputs, via the output unit 44, the operation information of the robot 30, and allows a user other than the operator to view a video representing the operation information. The operator is also referred to as a first user. A user who views the video representing the operation information is also referred to as a second user.

The learning data generating device 40 acquires, by the acquisition unit 43, the reaction of the second user when the second user watches the video representing the motion information. The learning data generating device 40 analyzes the reaction of the second user by the analysis unit 45. The learning data generating device 40 generates learning data by the generation unit 46 based on the analysis result of the reaction of the second user and the motion information. The learning data generating device 40 outputs the generated learning data from the output unit 44 to the teaching data generating device 10.

The control unit 41 may be configured to include at least one processor or dedicated circuit. The processor may execute a program that realizes various functions of the training data generation device 40. The control unit 41 may be configured in the same manner as or similarly to the control unit 11 of the teaching data generation device 10.

The memory unit 42 stores various information used by the learning data generating device 40 and programs executed by the processor. The memory unit 42 may be configured in the same manner as or similarly to the memory unit 12 of the teaching data generating device 10.

The acquisition unit 43 and the output unit 44 are configured to be capable of wired or wireless communication with other devices. The acquisition unit 43 and the output unit 44 may be configured in the same manner as or similar to the communication unit 13 of the teaching data generation device 10.

The acquisition unit 43 may include a device that acquires the state when the first user is operating the robot 30. The acquisition unit 43 may include a camera that captures the environment in which the robot 30 is operating or the first user. The acquisition unit 43 may include a microphone that captures the sound of the environment in which the robot 30 is operating or the contents of the speech of the first user. The acquisition unit 43 may include a 360-degree camera that is worn on the head of the first user. The 360-degree camera may capture video to reproduce the state of the first user when operating the robot 30. The acquisition unit 43 may include a sensor that detects the line of sight of the first user.

The acquisition unit 43 may include a device that acquires the reaction of the second user when the second user is viewing the operation information when the first user is operating the robot 30. The acquisition unit 43 may include a microphone that acquires the contents of the speech of the second user. The acquisition unit 43 may include a sensor that detects the line of sight of the second user.

The acquisition unit 43 may include an input device that accepts input of instructions or advice from the second user in response to the operation of the first user. The input device may include, for example, a touch panel or a touch sensor, or a pointing device such as a mouse. The input device may include physical keys. The input device may include a voice input device such as a microphone. The input device is not limited to these examples and may include various other devices.

The input device may be realized as an input terminal separate from the learning data generation device 40. The number of input terminals is not limited to one, and may be two or more. The input terminal may be configured as a tablet terminal, or as a PC terminal such as a desktop PC (Personal Computer) or a notebook PC. The input terminal is not limited to these examples, and may be configured as various devices.

The output unit 44 may include a display device. The display device may be configured to include, for example, a liquid crystal display (LCD), an organic electroluminescence (EL) display or an inorganic electroluminescence (EL) display, or a plasma display panel (PDP). The display device is not limited to these displays, and may be configured to include various other types of displays. The display device may be configured to include a light-emitting device such as an LED (Light Emitting Diode). The display device may be configured to include various other devices.

The output unit 44 may be configured to include an audio output device such as a speaker that outputs auditory information such as voice. The output unit 44 may be configured to include a device that provides a VR (Virtual Reality) environment. The output unit 44 is not limited to these examples and may be configured to include various other devices.

The training data generating device 40 may be configured to include one or more servers. The training data generating device 40 may be configured to cause multiple servers to execute parallel processing. The training data generating device 40 does not need to be configured to include a physical housing, and may be configured based on virtualization technology such as a virtual machine or a container orchestration system. The training data generating device 40 may be configured using a cloud service. When the training data generating device 40 is configured using a cloud service, it may be configured by combining managed services. In other words, the functions of the training data generating device 40 may be realized as a cloud service.

The learning data generating device 40 may include at least one server and at least one database. The server functions as a control unit 41. The database functions as a storage unit 42. The number of servers may be one or may be two or more. When the number of servers is one, the functions realized by one server include functions realized by multiple servers. The servers are connected to each other so as to be able to communicate with each other via wired or wireless communication. The number of databases may be one or may be two or more. The number of databases may be appropriately increased or decreased based on the capacity of data managed by the learning data generating device 40 and the availability requirements required for the learning data generating device 40. The databases are connected to each server so as to be able to communicate with each other via wired or wireless communication. The learning data generating device 40 may be connected to an external database. A teaching data generating system may be configured that includes the learning data generating device 40 and an external database.

Although the training data generation device 40 is shown as one configuration in FIG. 1, multiple configurations can be operated as one system as necessary. In other words, the training data generation device 40 is configured as a platform with variable capacity. By using multiple configurations as the training data generation device 40, even if one configuration becomes inoperable due to an unforeseen event such as a natural disaster, the operation of the system can continue using the other configurations. In this case, each of the multiple configurations is connected by a line, whether wired or wireless, and is configured to be able to communicate with each other. The multiple configurations may be constructed across cloud services and on-premise environments.

(Operation example of robot control system 1)
In the robot control system 1, the robot controller 20 accepts manual operation of the robot 30 by a first user through the input unit 23. The robot controller 20 controls the robot 30 based on the manual operation of the first user, and generates operation information when the robot 30 is manually operated by the first user. The robot controller 20 outputs the operation information of the robot 30 to the learning data generation device 40. The input unit 23 may be remotely operated.

The learning data generating device 40 acquires the motion information of the robot 30. The learning data generating device 40 has the second user watch a video representing the motion information of the robot 30, and acquires the second user's reaction. The learning data generating device 40 generates learning data based on the results of analyzing the reaction of the second user and the motion information, and outputs the learning data to the teaching data generating device 10. Specifically, the learning data can be generated by performing an operation of extracting a portion of the motion information of the robot 30 as learning data based on the results of analyzing the reaction of the second user, or an operation of modifying the motion information so that it can be used as learning data, etc.

The teaching data generating device 10 uses the learning data to perform learning so as to be able to imitate the behavior of the robot 30 when the first user manually operates the robot 30, and generates teaching data for the robot 30. Specifically, the teaching data generating device 10 can perform machine learning of the robot 30 by labeling information such as the operating environment of the robot 30 with the acquired learning data. Note that information such as the operating environment of the robot 30 may be acquired as appropriate. Learning that imitates the behavior of the robot 30 when the first user manually operates the robot 30 is also called imitation learning. The robot controller 20 generates teaching data using a learned model generated by learning, and automatically controls the robot 30 so as to imitate the behavior of the robot 30 operated by the first user based on the teaching data. Specifically, when the robot 30 is operated autonomously, for example, information such as the operating environment of the robot 30 may be input to the learned model, and teaching data may be output. Note that the learned model may be, for example, an AI such as a neural network.

In the robot control system 1 according to the present disclosure, even if the first user is an unskilled operator, learning data is generated based on the reaction of the second user who is a skilled operator. A skilled operator is also called an expert. An unskilled operator is also called an unskilled. By generating learning data based on the reaction of an expert, learning that imitates the operation of the robot 30 by an expert is performed simply by watching a video showing the operation information of the robot 30 when operated by an unskilled person, without the expert operating the robot 30. As a result, the accuracy of the teaching data generated by learning is improved. A specific operation example of the learning data generation device 40 is described below.

<Information acquired by the learning data generating device 40>
As described above, the learning data generation device 40 acquires, by the acquisition unit 43, the operation information of the robot 30 when the first user operates the robot 30. The acquisition unit 43 may acquire not only the operation information of the robot 30 but also various other information.

The acquisition unit 43 may acquire information on, for example, humans or objects present in the location where the robot 30 is operating. In other words, the acquisition unit 43 may acquire information representing the environment in which the robot 30 is operating. Information representing the environment in which the robot 30 is operating is also referred to as environmental information. The environmental information may include video such as an overhead video of the environment in which the robot 30 is operating. The environmental information may include audio that can be heard in the environment in which the robot 30 is operating, such as the operating sound of the robot 30, the operating sound of the input unit 23 of the robot controller 20, the sound of other objects, or the speech of a human other than the first user.

The acquisition unit 43 may acquire information regarding the state of the first user while operating the robot 30, for example. The information regarding the state of the first user is also referred to as state information. The state information may include the content of the first user's speech. The state information may include the first user's line of sight. The state information may include biometric information representing the first user's physical and mental state, such as the first user's heart rate. The state information may include a video of the first user.

<Acquisition of a response from a second user>
The learning data generation device 40 outputs the information acquired by the acquisition unit 43 through the output unit 44 to allow the second user to recognize the information and experience the operation of the first user. For example, the output unit 44 may display a video showing the operation information of the robot 30 to allow the second user to view the video. The output unit 44 may output at least a part of the environmental information or state information to allow the second user to recognize the information. The output unit 44 may output the speech content or the line of sight detection result of the first user as the state information.

When the first user is an unskilled person, the motion information of the robot 30 operated by the first user may include motions different from those of the robot 30 operated by the second user. Therefore, while watching the video of the motion information of the robot 30 operated by the first user, the second user may point out by speech or input to an input device the differences from the motion of the robot 30 operated by the second user himself. In addition, the second user may speak or move his body or eyes involuntarily due to a sense of immersion when recognizing the video of the motion information of the robot 30 or the environmental information or state information. The motions performed by the second user while watching the video of the motion information of the robot 30 or recognizing the environmental information or state information are also collectively referred to as the reaction of the second user. The learning data generation device 40 acquires the reaction of the second user by the acquisition unit 43.

The three modes illustrated below may be prepared as environments in which an expert second user evaluates an operation performed by an unskilled first user. The environment in which the second user performs evaluation is not limited to the illustrated modes, and various other modes may be prepared.

<<VR experience of operation information of robot 30>>
In the first mode, the output unit 44 may output the operation information of the robot 30 to a VR device so that a second user in a remote location can VR experience an environment in which the first user operates the robot 30. The output unit 44 may output the operation information so that the second user can view the video of the operation information of the robot 30 from a viewpoint of his/her choice regardless of the line of sight of the first user when the first user operated the robot 30.

The acquisition unit 43 detects the second user's gaze or the second user's reaction, such as speech, while experiencing the first user's operation in VR. The second user's reaction at this time is used to evaluate the operation information of the robot 30 as a result of the first user's operation.

<<Recognition of differences in gaze or speech between a first user and a second user>>
In a second mode, the output unit 44 may output the operation information and environmental information or status information of the robot 30 via a VR device so that the second user can further experience VR with the environmental information or status information in addition to the operation information of the robot 30.

The acquisition unit 43 detects the second user's gaze or the second user's reaction, such as speech, when the second user is experiencing the operation information of the robot 30 as operated by the first user in VR, and the environmental information or status information.

The output unit 44 may play back the gaze or speech of the second user acquired when the second user watches the video in the first mode so that it can be compared with the gaze or speech of the first user included in the state information. In other words, the output unit 44 may output information so that the second user can recognize the difference in gaze or speech between the first user and the second user.

The acquisition unit 43 detects the reaction of the second user to the difference in gaze or speech between the first user and the second user. The output unit 44 may prompt the second user to explain the difference between the first user and the second user by consciously speaking or inputting into an input device, rather than by a natural reaction. The explanation may be given in natural language. The acquisition unit 43 may acquire the explanation spoken or input by the second user in natural language. The acquisition unit 43 may perform language analysis of the explanation to acquire the analysis result.

<<Correction of Action by Second User>>
The output unit 44 may output the overhead video to a display or the like other than a VR device so that the second user can view the overhead video of the environment in which the robot 30 operates. The output unit 44 may display, in the overhead video, the movement path of the robot 30 when operated by the first user.

The output unit 44 may prompt the second user to input the motion path of the robot 30 that the second user envisions when operating the robot 30. The acquisition unit 43 may acquire the motion path that the second user envisions, input by the second user. In other words, the acquisition unit 43 may acquire information that the second user corrects the motion path of the robot 30 when operated by the first user. The corrections by the second user are included in the reaction of the second user. The output unit 44 may display the motion path of the robot 30 when operated by the first user and the motion path that the second user envisions so that they can be compared.

<Analysis based on second user responses>
The learning data generation device 40 analyzes the reaction of the second user by the analysis unit 45. The analysis unit 45 may, for example, determine, based on the reaction of the second user, a portion of the operation information of the robot 30 that can be used as is in the learning data and a portion that cannot be used as is in the learning data. The analysis unit 45 may determine, for a portion that cannot be used as is in the learning data, whether the portion can be used in the learning data if the portion is modified. The analysis unit 45 may determine, for a portion that cannot be used as is in the learning data, whether the portion needs to be deleted and replaced with another operation.

The analysis unit 45 may analyze the second user's evaluation of the first user's operation based on the tone or strength of the second user's voice when watching the video of the operation information of the robot 30 as operated by the first user.

The evaluation result may include, for example, denying the operation of the first user. The analysis unit 45 determines that the motion information corresponding to the denied operation is a part that cannot be used as is in the learning data.

The evaluation result may include, for example, sending improvement advice for the first user's operation. The analysis unit 45 may respect the first user's operation and determine that the operation information corresponding to the operation for which improvement advice has not been received is a part that can be used as is in the learning data. The analysis unit 45 may determine that the operation information corresponding to the operation for which improvement advice has been received is a part that cannot be used as is in the learning data.

The evaluation result may include, for example, praising or affirming the operation of the first user. The analysis unit 45 determines that the motion information corresponding to the operation that was praised or affirmed is a part that can be used as is in the learning data.

The analysis unit 45 may analyze the evaluation results of the second user by applying the second user's utterance to a predetermined relationship, for example, as shown in Table 1.

The analysis unit 45 may classify the second user's utterance into four categories: criticism, encouragement, advice, minor instructions, or major improvement. The analysis unit 45 may compare the utterance of the second user with an example utterance corresponding to each category. The analysis unit 45 may analyze the evaluation result of the second user based on the effect that the utterance corresponding to each category has on the operation data of the first user corresponding to the motion information.

The analysis unit 45 may analyze the evaluation result of the second user based on the ending of the second user's utterance. For example, the analysis unit 45 may analyze that an utterance ending with "You should do this" or "You should do this" corresponds to advice. The analysis unit 45 may analyze that an utterance ending with "Please do this" or "Please do this" corresponds to instructions. The analysis unit 45 is not limited to these examples and may analyze the second user's utterance based on various utterance patterns.

<Accumulation of reference data>
The analysis unit 45 may analyze the corrections made by the second user and generate reference data of the operation of the robot 30 when operated by an expert as a sample. The analysis unit 45 may generate a library that accumulates the reference data and store it in the storage unit 42. The reference data may be used to replace a part of the operation information of the robot 30 that cannot be used as is in the learning data.

<Generating learning data>
The generation unit 46 of the learning data generation device 40 generates learning data based on the operation information of the robot 30 acquired by the acquisition unit 43 and the analysis result by the analysis unit 45. The generation unit 46 outputs the generated learning data to the teaching data generation device 10.

<<Changes in behavior in response to operational mistakes>>
For example, as shown in Fig. 2A, it is assumed that the acquisition unit 43 acquires the motion information of the robot 30 including the operation error of the first user. Meanwhile, the motion path of the robot 30 originally planned by the first user is shown by a dashed line as the planned motion information. The motion of the robot 30 passing a point where the planned path is bent and returning corresponds to the operation error of the first user.

The analysis unit 45 may determine that the operation information of the robot 30 includes an operation corresponding to an operation error by the first user based on the state information of the first user. The analysis unit 45 may identify an operation corresponding to an operation error by the first user from among the operation information of the robot 30 based on the state information of the first user. The analysis unit 45 may identify an operation corresponding to an operation error by the first user, for example, based on the first user uttering an utterance that the first user is aware of the error. The analysis unit 45 may identify an operation corresponding to an operation error by the first user, for example, based on an increase in the heart rate of the first user. The analysis unit 45 may determine that the operation information of the robot 30 includes an operation corresponding to an operation error by the first user based on the reaction of the second user. The analysis unit 45 may identify an operation corresponding to an operation error by the first user based on the reaction of the second user.

The generation unit 46 acquires information about the operation corresponding to the operation error of the first user identified by the analysis unit 45. As illustrated in FIG. 2B, the generation unit 46 may delete the part of the operation information of the robot 30 that corresponds to the operation error of the first user, and generate the information changed to the planned operation information as learning data.

<<Replacing movements that deviate from those performed by an expert>>
In Fig. 2A, the path of the robot 30 when the second user operates the robot 30 is shown by a dashed line as the motion instructed by an expert. The planned path of the first user is significantly different from the path operated by the second user. In other words, the motion of the robot 30 operated by the first user deviates from the motion of the second user who is an expert. The motion of the second user who is an expert is considered to be suitable as learning data.

The analysis unit 45 may determine, based on the reaction of the second user, that the motion information of the robot 30 includes a motion that deviates from a motion resulting from the operation of the second user. The analysis unit 45 may identify, based on the reaction of the second user, a motion among the motion information of the robot 30 that deviates from a motion resulting from the operation of the second user.

For example, the analysis unit 45 may estimate that the movement intended by the second user is the movement represented by the dashed line based on the second user's utterance that the robot 30 should be moved diagonally in relation to the movement information of the robot 30 shown in FIG. 2A. The analysis unit 45 may identify, among the movement information of the robot 30, any movement that deviates from the movement intended by the second user's operation by comparing the movement information of the robot 30 operated by the first user with the estimated result of the movement intended by the second user.

The analysis unit 45 may adopt the path input by the second user as the motion of the robot 30 operated by the second user, based on the second user inputting a path for moving the robot 30 diagonally. The analysis unit 45 may identify, among the motion information of the robot 30, motions that deviate from the motions operated by the second user by comparing the motion information of the robot 30 operated by the first user with the motions input by the second user.

The generation unit 46 acquires, from the motion information of the robot 30 identified by the analysis unit 45, motions that deviate from motions resulting from the operation of the second user. As illustrated in FIG. 2C or FIG. 2D, the generation unit 46 may delete, from the motion information of the robot 30, motions that deviate from motions resulting from the operation of the second user, and leave motions represented by solid lines as valid portions.

When there is reference data corresponding to an action that connects the remaining valid actions, the generation unit 46 may generate information in which the deleted action is replaced by the reference data as learning data, as illustrated in FIG. 2C.

When replacing a deleted action, the generation unit 46 may select or customize a replacement action taking into consideration environmental information. For example, the generation unit 46 may select or customize a replacement action so as to avoid a predicted danger in the current environment, taking into consideration a difference between the environment when the reference data was acquired and the current environment. Even if reference data exists, the generation unit 46 may replace the deleted action with another action rather than replacing it with the reference data.

Even if reference data exists, the generation unit 46 may determine that it is not possible to replace the reference data as a result of taking into account predicted dangers in the current environment.

When there is no reference data corresponding to the motion connecting the remaining valid motions, or when the generation unit 46 determines that the motion cannot be replaced by reference data, the generation unit 46 generates information as learning data in which the deleted motion is left blank without replacing it with another motion, as exemplified in FIG. 2D. The teaching data generated by learning using the learning data including blank portions may be configured to leave the motion in the blank portion to the discretion of the robot 30.

<< Weighting Settings >>
The generating unit 46 sets weighting when generating learning data based on the operation information of the robot 30. The weighting of the learning data is set in order to identify characteristic data among a plurality of data included in the learning data. The weighting in the operation information of the robot 30 is set for the operation of the robot 30. The operation reproduced by the teaching data generated by learning using the operation information to which weighting is set as learning data is made to approach the operation to which a heavy weighting is set.

The analysis unit 45 may set a weighting for the movement information of the robot 30 based on the operation of the first user. The analysis unit 45 may divide the movement information into a plurality of movements and set a weighting for each divided movement.

The analysis unit 45 may set a weighting on the operation information of the robot 30 based on the content of the first user's utterance or the biometric information of the first user, etc. The analysis unit 45 may set a weighting on the operation information of the robot 30 based on the reaction of the second user.

For example, as shown in FIG. 3, the analysis unit 45 may set a weighting on the motion information of the robot 30 based on a comparison between the gaze of the second user who is an unskilled person and the gaze of the first user who is an expert. In the example of FIG. 3, the gaze of the unskilled person is at the current position of the robot 30. On the other hand, the gaze of the expert is further along the planned path of the robot 30 than the current position of the robot 30. The analysis unit 45 may focus on the gaze of the expert and set a light weighting on the motion information of the robot 30 that is not in the gaze of the expert, and a heavy weighting on the motion that is in the gaze of the expert.

<Generation of teaching data>
The teaching data generation device 10 acquires learning data from the learning data generation device 40 via the communication unit 13. The teaching data generation device 10 executes learning using the acquired learning data via the learning unit 14, and generates teaching data for operating the robot 30 under automatic control. The teaching data generation device 10 outputs the generated teaching data to the robot controller 20 via the communication unit 13. The teaching data generation device 10 may store the generated teaching data in the memory unit 12.

<Control of Robot 30>
The control unit 21 of the robot controller 20 controls the robot 30 based on the teaching data generated by the teaching data generating device 10. The control unit 21 converts the teaching data into, for example, control commands for the motors of the robot 30, and controls the robot 30 by transmitting the control commands to the robot 30, causing the robot 30 to execute actions that mimic actions performed by the first user.

<Example of procedure for generating training data>
The training data generation device 40 may execute a training data generation method including the steps of the flowchart illustrated in Fig. 4. The training data generation method may be realized as a training data generation program executed by a processor constituting the control unit 41 or the like of the training data generation device 40. The training data generation program may be a program that operates the training data generation device 40. The training data generation program may be stored in a non-transitory computer-readable medium.

The acquisition unit 43 acquires, from the robot controller 20, operation information of the robot 30 operated by the first user who is an unskilled person (step S1). The output unit 44 plays the operation information of the robot 30 (step S2). Specifically, the output unit 44 may display a video of the operation information of the robot 30 to be viewed by the second user who is an expert. The acquisition unit 43 acquires the reaction of the expert when the second user who is an expert views the operation information of the robot 30 (step S3).

Based on the reaction of the second user, who is an expert, the control unit 41 determines whether there is anything to be changed in the operation information of the robot 30 (step S4). The change object is a part of the operation information of the robot 30 that cannot be used as is in the learning data. If there is no change object (step S4: NO), the control unit 41 proceeds to the procedure of step S9.

If there is a change target (step S4: YES), the control unit 41 determines whether there is a replacement target (step S5). The replacement target may be an action among the change targets that deviates significantly from the action performed by the second user who is an expert. If there is no replacement target (step S5: NO), the control unit 41 proceeds to the procedure of step S8.

If there is a replacement target (step S5: YES), the control unit 41 determines whether there is reference data to replace the replacement target (step S6). If there is reference data (step S6: YES), the control unit 41 replaces the replacement target with the reference data (step S7). If there is no reference data (step S6: NO), the control unit 41 does not replace the replacement target and proceeds to the procedure of step S8.

The control unit 41 changes the replacement target (step S8). If there are replacement targets that have not been replaced, the control unit 41 may delete the replacement targets or may change the replacement targets.

The control unit 41 sets a weighting to the motion information (step S9). When the control unit 41 changes or replaces the motion information, it sets a weighting to the changed or replaced motion information. The control unit 41 generates learning data by executing the procedure up to step S9, and outputs the generated learning data to the teaching data generating device 10. After executing the procedure of step S9, the learning data generating device 40 ends the execution of the procedure in the flowchart of FIG. 4.

(summary)
As described above, in the robot control system 1 according to this embodiment, the learning data generation device 40 can correct the motion information of the robot 30 operated by the first user who is an unskilled person, based on the reaction of the second user who is an expert. As a result, the accuracy of the motion of the robot 30 can be improved even by learning based on the operation of the unskilled person. Even if an expert is not present at the site where the robot 30 is operated, the knowledge of the expert can be applied to the learning of the robot 30.

Other Embodiments
Hereinafter, configuration examples of a robot control system 1 and a learning data generating device 40 according to another embodiment will be described.

Information regarding the reaction of the second user, who is an expert, may be fed back to the first user, who is an unskilled person. The first user can improve the operation of the robot 30 by comparing the first user's own operation with the operation pointed out by the second user. In addition, the first user can understand which operations of the robot 30 should be particularly noted among the series of operations of the robot 30 by checking the weighting in the operation information of the robot 30, and improve the operation of the robot 30.

The first user may be a worker who actually uses the robot 30 at the customer of the robot system. The second user may be an engineer or trainer of the seller of the robot system. According to the learning data generation device 40 of the present disclosure, even if the worker at the customer of the robot system is not yet skilled, a skilled engineer or trainer can correct the motion information when the worker operates the robot 30. As a result, the engineer or trainer of the seller of the robot system can improve the accuracy of the robot 30's motion without visiting the customer of the robot system. In addition, training of the worker at the customer of the robot system is effectively carried out.

The above describes an embodiment of the robot control system 1. However, the present disclosure can also be embodied as a method or program for implementing the system or device, or as a storage medium on which a program is recorded (for example, an optical disk, a magneto-optical disk, a CD-ROM, a CD-R, a CD-RW, a magnetic tape, a hard disk, or a memory card). The program may be stored in a non-transitory computer-readable medium.

The form of implementation of the program is not limited to application programs such as object code compiled by a compiler or program code executed by an interpreter, but may be a program module incorporated into an operating system, etc. Furthermore, the program may or may not be configured so that all processing is performed only by the CPU on the control board. The program may be configured so that a part or all of it is executed by another processing unit mounted on an expansion board or expansion unit added to the board as necessary.

Although the embodiments of the present disclosure have been described based on the drawings and examples, it should be noted that a person skilled in the art would be able to make various modifications or alterations based on the present disclosure. Therefore, it should be noted that these modifications or alterations are included in the scope of the present disclosure. For example, the functions, etc. included in at least one component, etc. can be rearranged so as not to cause logical inconsistencies, and multiple components, etc. can be combined into one or divided.

In the above example, the learning data generating device 40, the teaching data generating device 10, and the robot controller 20 are provided separately. As another example, the learning data generating device 40 may be provided integrally with the teaching data generating device 10 or the robot controller 20. In this case, the control unit 41 of the learning data generating device 40 may be realized by the control unit 11 of the teaching data generating device 10 or the control unit 21 of the robot controller 20.

In the above example, the second user checks the robot operation of the first user through a VR experience or the like, but the present disclosure is not limited to this. For example, the second user may check the robot operation of the first user through a monitor or the like, or the second user may be present near the first user and check the operation of the first user directly. In this case, the reaction of the second user may be acquired by acquiring the line of sight or voice of the second user through a monitor or a camera or microphone installed around the first user or the second user, etc.

In the above example, the second user's reaction is acquired via a camera or microphone, but the second user's reaction may be acquired, for example, via a command or button to which the second user's reaction is assigned in advance. Specifically, if the first user's robot operation is good, a command indicating high evaluation may be input by clicking via an input device including a display unit.

All of the features described in this disclosure and/or all of the steps of all of the disclosed methods or processes may be combined in any combination except those combinations in which the features are mutually exclusive. Furthermore, each feature described in this disclosure may be replaced by an alternative feature serving the same, equivalent or similar purpose, unless expressly denied. Thus, unless expressly denied, each disclosed feature is merely one example of a generic series of the same or equivalent features.

Furthermore, the embodiments of the present disclosure are not limited to the specific configurations of any of the embodiments described above. The embodiments of the present disclosure may extend to any novel feature or combination of features described herein, or any novel method or process step or combination of features described herein.

The robot control system 1 and teaching data generation device 10 disclosed herein can be deployed in industrial product production sites, food processing sites that handle ingredients, or sites that manufacture cosmetics or pharmaceuticals.

In one embodiment, (1) a learning data generation device generates learning data for imitation learning of a robot. The learning data generation device includes an acquisition unit that acquires motion information when a first user operates the robot and a reaction of a second user when the second user recognizes the motion information, and a generation unit that generates the learning data based on the reaction of the second user and the motion information.

(2) In the learning data generation device described in (1) above, the acquisition unit may acquire at least one of environmental information of the robot or state information of the first user when the robot is operated by the first user. The generation unit may further generate the learning data based on the environmental information or the state information.

(3) The learning data generation device described in (1) or (2) above may further include an analysis unit that analyzes a reaction of the second user. The analysis unit may determine, based on the reaction of the second user, a portion of the motion information that can be used in the learning data and a portion that cannot be used in the learning data. The generation unit may generate the learning data by modifying, replacing, or deleting the portion that cannot be used in the learning data.

(4) In the learning data generation device described in (3) above, the generation unit may replace a portion of the learning data that cannot be used with reference data that serves as a model of the behavior of the robot 30 when the second user operates the robot.

(5) The learning data generation device described in any one of (1) to (4) above may further include an output unit that allows the second user to view the motion information in a VR environment.

(6) In the learning data generation device described in (5) above, the acquisition unit may acquire the second user's gaze as the second user's reaction.

(7) In the learning data generation device described in any one of (1) to (6) above, the acquisition unit may acquire an utterance of the second user as the reaction of the second user.

In one embodiment, (8) the learning data generation method includes generating learning data for imitation learning of a robot. The learning data generation method includes acquiring motion information when a first user operates the robot and a reaction of a second user when the second user recognizes the motion information, and generating the learning data based on the reaction of the second user and the motion information.

In one embodiment, (9) the program operates a learning data generation device that learns the robot's movements. The program causes the learning data generation device to acquire movement information when a first user operates the robot and a reaction of a second user when the second user recognizes the movement information, and generates the learning data based on the reaction of the second user and the movement information.

1 Robot control system 10 Teaching data generating device (11: control unit, 12: memory unit, 13: communication unit, 14: learning unit)
20 Robot controller (21: control unit, 22: memory unit, 23: input unit)
30 Robot 40 Learning data generation device (41: control unit, 42: storage unit, 43: acquisition unit, 44: output unit, 45: analysis unit, 46: generation unit)

Claims (9)

A learning data generation device for generating learning data for imitation learning of a robot, comprising:
an acquisition unit that acquires motion information when a first user operates the robot and a reaction of a second user when the second user recognizes the motion information;
A learning data generation device comprising: a generation unit that generates the learning data based on a reaction of the second user and the action information. The acquisition unit acquires at least one of environmental information of the robot and state information of the first user when the robot is operated by the first user,
The training data generating device according to claim 1 , wherein the generating unit generates the training data further based on the environmental information or the state information. An analysis unit that analyzes a reaction of the second user is further provided,
The analysis unit determines, based on a response of the second user, a portion of the motion information that can be adopted in the learning data and a portion that cannot be adopted in the learning data;
The training data generation device according to claim 1 , wherein the generation unit generates the training data by modifying, replacing, or deleting a portion that cannot be used in the training data. The learning data generating device according to claim 3, wherein the generating unit replaces the portion of the learning data that cannot be used with reference data that serves as a model of the behavior of the robot 30 when the second user operates the robot. The learning data generation device according to any one of claims 1 to 4, further comprising an output unit that allows the second user to view the motion information in a VR environment. The learning data generation device according to claim 5, wherein the acquisition unit acquires the second user's gaze as the second user's reaction. The learning data generation device according to claim 5, wherein the acquisition unit acquires the second user's utterance as the second user's reaction. A learning data generation method for generating learning data for imitation learning of a robot, comprising the steps of:
acquiring operation information when a first user operates the robot and a reaction of a second user when the second user recognizes the operation information;
A learning data generating method comprising: generating the learning data based on a reaction of the second user and the motion information. A program for operating a learning data generation device for learning a robot's movements, comprising:
causing the learning data generation device to acquire motion information when a first user operates the robot and a reaction of a second user when the second user recognizes the motion information;
A program that generates the learning data based on the reaction of the second user and the motion information.

Images