JP2024072129A - Computer system and robot control method - Google Patents

Abstract

To achieve automatic control of a robot by using an existing model without performing relearning.SOLUTION: A computer system generates a transformation formula for transforming measurement data acquired by a sensor such that a second dynamical system matches a first dynamical system on the basis of a difference between the first dynamical system, representing work in a learning environment, and the second dynamical system, representing work in a control environment, transforms the measurement data by using the transformation formula, inputs the transformed measurement data to a model, and controls a robot on the basis of an output from the model.SELECTED DRAWING: Figure 1

Description

The present invention relates to a robot control method and a robot control device.

There is a technology for automatically controlling a robot that performs tasks, such as assembly work, that involve gripping and moving a workpiece. In automatic control, a model is used that inputs measurement data obtained from a sensor attached to the robot. The model is generated by machine learning. For example, the technology described in Patent Document 1 is known.

JP 2022-61022 A JP 2013-106768 A

When utilizing an existing model, it is assumed that the actual work environment (control environment) is the same as the work environment (learning environment) in machine learning. However, if the control environment differs from the learning environment in terms of the sensor mounting position, the shape and orientation of the workpiece, and the position of the positioning pins, the magnitude and direction of the force applied to the workpiece by the robot will change, and the model will need to be re-learned.

In order to reuse an existing model, a method of converting the measurement data to be input to the model can be considered. The technology described in Patent Document 2 is known as a technology for converting measurement data.

Patent document 1 discloses a state detection device that "includes an acceleration information acquisition unit 110 that acquires detected acceleration from an acceleration sensor 200, a correction value acquisition unit 120 that acquires a first coordinate conversion correction value, a coordinate conversion unit 130 that performs coordinate conversion processing of the detected acceleration from the acceleration sensor coordinate system to a motion analysis coordinate system based on the first coordinate conversion correction value, and a motion analysis unit 140 that performs motion analysis processing based on the detected acceleration after the coordinate conversion processing."

By using the technology described in Patent Document 2, it is possible to eliminate the difference in the coordinate systems between environments. However, even if the technology described in Patent Document 2 is used, it is not possible to eliminate the difference in the dynamics systems that represent the work between environments.

The present invention provides a technology for automatically controlling tasks performed by a robot using an existing model by converting measurement data to eliminate differences in the dynamics of the tasks between environments.

To solve the above problem, the configuration described in the claims is adopted, for example.

A representative example of the invention disclosed in this application is as follows. That is, a computer system that uses a model generated by machine learning to control a robot that performs tasks including grasping and moving a workpiece, includes a calculation device, a storage device connected to the calculation device, and a computer having a network interface connected to the calculation device, and generates a first conversion formula for converting measurement data indicating the operating state of the robot based on a difference between a first dynamic system representing tasks in a learning environment and a second dynamic system representing tasks in a control environment so that the second dynamic system matches the first dynamic system, and when the measurement data is acquired, converts the measurement data using the first conversion formula, inputs the converted measurement data to the model, and controls the robot based on the output of the model.

According to the present invention, by converting measurement data so as to eliminate differences in the dynamics of the tasks between environments, tasks performed by a robot can be automatically controlled using an existing model. Problems, configurations, and effects other than those described above will become clear from the description of the embodiments below.

FIG. 1 illustrates an example of a system configuration according to a first embodiment. FIG. 2 is a diagram illustrating an example of a data structure of facility management information according to the first embodiment. FIG. 2 is a diagram illustrating an example of a data structure of facility management information according to the first embodiment. FIG. 2 is a diagram illustrating an example of a data structure of trajectory management information according to the first embodiment. FIG. 4 is a diagram illustrating an example of a data structure of model management information according to the first embodiment. 11 is a diagram illustrating an example of a data structure of conversion formula management information according to the first embodiment. FIG. 4 is a diagram illustrating an example of a data structure of work management information according to the first embodiment. 11 is a flowchart illustrating an example of a robot control process executed by a computer according to the first embodiment. FIG. 13 is a diagram illustrating an example of a screen presented by a computer according to the first embodiment. FIG. 13 is a diagram illustrating an example of a screen presented by a computer according to the first embodiment. 11 is a flowchart illustrating an example of a conversion formula (A) calculation process executed by a computer according to the first embodiment. FIG. 11 is a diagram showing a specific example of a conversion formula (A) calculation process executed by the computer of the first embodiment. FIG. 11 is a diagram showing a specific example of a conversion formula (A) calculation process executed by the computer of the first embodiment. FIG. 11 is a diagram showing a specific example of a conversion formula (A) calculation process executed by the computer of the first embodiment. 11 is a flowchart illustrating an example of a conversion formula (B) calculation process executed by the computer according to the first embodiment. FIG. 11 is a diagram showing a specific example of a conversion formula (B) calculation process executed by the computer of the first embodiment. FIG. 11 is a diagram showing a specific example of a conversion formula (B) calculation process executed by the computer of the first embodiment. FIG. 11 is a diagram showing a specific example of a conversion formula (B) calculation process executed by the computer of the first embodiment. FIG. 11 is a diagram showing a specific example of a conversion formula (B) calculation process executed by the computer of the first embodiment. FIG. 11 is a diagram showing a specific example of a conversion formula (B) calculation process executed by the computer of the first embodiment. FIG. 11 is a diagram showing a specific example of a conversion formula (B) calculation process executed by the computer of the first embodiment.

The following describes an embodiment of the present invention with reference to the drawings. However, the present invention should not be interpreted as being limited to the description of the embodiment shown below. It will be easily understood by those skilled in the art that the specific configuration can be changed without departing from the concept or spirit of the present invention.

In the configuration of the invention described below, the same or similar configurations or functions are given the same reference symbols, and duplicate explanations are omitted.

The terms "first," "second," "third," and the like used in this specification are used to identify components and do not necessarily limit the number or order.

The position, size, shape, range, etc. of each component shown in the drawings, etc. may not represent the actual position, size, shape, range, etc., in order to facilitate understanding of the invention. Therefore, the present invention is not limited to the position, size, shape, range, etc. disclosed in the drawings, etc.

Figure 1 is a diagram showing an example of the configuration of a system according to a first embodiment. The system is composed of a computer 100 and a robot 101. The computer 100 and the robot 101 are connected directly or via a network.

The computer 100 includes a calculation device 110, a storage device 111, a communication device 112, an input device 113, and an output device 114. Each hardware element is connected via an internal bus, for example, but is not limited to this method.

The input device 113 is a device for inputting data, commands, etc. to the computer 100. Examples of the input device 113 include a keyboard and a mouse.

The output device 114 is a device that outputs the results of control value changes, etc. Examples of the output device 114 include a display and a printer.

The storage device 111 is a device that stores the programs and information executed by the computing device 110. The storage device 111 is, for example, a hard disk drive (HDD) or a solid state drive (SSD). The storage device 111 also stores information input via the input device 113 and the results of program calculations. The storage device 111 is also used as a work area.

The storage device 111 stores equipment management information 130, trajectory management information 131, model management information 132, conversion formula management information 133, and work management information 134.

The equipment management information 130 stores equipment configuration information related to the configuration of the equipment on which the robot 101 performs work. The data structure of the equipment management information 130 will be described later with reference to Figures 2A and 2B.

The trajectory management information 131 stores trajectory information including control values for controlling the trajectory of the robot during work. The data structure of the trajectory management information 131 will be described later with reference to FIG. 3.

Model management information 132 stores models that output data values used to control the robot. The data structure of model management information 132 will be described later with reference to FIG. 4.

The model is, for example, a neural network, and is generated by reinforcement learning, deep learning, deep reinforcement learning, and the like. The model of Example 1 receives trajectory information, robot operating status information, and measurement data as input, and outputs a correction amount for the control value included in the trajectory information. Note that the present invention is not limited to the model structure, model output, and model learning method.

The conversion formula management information 133 stores the conversion formula for the measurement data. The data structure of the conversion formula management information 133 will be described later with reference to FIG. 5.

Work management information 134 stores work information related to work. The data structure of work management information 134 will be described later with reference to FIG. 5.

The arithmetic device 110 is a device that controls the computer 100, and is, for example, a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), and an FPGA (Field Programmable Gate Array). The arithmetic device 110 executes a program stored in the memory device 111. The arithmetic device 110 operates as a functional unit (module) that realizes a specific function by executing processing according to the program. In the following description, when the processing is described with the functional unit as the subject, it indicates that the arithmetic device 110 is executing a program that realizes the functional unit. The arithmetic device 110 of the first embodiment functions as an equipment position and orientation calculation unit 120, an equipment analysis unit 121, a conversion formula generation unit 122, a measurement data conversion unit 123, a correction amount determination unit 124, and a control value calculation unit 125.

The equipment position and orientation calculation unit 120 calculates the positions and orientations of the equipment and work that make up the equipment based on the equipment configuration information and trajectory information.

The equipment analysis unit 121 analyzes the differences between the environments based on the equipment configuration information of each of the learning environment and the control environment. For example, the equipment analysis unit 121 calculates the amount of difference in the position and orientation of the sensor attached to the tip of the robot 101 in each environment, and also determines whether or not there is a positioning pin for the workpiece in each environment.

The conversion formula generation unit 122 generates a conversion formula for converting the measurement data so that differences between environments are eliminated.

The measurement data conversion unit 123 uses the conversion equation generated by the conversion equation generation unit 122 to convert the measurement data acquired from the measurement device 140 described below.

The correction amount determination unit 124 calculates the correction amount by inputting the converted measurement data into the model.

The control value calculation unit 125 calculates the control value to be transmitted to the robot 101 based on the trajectory information and the correction amount calculated by the correction amount determination unit 124.

Regarding the functional units of the computer 100, multiple functional units may be combined into one functional unit, or one functional unit may be divided into multiple functional units for each function.

The communication device 112 is a device for communicating with external devices, such as a NIC (Network Interface Card).

The robot 101 performs tasks including grasping a workpiece and moving it from a starting point to an end point based on control information including control values calculated by the computer 100.

The robot 101 includes a measuring device 140, a group of working devices 141, and a controller 142.

The measuring device 140 measures values to grasp the state of the workpiece caused by the work of the robot 101, and generates measurement data. The measuring device 140 is, for example, a general sensing device such as an acceleration sensor, a force sensor, a camera, a contact sensor, and a current sensor.

The work device group 141 is a group of devices that grasp and move the workpiece, and is, for example, a group of general mechanical elements such as tools, links, and drive motors.

The controller 142 controls the group of working devices 141 based on the control values received from the computer 100. For example, the controller 142 moves the tool by controlling the drive motors that connect the links that function as joints in accordance with the control values. The controller 142 outputs operating status information to the computer 100, including the angles, angular velocities, and angular accelerations of the joints, as well as the torque and current values of the drive motors.

2A and 2B are diagrams showing an example of the data structure of facility management information 130 in Example 1.

The facility management information 130 stores entries including an ID 201, a facility name 202, a classification 203, an item 204, and details 205. One entry corresponds to one piece of facility configuration information. One entry includes rows for multiple elements that make up the facility.

ID 201 is a field that stores identification information for facility configuration information. Facility name 202 is a field that stores the name of the facility. Classification 203 is a field that stores the classification of the elements that make up the facility. Objects, robots, links, joints, etc. are stored in classification 203. Item 204 is a field that stores the management items of the elements. Content 205 is a field that stores the contents of the management items. Files, numbers, character strings, etc. are stored in content 205.

The "assembly cell stand" element manages the name, attachment target, relative position, relative orientation, and shape file information. The "force sensor" element manages the name, attachment target, relative position, relative orientation, and shape file information. The "hand" element manages the name, attachment target, relative position, relative orientation, and shape file information. The "work" element manages the name, work type, presence or absence of positioning pins, weight, attachment target, relative position, relative orientation, and shape file information. The "Robo" element manages the name, attachment target, relative position, and relative orientation. The "Link" element manages shape file information. The "Joint" element stores the parent link, child link, joint type, joint angle lower limit, joint angle upper limit, and joint angle speed upper limit.

The relative position and orientation of an element are the position and orientation of the element relative to the object to which it is attached. The assembly cell stand is the reference point (Root) of the equipment, and an absolute coordinate system is set with any point on the assembly cell stand as the origin.

Figure 3 is a diagram showing an example of the data structure of trajectory management information 131 in Example 1.

The trajectory management information 131 stores entries including an ID 301, a trajectory name 302, an attitude 303, and a time 304. One entry corresponds to one piece of trajectory information.

ID 301 is a field that stores identification information for trajectory information. Trajectory name 302 is a field that stores the name of the trajectory information. Posture 303 is a group of fields that stores the angles of the joint axes at the waypoints. Time 304 is a field that stores the time from when movement begins from the start waypoint to when the waypoint is reached. Time 304 for the start waypoint is "0.0".

Figure 4 is a diagram showing an example of the data structure of model management information 132 in Example 1.

The model management information 132 stores entries including an ID 401, a model 402, a facility name 403, and an orbit name 404. One entry corresponds to one model.

ID 401 is a field that stores identification information of the model. Model 402 is a field that stores the model. Equipment name 403 is a field that stores the name of the equipment at the time of learning the model. Equipment name 403 corresponds to equipment name 202. Trajectory name 404 is a field that stores the name of the trajectory at the time of learning the model. Trajectory name 404 corresponds to trajectory name 302.

Figure 5 is a diagram showing an example of the data structure of the conversion formula management information 133 in Example 1.

The conversion formula management information 133 stores entries including an ID 501, a model 502, a facility name 503, an orbit name 504, a conversion formula (A) 505, and a conversion formula (B) 506. One entry corresponds to one task.

ID 501 is a field that stores the identification information of the work. Model 502 is a field that stores the model to be applied to control the work. Equipment name 503 is a field that stores the name of the equipment in the work. Equipment name 503 corresponds to equipment name 202. Trajectory name 504 is a field that stores the name of the trajectory in the work. Trajectory name 504 corresponds to trajectory name 302. Conversion formula (A) 505 and conversion formula (B) 506 are fields that store conversion formulas for converting measurement data. The conversion formula stored in conversion formula (A) 505 is a formula for aligning the coordinate systems of the sensors in the two environments. The conversion formula stored in conversion formula (B) 506 is a formula for aligning the dynamic systems of the two environments. Here, the dynamic system of the environment means a dynamic system related to the gripping and movement of the work. The dynamic system of the environment is described using parameters (force parameters) related to the forces and moments acting on the work.

Figure 6 is a diagram showing an example of the data structure of work management information 134 in Example 1.

The work management information 134 stores entries including an ID 601, an equipment name 602, a work type 603, a moment of inertia parameter 604, and a positioning pin parameter 605. One entry corresponds to one piece of work information.

ID 601 is a field that stores identification information for the work information. Equipment name 602 is a field that stores the name of the equipment that handles the work. Equipment name 602 corresponds to equipment name 202. Work type 603 is a field that stores the type of work. Inertia moment parameters 604 are a group of fields that store parameters for calculating the moment of inertia of the work. Locating pin parameters 605 are a group of fields that store parameters related to the position of the locating pin of the work.

Parameters for calculating the force sense parameters are stored in the moment of inertia parameters 604 and the positioning pin parameters 605. Note that the moment of inertia parameters 604 and the positioning pin parameters 605 are merely examples and are not limiting. For example, fields for storing parameters related to material, rigidity, etc. may be included.

Figure 7 is a flowchart illustrating an example of a robot control process executed by the computer 100 of the first embodiment. Figures 8 and 9 are diagrams showing examples of screens presented by the computer 100 of the first embodiment.

The calculator 100 displays the screen 800 shown in FIG. 8 via the output device 114 and accepts user input regarding the model, equipment, trajectory, etc., of the work (step S101).

The screen 800 includes an input area 801, an environment display area 802, a conversion formula display area 803, and an operation button area 804.

The input area 801 is an area for inputting models, facilities, orbits, etc., and includes selection fields 811, 813, 815, load buttons 812, 814, 816, and a display field 817.

The selection field 811 is a field for selecting a model. When the load button 812 is operated, the model specified in the selection field 811 is read from the model management information 132. The selection field 813 is a field for selecting equipment. When the load button 814 is operated, the equipment configuration information of the equipment specified in the selection field 813 is read from the equipment management information 130. The selection field 815 is a field for selecting a trajectory. When the load button 816 is operated, the trajectory information of the trajectory specified in the selection field 815 is read from the trajectory management information 131. The display field 817 is a field for displaying the type of work used in the specified equipment.

In addition, model, facility configuration information, and orbit information may be accepted directly from the user.

The environment display area 802 is a field that displays the learning environment and the control environment. The environment display area 802 includes display fields 821 and 822.

The display field 821 is a field that displays information about the learning environment. The computer 100 refers to the model management information 132 and searches for an entry that corresponds to the model specified in the selection field 811. The computer 100 acquires facility configuration information and trajectory information from the facility management information 130 and trajectory management information 131 based on the facility name 403 and trajectory name 404 of the searched entry. The computer 100 generates an image representing the learning environment based on the acquired information and displays it in the display field 821.

The display field 822 is a field that displays information about the controlled environment. The computer 100 generates an image representing the actual work environment based on the information specified in the selection fields 813 and 815, and displays it in the display field 822.

In addition, display fields 821 and 822 may display the shape of the workpiece, parameters related to the moment of inertia, parameters related to the position of the positioning pin, etc.

The conversion formula display area 803 is a field that displays the conversion formula.

The operation button area 804 is a field that displays operation buttons for instructing the calculator 100 to execute a process. The operation button area 804 includes a conversion formula button 841 and a control button 842. The conversion formula button 841 is an operation button for instructing the execution of a conversion formula calculation process. The control button 842 is an operation button for instructing the execution of a control process.

After inputting the necessary information in the input area 801, the user operates the conversion formula button 841. When the calculator 100 receives this operation, it executes a conversion formula (A) calculation process and a conversion formula (B) calculation process (steps S102 and S103). The details of the conversion formula (A) calculation process and the conversion formula (B) calculation process will be described later.

The calculator 100 displays the processing results in the conversion formula display area 803. After checking the conversion formula display area 803, the user operates the control button 842. The calculator 100 starts the control process (step S104). The control process is repeatedly executed until the operation, including gripping and moving the workpiece, is completed.

The computer 100 acquires measurement data from the measuring device 140 via the communication device 112 and acquires operating status information from the controller 142 (step S105).

The calculator 100 converts the measurement data using the conversion formula (A) and the conversion formula (B) (step S106). Specifically, the measurement data converter 123 converts the measurement data into the first converted measurement data using the conversion formula (A), and converts the first converted measurement data into the second converted measurement data using the conversion formula (B).

The calculator 100 calculates the correction amount by inputting the trajectory information, the converted measurement data (second converted measurement data), and the operating state information into the model (step S107). The model outputs a correction amount that matches the operation cycle of the robot 101. For example, if the operation cycle of the robot 101 is Δt seconds, and T seconds have passed since the robot 101 started operating, the correction amount for each joint angle of the robot at a point at T+Δt is calculated.

The computer 100 corrects the control values included in the trajectory information based on the correction amount, and generates control information including the corrected control values (step S108). For example, if the operation period of the robot 101 is Δt seconds, and T seconds have passed since the robot 101 started operating, the correction is performed by adding the correction amount to the angle (control value) of each joint at T+Δt in the trajectory information.

The computer 100 transmits control information to the controller 142 via the communication device 112 (step S109).

If the task completion conditions are not met, the computer 100 returns to step S105 and executes the same process. If the task completion conditions are met, the computer 100 ends the control process.

The condition for completing the work is, for example, the elapse of Time 304 at the final waypoint of the trajectory from the start of control of the robot based on the trajectory information. The condition for completing the work may also be the calculation of the control value for the final waypoint of the trajectory.

The computer 100 presents the execution results via the output device 114 (step S110), and then terminates the robot control process.

For example, the computer 100 displays a screen 900. The screen 900 includes display fields 901 and 902. The display field 901 displays a model, equipment configuration information, trajectory information, work type, conversion formula, etc. The display field 902 displays, as execution results, a time series graph 921 of the measurement data before conversion, a time series graph 922 of the measurement data after conversion, and an image 923 showing the trajectory of the work.

The solid line in image 923 represents the trajectory of the robot 101 when controlled based on the trajectory information, and the dotted line represents the trajectory of the robot 101 when controlled based on the corrected control value.

In the robot control process of Example 1, an existing model can be used to control the work of the robot 101 in a controlled environment. For example, even if the workpiece is misaligned, the model can be used to correct the control value according to the amount of misalignment.

Figure 10 is a flowchart illustrating an example of the conversion formula (A) calculation process executed by the computer 100 of the first embodiment. Figures 11A, 11B, and 11C are diagrams illustrating a specific example of the conversion formula (A) calculation process executed by the computer 100 of the first embodiment.

The equipment position and orientation calculation unit 120 acquires equipment configuration information and trajectory information of the learning environment (step S201). Specifically, the following process is executed.

(S201-1) The equipment position and orientation calculation unit 120 refers to the model management information 132 and searches for an entry in which the model specified in the model 402 is stored.

(S202-2) The equipment position and orientation calculation unit 120 refers to the equipment management information 130 and searches for an entry whose equipment name 202 value matches the equipment name 403 value of the entry searched for in the model management information 132. The equipment position and orientation calculation unit 120 obtains the equipment configuration information corresponding to the searched entry.

(S202-3) The equipment position and orientation calculation unit 120 refers to the trajectory management information 131 and searches for an entry whose trajectory name 302 value matches the trajectory name 404 value of the entry searched for in the model management information 132. The equipment position and orientation calculation unit 120 obtains the trajectory information corresponding to the searched entry.

The equipment position and orientation calculation unit 120 calculates the position and orientation of the sensor in the learning environment based on the equipment configuration information and trajectory information of the learning environment (step S202). Here, the position and orientation of the sensor in the learning environment refers to the position and orientation of the sensor at the start of the work.

Figure 11A shows an example of the position and orientation of a sensor in a learning environment. Figure 11A shows the position and orientation of the sensor relative to an equipment coordinate system 1100. The equipment coordinate system 1100 is an absolute coordinate system of the learning environment with an arbitrary point on the equipment as its origin. In Figure 11A, the equipment coordinate system 1100 is an absolute coordinate system with the installation position of assembly cell stand A, whose mounting target is the Root, as its origin. Coordinate system 1101 is a relative coordinate system with the sensor as its origin.

The equipment position and orientation calculation unit 120 calculates the position and orientation of the sensor based on the equipment coordinate system 1100 and the equipment configuration information. First, RoboB is attached to the assembly cell stand A. The equipment position and orientation calculation unit 120 calculates the attachment position of RoboB based on the equipment coordinate system 1100 and the relative position and relative orientation of RoboB with respect to the assembly cell stand A. Link_ee is attached to RoboB. The equipment position and orientation calculation unit 120 calculates the position and orientation of Link_ee based on the attachment position of RoboB and the angles of each joint of the start via point of the trajectory information of RoboB. A force sensor is installed on Link_ee. The equipment position and orientation calculation unit 120 calculates the position and orientation of the force sensor based on the position and orientation of Link_ee and the relative position and relative orientation of the force sensor with respect to Link_ee.

The equipment position and orientation calculation unit 120 calculates the position and orientation of the sensor in the controlled environment based on the equipment configuration information and trajectory information acquired in step S101 (step S203). Here, the position and orientation of the sensor in the controlled environment means the position and orientation of the sensor at the start of the work. The calculation method is the same as in step S202.

Figure 11B shows an example of the position and orientation of a sensor in a controlled environment. Figure 11B shows the position and orientation of the sensor relative to an equipment coordinate system 1110. The equipment coordinate system 1110 is an absolute coordinate system of the controlled environment with an arbitrary point on the equipment as its origin. In Figure 11B, the equipment coordinate system 1110 is an absolute coordinate system with the installation position of the assembly cell stand, whose mounting target is the Root, as its origin. Coordinate system 1111 is a relative coordinate system with the sensor as its origin.

The equipment analysis unit 121 calculates the position deviation and attitude deviation of the sensor based on the position and attitude of the sensor in the learning environment and the position and attitude of the sensor in the control environment (step S204).

The positional deviation of the sensor is calculated as the deviation of the position of the origin of the coordinate systems 1101 and 1111. The attitude deviation of the sensor is calculated as the rotation angle θ of the coordinate systems 1101 and 1111 as shown in FIG. 11C.

The conversion equation generation unit 122 generates a conversion equation (A) based on the position shift and orientation shift of the sensor between the two environments (step S205). The conversion equation (A) is expressed as a translation in three-dimensional space to eliminate the position shift of the sensor, and a rotation in three-dimensional space to eliminate the orientation shift. For example, the rotation with respect to the Z axis is given by equation (1). The rotation angle θ is an angle that represents the orientation shift of the sensor.

The conversion formula generation unit 122 registers the conversion formula (A) in the conversion formula management information 133 (step S206) and terminates the conversion formula (A) calculation process.

Figure 12 is a flowchart illustrating an example of the conversion formula (B) calculation process executed by the computer 100 of the first embodiment. Figures 13A, 13B, 13C, 13D, 13E, and 13F are diagrams illustrating specific examples of the conversion formula (B) calculation process executed by the computer 100 of the first embodiment.

The equipment position and orientation calculation unit 120 calculates the position and orientation of the work in the learning environment based on the equipment configuration information and trajectory information of the learning environment (step S301). Here, the position and orientation of the work in the learning environment refers to the position and orientation of the work at the start of work. Note that the equipment configuration information and trajectory information at the time of model learning are assumed to have been acquired in the conversion formula (A) calculation process.

Figure 13A shows an example of the position and orientation of the workpiece in the learning environment. Figure 13A shows the position and orientation of the workpiece relative to the sensor.

The equipment position and orientation calculation unit 120 calculates the position and orientation of the sensor by processing similar to that of step S202. Hand B is attached to the force sensor. The equipment position and orientation calculation unit 120 calculates the position and orientation of hand B based on the position and orientation of the force sensor and the relative position and relative orientation of hand B with respect to the force sensor. Work B is grasped and moved by hand B. The equipment position and orientation calculation unit 120 calculates the position and orientation of work B based on the position and orientation of hand B and the relative position and relative orientation of work B with respect to hand B.

The equipment position and orientation calculation unit 120 calculates the position and orientation of the workpiece in the controlled environment based on the equipment configuration information and trajectory information acquired in step S101 (step S302). Here, the position and orientation of the workpiece in the controlled environment refers to the position and orientation of the workpiece at the start of the work. The calculation method is the same as in step S301.

Figure 13B shows an example of the position and orientation of the workpiece in a controlled environment. Figure 13B shows the position and orientation of the workpiece relative to the sensor.

The equipment analysis unit 121 obtains the parameters of each work in the learning environment and the control environment from the work management information 134 (step S303).

The conversion formula generation unit 122 calculates the difference in the force parameters of the dynamics of each environment based on the position and orientation of the sensor in each environment, the position and orientation of the workpiece, and the parameters of the workpiece (step S304). Here, the method for calculating the difference in the force parameters of the dynamics of each environment will be described.

(Example 1) As shown in Figures 13C and 13D, a method for generating a conversion formula (B) when parameters a, b, c, d, e, f, and g related to the positioning pin are acquired will be described. Parameters a, b, c, and d represent the distance from the edge of the workpiece in the learning environment to the positioning pin, and parameters e, f, and g represent the distance from the edge of the workpiece in the control environment to the positioning pin. The direction of rotation with respect to the central axis 1300 (1301) of the workpiece indicates the direction of the moment on the x-axis (x'-axis) of the sensor coordinate system 1101 (1111). When the hand comes into contact with the workpiece at the positioning pin, the direction and magnitude of the moment become the force sense parameters. The ratio of the distance from the edge of the workpiece to the positioning pin, which is the fulcrum of the lever principle, becomes the magnitude of the moment of the workpiece. Therefore, in FIG. 13C, the magnitude of the moment on the x-axis of coordinate system 1101 is b/a, and in FIG. 13D, the magnitude of the moment on the x'-axis of coordinate system 1111 is g/h.

The conversion formula generation unit 122 calculates the magnitude and direction of the moment on the x-axis (x'-axis) of each environment, and calculates the difference between the magnitude and direction of the moment on the x-axis (x'-axis) of each environment as the amount of change in the force sense parameters of the dynamics of each environment. A similar calculation is performed for the y-axis (y'-axis).

(Example 2) A method of generating a conversion formula (B) when parameters a, b, c, and d related to the moment of inertia are acquired as shown in FIG. 13E and FIG. 13F will be described. Parameters a and b represent the length and width of the workpiece in the learning environment, and parameters c and d represent the length and width of the workpiece in the control environment. The direction of rotation with respect to the central axis 1300 (1301) of the workpiece indicates the direction of the moment on the x-axis (x'-axis) of the sensor coordinate system 1101 (1111). In a workpiece without a positioning pin, the reciprocal of the moment of inertia represents the magnitude of the moment of the workpiece. In the case of a thin plate part as shown in FIG. 13E, the moment of inertia on the x-axis is given by formula (2). m _e is the weight of the workpiece. In the case of FIG. 13F, the moment of inertia on the x'-axis is given by formula (3). m _f is the weight of the workpiece.

The conversion formula generation unit 122 calculates the moment of inertia on the x-axis (x'-axis) of each environment, and calculates the ratio of the reciprocal of the moment of inertia on the x-axis (x'-axis) of each environment as the amount of change in the force sense parameter of the dynamics system of each environment. A similar calculation is performed for the y-axis (y'-axis).

The above is an explanation of the method for calculating the difference in the force parameters of the dynamics of each environment. Note that the above method is an example and is not limiting. The direction and magnitude of twisting may also be treated as force parameters.

The conversion formula generation unit 122 generates a conversion formula (B) based on the difference between the force sense parameters of the dynamics of each environment (step S305). When the parameters of the positioning pin are used, a conversion formula (B) such as that shown in formula (4) is generated. When the parameters of the moment of inertia are used, a conversion formula (B) such as that shown in formula (5) is generated.

The conversion formula generation unit 122 registers the conversion formula (B) in the conversion formula management information 133 (step S306) and terminates the conversion formula (B) calculation process.

By using the transformation formula (B), the dynamical systems of the two environments can be aligned. This allows the model to be used as is.

According to the present invention, it is possible to convert measurement data so as to eliminate the difference in the dynamics of the work between environments. This makes it possible to realize automatic control of robot work using existing models.

Note that conversion formula (A) is not a required configuration. In this case, only conversion formula (B) is used.

The present invention is not limited to the above-described embodiments, but includes various modified examples. For example, the above-described embodiments are provided to explain the present invention in detail, and are not necessarily limited to those including all of the described configurations. In addition, it is possible to add, delete, or replace part of the configuration of each embodiment with another configuration.

The above-mentioned configurations, functions, processing units, processing means, etc. may be realized in part or in whole by hardware, for example by designing them as integrated circuits. The present invention can also be realized by software program code that realizes the functions of the embodiments. In this case, a storage medium on which the program code is recorded is provided to a computer, and a processor included in the computer reads the program code stored in the storage medium. In this case, the program code itself read from the storage medium realizes the functions of the above-mentioned embodiments, and the program code itself and the storage medium on which it is stored constitute the present invention. Examples of storage media for supplying such program code include flexible disks, CD-ROMs, DVD-ROMs, hard disks, SSDs (Solid State Drives), optical disks, magneto-optical disks, CD-Rs, magnetic tapes, non-volatile memory cards, ROMs, etc.

In addition, the program code that realizes the functions described in this embodiment can be implemented in a wide range of program or script languages, such as assembler, C/C++, perl, Shell, PHP, Python, Java (registered trademark), etc.

Furthermore, the program code of the software that realizes the functions of the embodiment may be distributed over a network and stored in a storage means such as a computer's hard disk or memory, or in a storage medium such as a CD-RW or CD-R, and the processor of the computer may read and execute the program code stored in the storage means or storage medium.

In the above examples, the control lines and information lines are those that are considered necessary for the explanation, and not all control lines and information lines in the product are necessarily shown. All components may be interconnected.

100 Computer 101 Robot 110 Arithmetic device 111 Storage device 112 Communication device 113 Input device 114 Output device 120 Equipment position and orientation calculation unit 121 Equipment analysis unit 122 Conversion formula generation unit 123 Measurement data conversion unit 124 Correction amount determination unit 125 Control value calculation unit 130 Equipment management information 131 Trajectory management information 132 Model management information 133 Conversion formula management information 134 Work management information 140 Measurement device 141 Working device group 142 Controller 800, 900 Screen

Claims (8)

A computer system that controls a robot that performs a task including gripping and moving a workpiece using a model generated by machine learning,
A computer including an arithmetic unit, a storage device connected to the arithmetic unit, and a network interface connected to the arithmetic unit,
managing trajectory information including control values for controlling a trajectory of the robot in a task for each of a learning environment and a controlled environment in which an actual task is performed;
generating a first transformation formula for transforming measurement data acquired by a sensor installed in the robot based on a difference between a first dynamical system representing an operation in the learning environment and a second dynamical system representing an operation in the controlled environment so that the second dynamical system coincides with the first dynamical system;
a computer system characterized in that, when the measurement data and operating status information indicating the operating status of the robot are acquired, the measurement data is converted using the first conversion formula, the trajectory information, the operating status information, and the converted measurement data are input to the model, and the robot is controlled based on the output of the model. 2. The computer system of claim 1,
Manage equipment configuration information relating to equipment and work information relating to work for each of the learning environment and the controlled environment;
calculating a first force parameter of the first dynamic system and a second force parameter of the second dynamic system using the trajectory information, the facility configuration information, and the work information;
A computer system comprising: a computer that generates the first transformation formula based on a difference between the first haptic parameter and the second haptic parameter. 3. The computer system of claim 2,
A computer system, characterized in that the first force sense parameter and the second force sense parameter are parameters related to forces and moments acting on the workpiece. 2. The computer system of claim 1,
generating a second transformation equation that matches a coordinate system of the sensor in the learning environment with a coordinate system of the sensor in the controlled environment based on a position and an orientation of the sensor in the learning environment and a position and orientation of the sensor in the controlled environment;
A computer system which converts the measurement data using the first conversion formula and the second conversion formula. A method for controlling a robot that performs a task including gripping and moving a workpiece by using a model generated by machine learning, the method comprising the steps of:
The computer system includes:
A computer including an arithmetic unit, a storage device connected to the arithmetic unit, and a network interface connected to the arithmetic unit,
managing trajectory information including control values for controlling a trajectory of the robot in a task for each of a learning environment and a controlled environment in which an actual task is performed;
The method for controlling a robot includes the steps of:
a first step in which the computer generates a first conversion formula for converting measurement data acquired by a sensor installed in the robot based on a difference between a first dynamical system representing an operation in the learning environment and a second dynamical system representing an operation in the controlled environment so that the second dynamical system coincides with the first dynamical system;
a second step of, when the computer acquires the measurement data and operating status information indicating an operating status of the robot, converting the measurement data using the first conversion formula, inputting the trajectory information, the operating status information, and the converted measurement data into the model, and controlling the robot based on an output of the model. A method for controlling a robot according to claim 5, comprising the steps of:
the computer system manages equipment configuration information relating to equipment and work information relating to work for each of the learning environment and the controlled environment;
The first step includes:
a step of the computer calculating a first force parameter of the first dynamic system and a second force parameter of the second dynamic system by using the trajectory information, the facility configuration information, and the work information;
and generating the first conversion equation based on a difference between the first haptic parameter and the second haptic parameter by the computer. A method for controlling a robot according to claim 6, comprising the steps of:
A method for controlling a robot, wherein the first force sense parameter and the second force sense parameter are parameters related to forces and moments acting on the workpiece. A method for controlling a robot according to claim 5, comprising the steps of:
a step of generating a second transformation equation for matching a coordinate system of the sensor in the learning environment with a coordinate system of the sensor in the controlled environment, based on a position and an orientation of the sensor in the learning environment and a position and an orientation of the sensor in the controlled environment,
The robot control method according to the present invention, wherein the second step includes a step of converting the measurement data by the computer using the first conversion formula and the second conversion formula.

Images