JP2024098188A - Teaching data storage device - Google Patents

Abstract

To solve such a problem that if simulation of a linear body attached to a robot is performed after completion of a series of teachings, a teaching result could be wasted.SOLUTION: A teaching data accumulation apparatus 3 includes: a positional relation acquisition unit 33 which acquires a relative positional relation between a real environment and a display device 4; a reception unit 34 which receives operation and teaching instructions of a three-dimensional model 10 of a virtual robot; an image generation unit 35 which generates a display image that displays the operated three-dimensional model 10 on the basis of the three-dimensional model and the relative positional relation; an output unit 36 which outputs the display image to the display device 4; an accumulation unit 37 which accumulates teaching data at the time of reception of the teaching instructions; and a torsional amount acquisition unit 38 which acquires a torsional amount of a linear body attached to the three-dimensional model of the robot body of the virtual robot. The output unit 36 performs an output on the acquired torsional amount. In this way, it is possible to perform teaching while confirming the torsional amount of the linear body.SELECTED DRAWING: Figure 1

Description

The present invention relates to a teaching data storage device that stores teaching data.

Conventionally, systems are known that perform simulations of the posture of linear objects such as cables and hoses. Such simulations are also sometimes performed on linear objects attached to robots (see, for example, Patent Document 1).

JP 2016-087750 A

However, simulations of linear bodies attached to robots are usually performed after the robot's movements have been determined, i.e., after teaching has been completed. Therefore, if the simulation results show that the linear body has been twisted beyond the allowable range, new teaching must be performed again, which creates the problem of the possibility that the previous teaching may be wasted.

The present invention has been made to solve the above problems, and aims to provide a teaching data storage device that can prevent twisting beyond an acceptable range from occurring in a linear body attached to the robot body when the robot operates according to teaching data.

In order to achieve the above object, a teaching data storage device according to one aspect of the present invention includes a model storage unit in which a three-dimensional model of a virtual robot corresponding to a real robot is stored, a teaching data storage unit in which teaching data of the real robot is stored, a positional relationship acquisition unit that acquires a relative positional relationship between the real environment and a display device that displays an image superimposed on an image of the real environment or on the real environment itself, a reception unit that accepts operation and teaching instructions for the three-dimensional model of the virtual robot, an image generation unit that generates a display image for displaying the three-dimensional model of the operated virtual robot at a predetermined position in the real environment based on the relative positional relationship with the three-dimensional model of the virtual robot, an output unit that outputs the display image generated by the image generation unit to the display device, a storage unit that accumulates teaching data in the teaching data storage unit according to the three-dimensional model of the virtual robot when a teaching instruction for the three-dimensional model of the virtual robot is accepted by the reception unit, and a twist amount acquisition unit that acquires the twist amount of a linear body attached to the three-dimensional model of the robot body of the virtual robot, and the output unit also outputs the twist amount acquired by the twist amount acquisition unit.

According to a teaching data storage device according to one aspect of the present invention, the amount of twist in the linear body attached to the robot body can be known during teaching. Therefore, for example, if the amount of twist in the linear body is large, the amount of twist in the linear body can be reduced by changing the way the linear body is handled or by changing the teaching. As a result, when the robot operates according to the stored teaching data, it is possible to prevent the linear body attached to the robot body from twisting beyond the allowable range.

FIG. 1 is a block diagram showing a configuration of a teaching data storage device according to an embodiment of the present invention; A flowchart showing the operation of the teaching data storage device according to the embodiment. FIG. 13 is a diagram showing an example of a display of a three-dimensional model of a virtual robot and a twist amount in the embodiment. FIG. 13 is a diagram showing an example of an acquired torsion amount in the embodiment; FIG. 2 is a block diagram showing another example of the configuration of the teaching data storage device according to the embodiment;

The teaching data storage device according to the present invention will be described below using an embodiment. Note that in the following embodiments, components and steps with the same reference numerals are the same or equivalent, and repeated explanations may be omitted. The teaching data storage device according to this embodiment acquires the amount of twist of a linear body attached to the robot body during teaching, and outputs information related to the amount of twist.

FIG. 1 is a schematic diagram showing the configuration of an information processing system 100 including a teaching data storage device 3 according to this embodiment. As shown in FIG. 1, the information processing system 100 includes a robot 1, a robot control device 2, a teaching data storage device 3, and a display device 4. Note that the devices may be connected to each other, for example, by wire or wirelessly. Also, this embodiment will mainly describe a case where the robot 1, which is an actual robot, exists in a real environment as shown in FIG. 1, but this is not necessarily the case. For example, the robot 1 may not exist in the real environment. In this case, the information processing system 100 may not include, for example, the robot 1 or the robot control device 2.

The robot 1 is usually an industrial robot, and may be a manipulator having a plurality of arms (i.e., links) connected by joints driven by a motor. The robot 1 may be, for example, a vertical articulated robot or a horizontal articulated robot. The robot 1 may also be, for example, a transport robot, a welding robot, an assembly robot, a painting robot, or a robot for other purposes. In this embodiment, the case where the robot 1 is a welding robot will be mainly described. The robot 1 includes a robot main body and a linear body 1a attached to the robot main body. Note that the part of the robot 1 other than the linear body 1a may be considered to be the robot main body. The linear body 1a may be, for example, a wiring cable for supplying power, a wiring cable for transmitting control signals, etc., a conduit cable through which a welding wire passes, a pipe for supplying liquid or gas, or another linear configuration attached to the robot main body. It is usually preferable that the linear body 1a is one whose shape changes depending on the posture of the robot 1, that is, one that has flexibility. FIG. 1 shows a case where the linear body 1a is fixed to the housing of the robot 1 at both ends 1b and 1c, but the linear body 1a may also be fixed to the housing of the robot 1 at other places than both ends 1b and 1c. The position where the linear body 1a is fixed to the housing of the robot 1 may be called a fixed position. FIG. 1 also shows a case where a part of the linear body 1a passes through the inside of the housing of the robot 1 as an example, but this is not necessary. The linear body 1a may be, for example, entirely outside the housing, or entirely inside the housing. Note that the robot 1 is a robot that exists in a real environment, and is therefore sometimes called a real robot to distinguish it from a three-dimensional model virtual robot that exists in a virtual environment. The real environment refers to the environment in real space.

The robot control device 2 controls the operation of the robot 1 using a teaching playback method. The robot control device 2 that operates the robot 1 based on teaching data is already publicly known, and a detailed description thereof will be omitted. Note that the information processing system 100 may further include, for example, a teaching pendant for teaching the robot 1, and if the robot 1 is a welding robot, may further include a welding power source, a wire feeder, etc., as necessary.

The teaching data storage device 3 stores teaching data for the robot 1 using a three-dimensional model 10 of a virtual robot corresponding to the robot 1. Details of the teaching data storage device 3 will be described later.

The display device 4 displays an image superimposed on an image of the real environment or the real environment itself. That is, the operator who gives the instruction can see both the real environment and the virtual environment images by using the display device 4. The display device 4 may be a wearable display device that the operator who gives the instruction wears on his/her head, or may be a display device that is a portable information processing terminal such as a tablet terminal. The wearable display device may be, for example, a head-mounted display. The display device 4 may also have, for example, a transmissive display. In this case, the display device 4 displays an image superimposed on the real environment itself. As a wearable display device 4 having a transmissive display, for example, HoloLens (registered trademark) is known. Such a display device 4 having a transmissive display can also be considered as a display device for realizing mixed reality (MR: Mixed Reality). The display device 4 may also have, for example, a non-transmissive display. In this case, the display device 4 displays an image superimposed on an image of the real environment. Therefore, it is preferable that the display device 4 having a non-transmissive display has a camera for photographing the real environment or is connected to a camera for photographing the real environment. The image of the real environment captured by the camera is displayed on the non-transmissive display in real time. For example, Oculus Quest is known as a wearable display device 4 having a non-transmissive display. Such a display device 4 having a non-transmissive display can also be considered as a display device for realizing augmented reality (AR). The display device 4, which is a portable information processing terminal such as a tablet terminal, has, for example, a camera and a display, and may display an image of the real environment captured by the camera on the display in real time. In this embodiment, the case where the display device 4 is a head-mounted display having a transmissive display will be mainly described.

As shown in FIG. 1, the teaching data storage device 3 is connected to the robot control device 2 and the display device 4, and includes a model memory unit 31, a teaching data storage unit 32, a positional relationship acquisition unit 33, a reception unit 34, an image generation unit 35, an output unit 36, a storage unit 37, and a twist amount acquisition unit 38.

The model storage unit 31 stores a three-dimensional model 10 of a virtual robot that corresponds to the robot 1, which is an actual robot. The three-dimensional model 10 of the virtual robot is assumed to have the same size and configuration as the robot 1, which is an actual robot, except that it is configured as a three-dimensional model. Furthermore, like the robot 1, this three-dimensional model 10 is assumed to be able to change the angles of the joints of multiple arms. Furthermore, like the robot 1, the three-dimensional model 10 of the virtual robot also has a linear body 10a fixed to the housing at both ends 10b, 10c.

The process by which the information on the three-dimensional model of the virtual robot is stored in the model storage unit 31 is not important. For example, the information may be stored in the model storage unit 31 via a recording medium, or information transmitted via a communication line or the like may be stored in the model storage unit 31. The model storage unit 31 is preferably realized by a non-volatile recording medium, but may also be realized by a volatile recording medium. The recording medium may be, for example, a semiconductor memory or a magnetic disk. In addition, the shape of the three-dimensional model 10 and the angles of each joint stored in the model storage unit 31 may be changed, for example, in response to a received operation.

The teaching data storage unit 32 stores the teaching data of the robot 1, which is an actual robot. The teaching data of the robot 1 is stored in the teaching data storage unit 32 by the storage unit 37. The teaching data stored in the teaching data storage unit 32 is teaching data used to control the robot 1, which is an actual robot. Therefore, for example, the teaching data stored in the teaching data storage unit 32 may be output to the robot control device 2. In this case, the teaching data storage device 3 may have, for example, a teaching data output unit (not shown) for outputting the teaching data stored in the teaching data storage unit 32 to the robot control device 2, other devices, removable recording media, etc. Also, for example, when the robot control device 2 controls the robot 1, it may access the teaching data stored in the teaching data storage unit 32. The teaching data may, for example, indicate the position and posture of the robot 1 for each teaching point. The information indicating the position and posture of the robot 1 may, for example, be information indicating the position and posture of the hand of the robot 1, or information indicating the angle in each axis of the robot 1. The teaching data may also include information indicating an interpolation method between teaching points, such as linear interpolation, curved interpolation, or circular interpolation. The teaching data may also include, for example, a teaching line in which multiple teaching points are interpolated. When the teaching data includes information on the positions of teaching points, the position information may be, for example, information in the local coordinate system of the robot, or information in the world coordinate system. The former is preferable when the three-dimensional model 10 of the virtual robot is displayed at a position different from the robot 1, which is the actual robot. In the latter case, the teaching data is usually teaching data used for the robot 1 placed at a predetermined position in the world coordinate system. The teaching data storage unit 32 is preferably realized by a non-volatile recording medium, but may be realized by a volatile recording medium. The recording medium may be, for example, a semiconductor memory or a magnetic disk.

The model storage unit 31 and the teaching data storage unit 32 may be realized by the same recording medium, or may be realized by separate recording media. In the former case, for example, the area storing the 3D model 10 of the virtual robot becomes the model storage unit 31, and the area storing the teaching data becomes the teaching data storage unit 32.

The positional relationship acquisition unit 33 acquires the relative positional relationship between the real environment and the display device 4. The real environment may be, for example, the real environment in which the robot 1 exists. In this embodiment, this case will be mainly described. Acquiring the relative positional relationship between the real environment and the display device 4 may be, for example, acquiring the relative positional relationship between the world coordinate system, which is the coordinate system of the real environment, and the display coordinate system, which is the local coordinate system of the display device 4. The relative positional relationship may be indicated, for example, by a homogeneous transformation matrix indicating the transformation between the two coordinate systems. The relative positional relationship between the real environment and the display device 4 may be, for example, the relative positional relationship between the robot 1 or other objects existing in the real environment, or a reference marker, which will be described later, and the display device 4, or may be information indicating the position and attitude of the display device 4 in the real environment.

The method by which the positional relationship acquisition unit 33 acquires this relative positional relationship is not important. For example, when an environmental map of the real environment is prepared, the positional relationship acquisition unit 33 may acquire the relative positional relationship, which is information indicating the position and attitude of the display device 4 in the three-dimensional real environment, using the environmental map of the real environment and the distance to the surrounding objects and the surrounding image acquired by the display device 4, using a method such as SLAM (Simultaneous Localization and Mapping) or Visual-SLAM. In this case, for example, the environmental map is stored in the teaching data storage device 3, and the positional relationship acquisition unit 33 may acquire the relative positional relationship using the distance to the surrounding objects and the surrounding image received from the display device 4 and the environmental map. Also, for example, the acquisition of the relative positional relationship using the SLAM or Visual-SLAM method may be performed in the display device 4, and the relative positional relationship may be passed to the positional relationship acquisition unit 33. In this case, the acquisition of the relative positional relationship by the positional relationship acquisition unit 33 may be the acceptance of the relative positional relationship. In this case, the display device 4 may have a depth sensor capable of measuring the distance to surrounding objects and a camera capable of acquiring images of the surroundings. The positional relationship acquisition unit 33 may receive an image captured by the camera of the display device 4, and acquire a homogeneous transformation matrix indicating the transformation between the world coordinate system and the display coordinate system using three or more feature points of the robot 1 or other objects, or a reference marker (described later), contained in the image.

The positional relationship acquisition unit 33 may acquire the relative positional relationship between the real environment and the display device 4 by a method other than the above. For an example of the method, see, for example, the following documents 1 to 3. The relative positional relationship may be acquired by the positional relationship acquisition unit 33 as described above for each generation of a display image described later, or may not be acquired. In the latter case, for example, the relative positional relationship between the real environment and the display device 4 may be updated using a change in the position or attitude of the display device 4 after the relative positional relationship is acquired. This update may be performed by, for example, the positional relationship acquisition unit 33. When this update is performed, the display device 4 may include, for example, an acceleration sensor or a sensor for acquiring the orientation of the display device 4, and the change in position or attitude may be acquired using these sensors. Note that the values acquired by these sensors may be passed to a component that updates the relative positional relationship, such as the positional relationship acquisition unit 33. The sensor for acquiring the orientation of the display device 4 may be, for example, a gyro sensor or an orientation sensor.
Document 1: JP 2017-100234 A Document 2: JP 2020-055075 A Document 3: JP 2020-069538 A

The reception unit 34 receives an operation for the three-dimensional model 10 of the virtual robot. The reception unit 34 also receives a teaching instruction for the three-dimensional model of the virtual robot. For example, an operation or teaching instruction for the three-dimensional model 10 may be received from a worker who is viewing the display image of the display device 4. The operation for the three-dimensional model 10 may be, for example, an operation for changing the position or posture of at least a part of the three-dimensional model 10. The reception of the operation or teaching instruction may be performed, for example, via an input device such as a teaching pendant present in the real environment, or may be performed via a virtual input interface such as a virtual button or a virtual teaching pendant displayed on the display of the display device 4. In the latter case, for example, the reception of the operation or teaching instruction for the three-dimensional model 10 of the virtual robot may be performed, for example, via the display device 4. The virtual input interface may be displayed on the display of the display device 4 in response to an operation such as an air tap, and when a button or the like is selected by the worker's finger or pointing device, an input corresponding to the operation of the button or the like may be passed from the display device 4 to the reception unit 34. In addition, the position and posture of at least a part (e.g., a hand tip of a tool, etc.) of the three-dimensional model 10 of the virtual robot may be changed by a gesture of the worker's hand or the like. In this case, for example, when the worker performs an action of pinching the three-dimensional model 10 displayed on the display with his/her hand (holding action), the part pinched with the hand is specified as the target of operation, and an operation of changing the position and posture of the specified target is performed by changing the position and posture of the hand, and the operation on the specified target may be ended by ending the pinching action. In this case, for example, information indicating the target of operation (e.g., information indicating the position or part in the three-dimensional model 10) and information indicating the content of the operation (e.g., information indicating a change in position or a change in posture) may be passed from the display device 4 to the reception unit 34, or the result of hand tracking acquired by the display device 4 may be passed to the reception unit 34, and the target of operation and the content of the operation may be specified in the teaching data storage device 3. When the display device 4 acquires information indicating the target of the operation and information indicating the content of the operation, the display device 4 may be able to access information on the current 3D model 10 of the virtual robot in the virtual space, which is stored in the teaching data storage device 3. When the operation is performed using the worker's hand, the display device 4 may have a camera, and the position of the worker's hand on the display may be identified by performing hand tracking of the worker's hand photographed by the camera. In addition, a method of converting an operation on a display image of a 3D model according to a gesture of the worker's hand or the like into a change in the position or posture of the 3D model in a 3D virtual space is already known, and a detailed description thereof will be omitted.

The reception unit 34 may also receive a change in the way the linear body 10a is attached to the three-dimensional model 10 of the robot body of the virtual robot. The reception unit 34 may receive a change in the way the linear body 10a is handled, for example, input via an input device in the real environment or a virtual input interface. The change in the way the linear body 10a is handled may be, for example, at least one of a change in the initial twist amount of the linear body 10a, a change in the length of the linear body 10a, and a change in the fixed position of the linear body 10a. The initial twist amount of the linear body 10a may be an angle at which the linear body 10a is rotated from an initial angle at at least one of the fixed positions of the ends 10b and 10c of the linear body 10a. When the change in the way the linear body 10a is handled is a change in the initial twist amount, the reception unit 34 may receive the changed initial twist amount. The length of the linear body 10a may be the length along the linear body 10a from the end 10b, which is one fixed position of the linear body 10a, to the end 10c, which is the other fixed position. When the change in the layout of the linear body 10a is a change in the length, the reception unit 34 may receive the changed length. The fixed position of the linear body 10a may be, for example, a fixed position of the end of the linear body 10a. The fixed position may be changeable within a predetermined range, for example. For example, when the linear body 10a is a distribution cable, the fixed position of the end may be changeable by about several centimeters by adding an extension adapter to the end of the linear body 10a. When the change in the layout of the linear body 10a is a change in the fixed position, the reception unit 34 may receive an identifier that identifies the fixed position after the change. In this case, it is preferable that the correspondence between the fixed position of the linear body 10a and the identifier that identifies the fixed position is held in advance in the teaching data storage device 3. Furthermore, if the change in the way the linear body 10a is handled is a change in the length of the linear body 10a or a change in the fixed position, the part of the linear body 10a in the three-dimensional model 10 of the virtual robot may be changed in response to the acceptance of the change. This change may be performed by, for example, the image generating unit 35. In this embodiment, a case where the change in the way the linear body 10a is handled is a change in the amount of initial twist of the linear body 10a will be mainly described.

The reception unit 34 may receive information other than the above. For example, the reception unit 34 may receive a notification that teaching has ended. For example, the reception unit 34 may receive information input from an input device or the display device 4, or may receive information transmitted via a wired or wireless communication line. The reception unit 34 may or may not include a device for receiving the information (for example, an input device or a communication device). The reception unit 34 may be realized by hardware, or may be realized by software such as a driver that drives a specific device.

The image generating unit 35 generates a display image for displaying the three-dimensional model 10 at a predetermined position in the real environment based on the three-dimensional model 10 stored in the model storage unit 31 and the relative positional relationship acquired by the positional relationship acquisition unit 33. When an operation on the three-dimensional model 10 of the virtual robot is accepted, the image generating unit 35 generates a display image for displaying the operated three-dimensional model 10. The image generating unit 35 may change the angle of each joint of the three-dimensional model 10 stored in the model storage unit 31 according to the accepted operation. Displaying the three-dimensional model 10 at a predetermined position in the real environment may be, for example, displaying the three-dimensional model 10 so that the robot 1, which is the real robot, and the three-dimensional model 10 of the virtual robot have a predetermined positional relationship, or, as described later, displaying the three-dimensional model 10 so that the reference marker and the three-dimensional model 10 of the virtual robot have a predetermined positional relationship. The predetermined positional relationship may be, for example, a predetermined positional relationship, or a positional relationship that can be changed by the operator. The three-dimensional model 10 of the virtual robot may be displayed at the position of the robot 1 or the reference marker, for example, or at a position different from the robot 1 or the reference marker. Displaying the three-dimensional model 10 of the virtual robot at the position of the robot 1 means displaying the three-dimensional model 10 so that the virtual robot is placed at the same position as the robot 1. For example, the three-dimensional model 10 may be displayed so that the end portion on the base end side of the three-dimensional model 10 of the virtual robot (for example, the attachment portion to the floor surface, etc.) overlaps with the end portion on the base end side of the robot 1. Displaying the three-dimensional model 10 of the virtual robot at the position of the reference marker may be displayed so that the situation in which a real robot is placed at the position of the reference marker is virtually reproduced. For example, the three-dimensional model 10 of the virtual robot may be displayed so that the end surface on the base end side of the three-dimensional model 10 of the virtual robot (for example, the attachment surface to the floor surface, etc.) coincides with the surface of the reference marker. When the three-dimensional model 10 of the virtual robot is displayed at a position different from the robot 1 or the reference marker, for example, the three-dimensional model may be displayed next to the robot 1 or the reference marker. In either case, the three-dimensional model 10 of the virtual robot is displayed so that its position in the real environment does not change. Therefore, even if the worker changes the orientation of the display device 4, the display position of the three-dimensional model 10 of the virtual robot in the real environment does not change.

The positions of the robot 1 and the reference marker in the world coordinate system are known. For example, the positional relationship between the robot 1 and the reference marker and the three-dimensional model 10 of the virtual robot is also determined. Therefore, by using these pieces of information, the image generation unit 35 can place the three-dimensional model 10 of the virtual robot so that it has a predetermined positional relationship with the position of the robot 1 in a virtual space in which the positions of the robot 1 and the reference marker are determined in advance. The angles of each joint in the three-dimensional model 10 of the virtual robot are set to initial values when no operation is performed, and are set to values after operation when an operation is performed. The angles of each joint after operation may be calculated by inverse kinematics using the position and posture of the hand in the three-dimensional model 10 of the virtual robot after operation, for example, as with a real robot. Furthermore, the image generation unit 35 can know the relationship between the world coordinate system and the display coordinate system, which is the local coordinate system of the display device 4, based on the relative positional relationship acquired by the positional relationship acquisition unit 33, and therefore can specify the position and orientation of the display device 4 in the virtual space. Therefore, the image generating unit 35 can generate a two-dimensional display image for displaying the three-dimensional model 10 by rendering the three-dimensional model 10 of the virtual robot in the virtual space based on the position and orientation of the display device 4. When the three-dimensional model 10 of the virtual robot is operated, the shape of the three-dimensional model 10 of the virtual robot in the virtual space is changed accordingly, as described above. Furthermore, when the position or orientation of the display device 4 is changed in the real environment, the position and direction of the viewpoint in the virtual space are changed accordingly. Then, rendering is performed after the change, thereby generating a display image of the three-dimensional model 10 after the operation and a display image of the three-dimensional model 10 after the change in the position or orientation of the display device 4. Note that the image generating unit 35 generates the display image such that the size of the display image is consistent with the real environment when the display image of the three-dimensional model 10 of the virtual robot is displayed on the display of the display device 4. In other words, the display image is generated so that the three-dimensional model 10 of the virtual robot displayed on the display of the display device 4 and the real robot placed in the real environment so as to have the same relative positional relationship as the three-dimensional model 10 appear to be the same size when viewed through the display device 4.

It may not be possible to operate the virtual robot according to the accepted operation. For example, an operation to move the hand of the three-dimensional model 10 of the virtual robot beyond the movable range or rotate it beyond the rotatable range may be accepted. In such a case, the image generating unit 35 may not generate a display image according to the operation, or may generate a display image of the three-dimensional model 10 of the virtual robot that has been moved or rotated within the possible range.

The image generating unit 35 may generate a display image for displaying a teaching position according to the teaching data. That is, the teaching position may also be displayed in the display image. By displaying the teaching position, the worker can know the taught position. Displaying the teaching position may mean, for example, displaying a teaching point, or displaying a teaching line that is an interpolation result of a plurality of teaching points. Note that whether the position information included in the teaching data is information in the robot's local coordinate system or in the world coordinate system, the positional relationship between the three-dimensional model 10 of the virtual robot and the teaching position can be known from the teaching data. Therefore, since the teaching position in the virtual space is determined, the image generating unit 35 can generate a display image including the three-dimensional model 10 of the virtual robot and the teaching position in the same manner as described above.

The image generating unit 35 may generate a display image for displaying the amount of twist acquired by the twist amount acquiring unit 38, for example. The amount of twist may be displayed, for example, by a numerical value or a character string, or may be displayed by a color. In the former case, a character string such as "Twist amount: 14 degrees" may be displayed, for example. In the latter case, the image generating unit 35 may generate a display image for displaying the amount of twist by a color, and, for example, the linear body 10a in the three-dimensional model 10 of the virtual robot may be displayed in a color corresponding to the amount of twist, or the entire three-dimensional model 10 of the virtual robot may be displayed in a color corresponding to the amount of twist. For example, the color corresponding to the amount of twist may be blue when the absolute value of the amount of twist is 0 degrees or more and less than 10 degrees, green when it is 10 degrees or more and less than 15 degrees, yellow when it is 15 degrees or more and less than 20 degrees, and red when it is 20 degrees or more. The image generating unit 35 may also generate a display image for displaying an alert, which will be described later. The alert may be displayed, for example, as a text string such as "Warning" or a graphic, or may be displayed by highlighting part or all of the displayed image (for example, by blinking or changing the color to a warning color such as red).

The output unit 36 outputs the display image generated by the image generation unit 35 to the display device 4. The output unit 36 may output only the display image. In this case, the display image is displayed on the display device 4 by superimposing it on the image of the real environment or the real environment itself. On the other hand, if the display device 4 has a non-transparent display and the image of the real environment captured by the display device 4 is accepted by the teaching data storage device 3, the result of combining the image of the real environment and the display image may be output to the display device 4. This combination may be performed, for example, by a combination unit (not shown) included in the teaching data storage device 3.

The output unit 36 may also output information about the amount of twist acquired by the amount of twist acquisition unit 38. This output may be, for example, the amount of twist itself, or an alert indicating that the absolute value of the amount of twist has exceeded a threshold value. The output of the amount of twist itself may be, for example, a display image for displaying the amount of twist, or an output indicating the amount of twist by sound or the like. The output of the alert may be, for example, a display image for displaying the alert, or an output indicating the alert by sound or by turning on a warning light or the like. Note that, for example, if the absolute value of the amount of twist does not exceed the threshold value, an alert may not be output. The display image output by the output unit 36 may be, for example, one generated by the image generation unit 35.

When the reception unit 34 receives a teaching instruction for the three-dimensional model 10 of the virtual robot, the accumulation unit 37 accumulates teaching data in the teaching data storage unit 32 according to the three-dimensional model 10 of the virtual robot. That is, the accumulation unit 37 accumulates information indicating the position and posture of the three-dimensional model 10 displayed when the teaching instruction is received in the teaching data storage unit 32 as teaching data. Usually, teaching data corresponding to one teaching point is accumulated in response to the reception of one teaching instruction. By repeating the accumulation of this teaching data, for example, teaching data corresponding to the path of the robot's hand is stored in the teaching data storage unit 32. As described above, the teaching data to be accumulated may be, for example, information indicating the position and posture of the hand of the three-dimensional model 10 for each teaching point, or information indicating the angle of each joint of the three-dimensional model 10 for each teaching point. Note that the teaching data may include, for example, information indicating the teaching position. This is to make it possible to display the teaching position. The storage unit 37 may acquire information necessary for storing teaching data, such as the position and orientation of the hand in the robot coordinate system, which is the local coordinate system of the 3D model 10 of the virtual robot, or information such as the angle of each joint, from the image generation unit 35, from the 3D model 10 stored in the model storage unit 31, or from another configuration. The position and orientation of the hand in the robot coordinate system of the 3D model 10 of the virtual robot can be acquired, for example, by transforming the position and orientation of the hand in the virtual space using a homogeneous transformation matrix that indicates the transformation between the virtual space and the robot coordinate system of the 3D model 10 of the virtual robot.

The twist amount acquisition unit 38 acquires the twist amount of the linear body 10a attached to the three-dimensional model of the robot body of the virtual robot. The twist amount to be acquired is the twist amount of the linear body 10a as a whole. The twist amount may be acquired, for example, by performing a simulation of the linear body 10a. This simulation may be, for example, the simulation described in the above-mentioned Patent Document 1, a simulation using a commercially available simulation tool for a linear body such as a cable or a hose, or other physical simulation. This simulation may be performed, for example, using a three-dimensional model of the linear body 10a. The three-dimensional model of the linear body 10a may be, for example, a part of the three-dimensional model 10 of the virtual robot, or may be a three-dimensional model separate from it. In addition, when the twist amount is acquired by performing a simulation of the linear body 10a, parameters of the linear body 10a required for the simulation, for example, parameters indicating elastic properties such as length, thickness, and Young's modulus, etc., are set in advance, and the simulation may be performed using these parameters. The twist amount acquiring unit 38 can specify the positions and orientations of the linear body 10a with respect to the fixed positions of both ends of the linear body 10a in the three-dimensional model 10 by using the current three-dimensional model 10. The twist amount acquiring unit 38 may then acquire the twist amount of the linear body 10a by performing a simulation of the linear body 10a using the specified positions and orientations. Note that, for example, as shown in FIG. 1, when the position of a part of the linear body 10a is determined other than the fixed positions, such as when the linear body 10a passes through the inside of the housing of the three-dimensional model 10 of the virtual robot, the simulation may be performed using the determined position as a constraint condition. For example, the twist amount acquiring unit 38 may acquire the twist amount at predetermined intervals in the longitudinal direction of the linear body 10a, and may acquire the total twist amount of the linear body 10a by accumulating the twist amounts at the predetermined intervals. The twist amount acquisition unit 38 may acquire the twist amount in real time so that the twist amount according to the current shape of the three-dimensional model 10 can be known, or may acquire the twist amount at a predetermined time interval or each time a predetermined event (for example, reception of a teaching instruction) occurs. In this embodiment, the case where the twist amount is acquired in real time will be mainly described. Even when the twist amount is acquired in real time, the twist amount does not change when no operation is performed on the three-dimensional model 10. Therefore, the twist amount acquisition unit 38 may acquire a new twist amount, for example, each time an operation on the three-dimensional model 10 is accepted. In addition, the acquired twist amount may be stored in the teaching data storage unit 32 in association with the teaching data when the teaching data is accumulated. The twist amount stored in association with the teaching data is the twist amount corresponding to the shape of the three-dimensional model 10 corresponding to the teaching data. Additionally, the positive and negative amounts of twist may be defined such that, for example, in the longitudinal direction of the linear body 10a, from the base end 10b toward the hand end 10c, a clockwise twist is positive and a counterclockwise twist is negative, or vice versa.

In addition, when the reception unit 34 receives a change in the handling of the linear body 10a, the twist amount acquisition unit 38 may acquire the twist amount of the linear body 10a corresponding to the received changed handling. For example, when the initial twist amount is changed, the twist amount acquisition unit 38 may acquire the twist amount of the linear body 10a corresponding to the changed initial twist amount. More specifically, if the twist amount of the linear body 10a acquired for an initial twist amount of 0 degrees is 15 degrees, when the initial twist amount is subsequently changed to -10 degrees, the twist amount acquisition unit 38 may acquire the twist amount of the linear body 10a of 5 degrees (= 15 degrees - 10 degrees) corresponding to the changed initial twist amount. Note that the twist amount after the change in handling may be acquired, for example, by performing a simulation again for the linear body 10a after the change in handling. If the change in handling is, for example, a change in the length of the linear body 10a or a change in the fixed position of the linear body 10a, it is preferable to perform a simulation again to obtain the amount of twist corresponding to the changed handling.

Next, the operation of the teaching data storage device 3 will be described with reference to the flowchart of FIG.
(Step S101) The positional relationship acquisition unit 33 acquires the relative positional relationship between the real environment and the display device 4.

(Step S102) The twist amount acquisition unit 38 acquires the twist amount of the linear body 10a according to the shape of the three-dimensional model 10 of the virtual robot at this time. Note that when a change in the handling of the linear body 10a is accepted, the twist amount of the linear body 10a according to the changed handling is acquired.

(Step S103) The image generation unit 35 uses the 3D model 10 of the virtual robot, the relative positional relationship acquired in step S101, and the acquired amount of twist to generate a display image for displaying the 3D model 10 of the virtual robot, the display image being for displaying according to the amount of twist of the linear body 10a. Note that the 3D model 10 to be displayed has an initial shape before the operation is accepted, and has a shape after the operation is accepted.

(Step S104) The output unit 36 outputs the generated display image to the display device 4.

(Step S105) The reception unit 34 determines whether an operation on the three-dimensional model 10 of the virtual robot has been received. If an operation on the three-dimensional model 10 of the virtual robot has been received, the process proceeds to step S106, and if not, the process proceeds to step S110.

(Step S106) The positional relationship acquisition unit 33 acquires the relative positional relationship between the real environment and the display device 4.

(Step S107) The twist amount acquisition unit 38 acquires the twist amount of the linear body 10a according to the shape of the three-dimensional model 10 of the virtual robot at this time.

(Step S108) The image generation unit 35 uses the three-dimensional model 10 of the virtual robot, the relative positional relationship acquired in step S106, and the amount of twist acquired in step S107 to generate a display image for displaying the three-dimensional model 10 of the virtual robot after operation, the display image being based on the amount of twist of the linear body 10a.

(Step S109) The output unit 36 outputs the generated display image to the display device 4. Then, the process returns to step S105.

(Step S110) The reception unit 34 determines whether a teaching instruction has been received. If a teaching instruction has been received, the process proceeds to step S111. If not, the process proceeds to step S112.

(Step S111) The accumulation unit 37 accumulates the teaching data corresponding to the position and posture of the 3D model 10 of the virtual robot at that time in the teaching data storage unit 32. Then, the process returns to step S105.

(Step S112) The image generating unit 35 determines whether or not to generate a display image. If a display image is to be generated, the process returns to step S101. If not, the process proceeds to step S113. Note that the image generating unit 35 may, for example, periodically determine to generate a display image. By making this determination, for example, even if no operation is performed, when the position or orientation of the display device 4 is changed, a display image corresponding to the changed position or orientation will be displayed on the display device 4.

(Step S113) The reception unit 34 determines whether a change in the routing of the linear body 10a has been received. If a change in the routing has been received, the process returns to step S101; if not, the process proceeds to step S114.

(Step S114) The image generating unit 35 determines whether the teaching of the teaching data has been completed. If the teaching has been completed, the series of processes for generating the teaching data is completed, and if not, the process returns to step S105. The image generating unit 35 may determine that the teaching of the teaching data has been completed, for example, when the receiving unit 34 receives information indicating that the accumulation of the teaching data has been completed.

In the flowchart of FIG. 2, when it is determined that teaching has ended, teaching data may be output. Also, the order of the processes in the flowchart of FIG. 2 is an example, and the order of each step may be changed as long as the same results are obtained.

Next, the operation of the teaching data storage device 3 according to this embodiment will be described using a specific example. In this specific example, a case will be described in which a worker wearing a head-mounted display device 4 performs teaching along a welding line. In addition, the initial twist amount of the linear body 10a is set to "0 degrees" at the start of the teaching operation.

First, when the operator starts the teaching data storage device 3, the positional relationship acquisition unit 33 acquires the relative positional relationship between the robot 1 arranged in the real environment and the display device 4 using the captured image received from the display device 4, and passes it to the image generation unit 35 (step S101). In addition, the twist amount acquisition unit 38 acquires the twist amount of the linear body 10a attached to the three-dimensional model of the robot body of the virtual robot at that time (step S102). Then, the image generation unit 35 uses the relative positional relationship, the three-dimensional model 10 stored in the model storage unit 31, and the acquired twist amount to generate a display image for displaying the three-dimensional model 10 of the virtual robot at the position of the reference marker 6, and passes it to the output unit 36 (step S103). Upon receiving the display image, the output unit 36 outputs the display image to the display device 4 (step S104). As a result, the operator can see the three-dimensional model 10 displayed at the position of the reference marker 6. The display image may be updated periodically (steps S112, S101 to S104).

Here, the reference marker 6 is a predetermined two-dimensional image. The reference marker 6 may be, for example, an AR marker, a QR code (registered trademark), or any other two-dimensional image with a predetermined shape. The size of the reference marker 6 may be, for example, predetermined. The reference marker 6 may be, for example, printed on paper or a resin sheet. When the three-dimensional model 10 of the virtual robot is displayed at the position of the reference marker 6, the operator can place the reference marker 6 at a desired position so that the three-dimensional model 10 is displayed at the position of the reference marker 6.

The three-dimensional model 10 may be displayed at a position other than the reference marker 6. Even in this case, it is preferable that the three-dimensional model 10 is displayed so as to have a predetermined positional relationship with the reference marker 6. For example, the three-dimensional model 10 may be displayed next to the reference marker 6.

Figure 3 is a diagram showing an example of the display of a three-dimensional model 10 of a virtual robot. As shown in Figure 3, the worker can see the three-dimensional model 10 of the virtual robot displayed at the position of the reference marker 6, and a display 51 showing the amount of twist. In this way, the worker can know the amount of twist according to the shape of the three-dimensional model at that time.

Then, the worker operates the three-dimensional model 10 by an input device such as a teaching pendant or gestures, etc., so that the tip of the welding torch of the three-dimensional model 10 is at the position of the weld line. As a result, a display image for displaying the three-dimensional model 10 and the amount of twist after the operation is output to the display device 4, and the worker can check the position and posture of the welding torch of the three-dimensional model 10 after the operation, and the corresponding amount of twist of the linear body 10a (steps S105 to S109). After that, when the worker inputs a teaching instruction by an input device such as a teaching pendant or gestures, teaching data corresponding to the teaching point is accumulated in the teaching data storage unit 32 according to the teaching instruction (steps S110, S111). Note that the amount of twist at that time may also be accumulated in association with the accumulation of teaching data. After that, teaching data is accumulated in the same manner for other teaching points of the weld line (steps S105 to S111).

The operator can know how much the linear body 10a is twisted by the displayed twist amount. Therefore, for example, when the absolute value of the twist amount exceeds the allowable range, the operator can change the posture of the welding torch of the three-dimensional model 10 to teach a posture in which the absolute value of the twist amount is smaller. In addition, when the absolute value of the twist amount cannot be reduced by changing the posture of the welding torch alone, the operator may change the handling of the linear body 10a. For example, when the operator wants to make the absolute value of the twist amount 20 degrees or less, when the twist amount in the posture of the three-dimensional model 10 corresponding to the new teaching point becomes 21 degrees, the operator may input the initial twist amount "-10 degrees" via an input device in the real environment or a virtual input interface. The input change in handling is accepted by the acceptance unit 34 and passed to the twist amount acquisition unit 38 (step S113). Then, when the twist amount acquisition unit 38 acquires a new twist amount, it may acquire a twist amount of "11 degrees" (= 21 degrees - 10 degrees) according to the initial twist amount of "-10 degrees", which is the handling after the change, and display an image including that twist amount (steps S101 to S104).

In this way, the operator can perform teaching while appropriately changing the handling of the linear body 10a during teaching. Therefore, it is possible to avoid a situation in which the absolute value of the twist amount is found to exceed the allowable range in a simulation after teaching, and all teaching data is wasted. In addition, if the absolute value of the twist amount cannot be suppressed within the allowable range even by changing the posture of the welding torch or changing the handling of the linear body 10a, for example, the teaching work may be stopped and new teaching may be performed by changing the positional relationship between the three-dimensional model 10 of the virtual robot and the workpiece to be welded. Even in this case, for example, it is possible to reduce the amount of wasted teaching data. Note that after teaching is completed (step S114), the teaching data stored in the teaching data storage unit 32 may be output. In addition, if the handling of the linear body 10a is changed during teaching, the teaching data may be output together with information indicating the changed handling (for example, the initial twist amount, etc.).

In addition, when the reception unit 34 receives a change in the handling of the linear body 10a, the twist amount acquisition unit 38 may also acquire a representative value of the twist amount corresponding to the teaching data stored in the teaching data storage unit 32 according to the changed handling. In other words, when the handling of the linear body 10a is changed, the twist amount acquisition unit 38 may acquire the twist amount of the linear body 10a according to the position and posture of the three-dimensional model 10 of the virtual robot corresponding to each teaching data accumulated up to that point, and acquire one or more representative values of the acquired twist amounts. The representative value may be, for example, a maximum value and a minimum value, or may be the maximum absolute value of the twist amount. In this case, the output unit 36 may also output the representative value of the twist amount acquired by the twist amount acquisition unit 38. The output of the representative value may be, for example, the output of the representative value itself, or an alert indicating that the representative value has exceeded a threshold value. Furthermore, when a display image for displaying a representative value or a display image for displaying an alert is output, the display image may be generated, for example, by the image generating unit 35. Furthermore, for example, when the amount of twist is stored in association with the teaching data, when a new amount of twist is acquired for each of one or more pieces of teaching data stored in accordance with the changed handling of the linear body 10a, the new amount of twist may also be stored in association with the teaching data. In this case, for example, the amount of twist may or may not be overwritten. Even in the latter case, it is preferable that the latest amount of twist is stored so as to be distinguishable. Furthermore, the new amount of twist may be stored in association with information indicating the changed handling, for example.

For example, in the above specific example, when the current twist amount becomes "21 degrees", the teaching data storage unit 32 stores information associating the teaching data with the twist amount as shown in FIG. 4. Then, when the initial twist amount "-10 degrees" is received, the twist amount acquisition unit 38 may subtract 10 degrees from each of the twist amounts associated with the teaching data in FIG. 4, overwrite and accumulate the results, and acquire the maximum value "9 degrees" and minimum value "-3 degrees" of the subtraction results. Then, a display image including the maximum value "9 degrees" and the minimum value "-3 degrees" may be generated and output. In this way, the operator can check not only the current twist amount according to the change in the handling of the linear body 10a, but also the twist amount according to the previous teaching data. Therefore, for example, even if a change in the way the linear body 10a is handled brings the current amount of twist into the allowable range, if the amount of twist according to the previous teaching data does not fall within the allowable range, the way the linear body 10a is handled can be changed again to adjust both the current amount of twist and the past amount of twist into the desired range. Note that, for example, information indicating the received change in handling may also be stored in the teaching data storage unit 32 in association with the teaching data.

As described above, according to the teaching data storage device 3 of this embodiment, when teaching using the three-dimensional model 10 of the virtual robot, the amount of twist of the linear body 10a can be known. Therefore, when the absolute value of the amount of twist is large, for example, by changing the posture of the hand of the three-dimensional model 10, teaching can be performed so that the absolute value of the amount of twist becomes smaller. Also, when the absolute value of the amount of twist cannot be reduced by changing the posture alone, the absolute value of the amount of twist can be reduced by changing the way the linear body 10a is handled. At that time, for example, a representative value of the amount of twist according to the teaching data corresponding to the taught teaching point is also acquired and output, so that the amount of twist for each taught teaching point can be within a desired range. Also, when the amount of twist is displayed in color, the operator can intuitively know the amount of twist without checking the numerical value indicating the amount of twist.

In this embodiment, the case where the operator changes the handling of the linear body 10a through trial and error has been mainly described, but this is not necessary. For example, a change in the handling of the linear body 10a may be proposed so that the representative value of the amount of twist corresponding to the teaching data approaches zero, and the handling may be changed accordingly. In this case, the teaching data storage device 3 may further include a specification unit 39, for example, as shown in FIG. 5. The specification unit 39 may specify a change in the handling of the linear body 10a so that the representative value of the amount of twist corresponding to the teaching data stored in the teaching data storage unit 32 approaches zero. The change in the handling of the linear body 10a so that the representative value of the amount of twist corresponding to the teaching data approaches zero may be a change in the handling so that the representative value of the amount of twist corresponding to the teaching data after the change is closer to zero than the representative value of the amount of twist corresponding to the teaching data before the change. The specification unit 39 may perform the specification in response to, for example, a specific instruction being accepted by the acceptance unit 34. When the identification unit 39 identifies a change in the handling of the linear body 10a, the output unit 36 may also output information about the change in handling of the linear body 10a identified by the identification unit 39. This output may be, for example, an output of information indicating the identified change in handling. This output may be, for example, an output of a display image indicating the identified change in handling, or an output of a sound or the like indicating the identified change in handling. The display image may be generated by the image generation unit 35, for example.

For example, the determination unit 39 may determine the change in the handling of the linear body 10a by optimizing an objective function according to a representative value of the amount of twist after the change for each teaching data. That is, the determination unit 39 may determine the change in the handling that is the optimal solution for optimizing the objective function as the change in the handling of the linear body 10a so that the representative value of the amount of twist corresponding to the teaching data approaches zero. The optimization of the objective function may be, for example, minimization of the objective function. When the representative value of the amount of twist is the maximum absolute value of the amount of twist, the objective function may be, for example, an increasing function of the maximum absolute value of the amount of twist. For example, when the amount of twist for each teaching data is acquired as shown in FIG. 4, the determination unit 39 may determine the initial amount of twist "-13 degrees" so that the maximum absolute value of the amount of twist is minimized. In response to the output of the determination result, the operator may input the initial amount of twist "-13 degrees" to the teaching data storage device 3 as the handling after the change. This allows the worker to make appropriate changes to the handling of the linear body 10a without having to resort to trial and error.

In addition, in this embodiment, the image generating unit 35 may generate a display image for displaying the linear body 10a present inside the housing of the three-dimensional model 10 of the virtual robot in a see-through manner. In this way, the operator can check the shape of the linear body 10a including the inside of the housing, and can, for example, provide instructions to prevent excessive force from being applied to the linear body 10a. Note that the linear body 10a present inside the housing may, for example, be a linear body 10a that is not visible from outside the housing. Also, the linear body 10a present inside the housing may, for example, be a part of the linear body 10a.

In addition, in this embodiment, the case where the twist amount is acquired in real time by the twist amount acquisition unit 38 has been mainly described, but this is not necessary. For example, the twist amount may be acquired each time teaching data is accumulated. In this case, if the operator determines after inputting a teaching instruction that the twist amount according to that instruction exceeds the allowable range, the operator may cancel the latest teaching and perform the teaching again.

In addition, in this embodiment, the case where teaching for welding is mainly described, however, it goes without saying that teaching for other purposes may also be performed. When teaching for purposes other than welding is performed, the above-mentioned welding line may be, for example, a work line or a movement path of the robot's hand.

In addition, in this embodiment, the case where the reference marker 6 is a specified two-dimensional image has been mainly described, but this is not necessarily the case. As with markerless AR, an object that exists in the real environment may be used as the reference marker. There is no particular limitation on the object that exists in the real environment, but it may be, for example, a base part used to install the real robot, or a jig that exists near the position where the robot is to be placed. Also, a marker on an object with a three-dimensional shape (e.g., a rectangular parallelepiped shape) with markers attached to each surface may be used as the reference marker. The reference marker may also be three-dimensional, such as a base or a jig.

In addition, in this embodiment, the case where the handling of the linear body 10a can be changed has been mainly described, but this is not necessarily the case. The handling of the linear body 10a does not necessarily have to be changeable. Even in this case, it is possible to perform teaching that reduces the amount of twisting by appropriately changing the posture of the hand of the three-dimensional model 10, for example.

In addition, in the above embodiment, each process or function may be realized by centralized processing by a single device or a single system, or may be realized by distributed processing by multiple devices or multiple systems. For example, at least a portion of the configuration of the teaching data storage device 3 may be physically included in a device having a display. Therefore, the division of the devices shown in FIG. 1 may be considered to be for convenience according to function, rather than according to physical devices.

In addition, in the above embodiment, when two or more components included in the teaching data storage device 3 have a communication device, an input device, etc., the two or more components may have a single physical device, or may have separate devices.

In the above embodiment, each component may be configured by dedicated hardware, or a component that can be realized by software may be realized by executing a program. For example, each component may be realized by a program execution unit such as a CPU reading and executing a software program recorded on a recording medium such as a hard disk or semiconductor memory. During execution, the program execution unit may execute the program while accessing the storage unit or recording medium. The program may be executed by being downloaded from a server or the like, or may be executed by reading a program recorded on a predetermined recording medium (e.g., an optical disk, a magnetic disk, a semiconductor memory, etc.). This program may be used as a program that constitutes a program product. The computer that executes the program may be a single computer or multiple computers. In other words, centralized processing or distributed processing may be performed.

The above embodiments are merely examples for specifically implementing the present invention, and are not intended to limit the technical scope of the present invention. The technical scope of the present invention is indicated by the claims, not by the description of the embodiments, and is intended to include modifications within the literal scope of the claims and within the scope of equivalent meanings.

3 Teaching data storage device, 31 Model storage unit, 32 Teaching data storage unit, 33 Positional relationship acquisition unit, 34 Reception unit, 35 Image generation unit, 36 Output unit, 37 Storage unit, 38 Twist amount acquisition unit, 39 Identification unit

Claims (6)

a model storage unit in which a three-dimensional model of a virtual robot corresponding to a real robot is stored;
a teaching data storage unit in which teaching data of the actual robot is stored;
a positional relationship acquisition unit that acquires a relative positional relationship between a real environment and a display device that displays an image of the real environment or the real environment itself by superimposing the image;
a reception unit that receives an instruction for operation and instruction for the three-dimensional model of the virtual robot;
an image generation unit that generates a display image for displaying the three-dimensional model of the operated virtual robot at a predetermined position in a real environment based on the three-dimensional model of the virtual robot and the relative positional relationship;
an output unit that outputs the display image generated by the image generation unit to the display device;
a storage unit that stores teaching data in the teaching data storage unit according to the three-dimensional model of the virtual robot when the instruction for teaching the three-dimensional model of the virtual robot is accepted by the accepting unit;
a torsion amount acquisition unit that acquires a torsion amount of a linear body attached to a three-dimensional model of a robot body of a virtual robot,
The output unit also outputs information related to the amount of twist acquired by the amount of twist acquisition unit. The reception unit also receives a change in the routing of the linear object,
The teaching data storage device according to claim 1 , wherein the twist amount acquisition unit acquires an amount of twist of the linear body according to the changed handling accepted by the acceptance unit. when the reception unit receives a change in the handling of the linear body, the twist amount acquisition unit also acquires a representative value of the twist amount corresponding to the teaching data stored in the teaching data storage unit in accordance with the changed handling,
3. The teaching data storage device according to claim 2, wherein the output unit also outputs a representative value of the amount of twist acquired by the amount of twist acquisition unit. a specifying unit that specifies a change in the routing of the linear body so that a representative value of the amount of twist corresponding to the teaching data stored in the teaching data storage unit approaches zero,
3. The teaching data storage device according to claim 2, wherein the output unit also outputs information related to a change in the way the linear body is handled, the change being specified by the specification unit. A teaching data storage device according to any one of claims 2 to 4, wherein the change in the handling of the linear body is at least one of a change in the initial twist amount of the linear body, a change in the length of the linear body, and a change in the fixed position of the linear body. The teaching data storage device according to any one of claims 1 to 4, wherein the image generating unit generates a display image that displays the amount of twist acquired by the amount of twist acquiring unit using color.

Images