JP2018001332A - Robot, control device, and robot system - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a robot capable of reducing variations due to an artificial error and a worker in comparison with the case where an operator sets an offset of a tool while teaching a position of a reference point by operating an arm.SOLUTION: A robot comprises an arm. The arm rotates a tool with a rotating axis passing through a predetermined portion of the tool mounted on the arm and being parallel to an optical axis of an imaging section as a center in a second state moved from a first state where the imaging section can image the predetermined portion, and moves the predetermined portion to a third state having the same position as a position in the first state while holding a change in posture by rotation. The robot is controlled by using an offset of the predetermined portion with respect to the arm set on the basis of images imaged by the imaging section in the first state and the third state.SELECTED DRAWING: Figure 8

Description

  The present invention relates to a robot, a control device, and a robot system.

  2. Description of the Related Art Conventionally, processing for setting an offset of a tool with respect to an arm has been performed before processing a workpiece using a tool attached to the arm. Patent Document 1 discloses a method for deriving an offset of a tool with respect to an arm based on a result of performing an operation of aligning a tool attached to an arm with a reference point in real space a plurality of times while changing the posture of the arm. It is disclosed.

JP-A-8-85083

  According to the technique described in Patent Document 1, it is necessary to teach the position of the reference point by the operator operating the arm so as to touch the reference point with a tool. However, it is not easy to accurately operate the arm while visually identifying the boundary state whether the arm touches or does not touch the reference point. That is, with the technique described in Patent Document 1, it may be difficult to accurately teach the position of the reference point. If the tool offset is set while accurately teaching the position of the reference point, the required time for setting may be prolonged, and the required time becomes longer as the number of robots to be set increases.

In order to solve at least one of the above problems, an aspect of the present invention includes an arm, and the arm is capable of imaging a predetermined part of a tool attached to the arm by an imaging unit. In the second state, the tool is rotated about a rotation axis that passes through the predetermined part and is parallel to the optical axis of the imaging unit, and the predetermined part is maintained in the posture change due to the rotation. It is moved to a third state that is the same position as the position in the first state, and an offset with respect to the arm of the predetermined part set based on images captured by the imaging unit in the first state and the third state is used. Controlled robot.
With this configuration, the robot is set with an offset with respect to the arm at the predetermined part based on images captured by the imaging unit in the first state and the third state. As a result, the robot can reduce human error and variations due to the operator, rather than setting the tool offset while teaching the position of the reference point by the operator operating the arm.

According to another aspect of the present invention, in the robot, the arm moves the tool from the first state to a second state without the obstacle when there is an obstacle in the first state. May be used.
With this configuration, the robot rotates the tool around a rotation axis that passes through a predetermined part and is parallel to the optical axis of the imaging unit at a position where there is no obstacle. As a result, the robot can avoid the collision with the obstacle and rotate the tool around the rotation axis parallel to the optical axis of the imaging unit through the predetermined part. Even when there is an object, the offset can be calculated with high accuracy.

Moreover, the structure which is a control apparatus which controls the robot as described in the other aspect of this invention may be used.
With this configuration, the control device sets the offset for the arm at the predetermined part in the robot based on the images captured by the imaging unit in the first state and the third state. Accordingly, the control device can set the offset with higher accuracy than setting the offset of the tool in the robot while teaching the position of the reference point by operating the arm by the operator.

Moreover, the structure which is a robot system provided with the robot described above and the control apparatus described above may be used for the other aspect of this invention.
With this configuration, in the robot system, the control device sets the offset for the arm at the predetermined site in the robot based on the images captured by the imaging unit in the first state and the third state. Thereby, in the robot system, the control device can set the offset with higher accuracy than setting the offset of the tool in the robot while teaching the position of the reference point by operating the arm by the operator.

As described above, the robot sets the offset with respect to the arm at the predetermined part based on the images captured by the imaging unit in the first state and the third state. As a result, the robot can reduce human error and variations due to the operator, rather than setting the tool offset while teaching the position of the reference point by the operator operating the arm.
Further, the control device sets an offset with respect to the arm of the predetermined part in the robot based on the images captured by the imaging unit in the first state and the third state. Thus, the control device can set the offset with higher accuracy than setting the offset of the tool while teaching the position of the reference point by operating the arm by the operator.
Further, in the robot system, the control device sets an offset for the arm of the predetermined part in the robot based on the images captured by the imaging unit in the first state and the third state. As a result, in the robot system, the control device sets the offset of the tool with accuracy and more easily than setting the offset of the tool in the robot while teaching the robot the position of the reference point by operating the arm. be able to.

1 is a schematic perspective view showing a robot system according to an embodiment. It is the schematic of the robot shown in FIG. It is a block diagram of the robot system shown in FIG. It is a flowchart which shows the calibration method of the imaging part using the robot system shown in FIG. FIG. 5 is a plan view of a calibration member used for calibration of the imaging unit shown in FIG. 4. 5 is a flowchart for explaining calibration of the mobile camera shown in FIG. 4. It is the schematic of the robot for demonstrating calibration of the mobile camera shown in FIG. It is a flowchart for demonstrating the process which calculates | requires the offset component shown in FIG. It is a figure for demonstrating the process which calculates | requires offset component (DELTA) u, (DELTA) v, and (DELTA) w shown in FIG. It is a flowchart for demonstrating the process which calculates | requires offset component (DELTA) x and (DELTA) y shown in FIG. It is a figure for demonstrating the process which calculates | requires offset component (DELTA) x and (DELTA) y shown in FIG. It is a figure for demonstrating the process which calculates | requires offset component (DELTA) x and (DELTA) y shown in FIG. It is a figure for demonstrating the process which calculates | requires offset component (DELTA) x and (DELTA) y shown in FIG. It is a figure for demonstrating the process which calculates | requires offset component (DELTA) x and (DELTA) y shown in FIG. It is a figure for demonstrating the process which calculates | requires offset component (DELTA) x and (DELTA) y shown in FIG. It is a coordinate diagram for demonstrating the process which calculates | requires offset component (DELTA) x and (DELTA) y shown in FIG. It is a figure for demonstrating the process which calculates | requires offset component (DELTA) z shown in FIG.

  Hereinafter, a robot, a control device, and a robot system according to the present invention will be described in detail based on preferred embodiments shown in the accompanying drawings.

[Embodiment]
≪Robot system≫
FIG. 1 is a schematic perspective view showing a robot system according to an embodiment of the present invention. FIG. 2 is a schematic diagram of the robot shown in FIG. FIG. 3 is a block diagram of the robot system shown in FIG.

  Hereinafter, for convenience of explanation, the upper side (zr direction) in FIG. 2 is referred to as “upper” or “upper”, and the lower side (−zr direction) is referred to as “lower” or “lower”. Also, the vertical direction in FIG. 2 is defined as “vertical direction”, the plane intersecting the vertical direction is defined as “horizontal plane”, and the direction parallel to the horizontal plane is defined as “horizontal direction”. Here, “horizontal” as used in the specification of the present application is not limited to complete horizontal, but includes a case where the horizontal axis is inclined within a range of 5 ° or less. In addition, the term “vertical” as used in the specification of the present application is not limited to complete vertical, but includes a case where the vertical inclination is within a range of 5 ° or less. Further, the base side of the robot shown in FIG. 2 is referred to as “base end”, and the opposite side (hand side) is referred to as “tip”.

A robot system 100 shown in FIG. 1 is an apparatus used in operations such as gripping, transporting, and assembling objects such as electronic components and electronic devices.
As shown in FIG. 1, the robot system 100 includes a robot 1 having a robot arm 10, a fixed camera 2 (imaging unit) having an imaging function fixed in a work area 90, and an imaging function attached to the robot 1. And a control device 5 (calibration device) that controls the robot 1, the fixed camera 2, and the mobile camera 3, respectively.
In the present embodiment, a work table 61 (assembly table) for assembling an object and a supply table 62 for supplying the object by, for example, an operator are provided in the work area 90. The work table 61 and the supply table 62 are each provided within the drive range of the robot arm 10 of the robot 1.

Hereinafter, each unit of the robot system 100 will be sequentially described.
<Robot>
The robot 1 shown in FIGS. 1 and 2 can perform operations such as gripping, conveying, and assembling an object.
The robot 1 is a six-axis vertical articulated robot, and includes a base 101, a robot arm 10 connected to the base 101, and a hand 102 (tool) provided at the tip of the robot arm 10. . As shown in FIG. 3, the robot 1 includes a plurality of driving units 130 that generate power for driving the robot arm 10 and a plurality of motor drivers 120.

The base 101 is a part for attaching the robot 1 to a predetermined location in the work area 90.
The robot arm 10 includes a first arm 11 (arm), a second arm 12 (arm), a third arm 13 (arm), a fourth arm 14 (arm), a fifth arm 15 (arm), And a sixth arm 16 (arm). The first arm 11 is connected to the base 101, and the first arm 11, the second arm 12, the third arm 13, the fourth arm 14, the fifth arm 15, and the sixth arm 16 are on the base end side. Are connected in this order from the tip toward the tip.

  As shown in FIG. 2, the first arm 11 has a rotating shaft member 111 connected to the base 101, and rotates with respect to the base 101 about the central axis of the rotating shaft member 111. It is possible. Further, the second arm 12 has a rotation shaft member 121 connected to the first arm 11, and can rotate with respect to the first arm 11 with the central axis of the rotation shaft member 121 as the rotation center. ing. The third arm 13 has a rotation shaft member 131 connected to the second arm 12, and can rotate with respect to the second arm 12 about the central axis of the rotation shaft member 131. ing. The fourth arm 14 has a rotation shaft member 141 connected to the third arm 13, and can rotate with respect to the third arm 13 with the central axis of the rotation shaft member 141 as the rotation center. ing. Further, the fifth arm 15 has a rotation shaft member 151 connected to the fourth arm 14, and is rotatable with respect to the fourth arm 14 with the central axis of the rotation shaft member 151 as the rotation center. ing. Further, the sixth arm 16 has a rotation shaft member 161 connected to the fifth arm 15, and is rotatable with respect to the fifth arm 15 about the central axis A 6 of the rotation shaft member 161. It has become. Here, a point where the center axis A6 and the tip surface of the sixth arm 16 intersect (the center of the tip surface of the sixth arm 16) is referred to as an axis coordinate O6 (predetermined part).

The hand 102 is attached to the distal end surface of the sixth arm 16, and the center axis of the hand 102 coincides with the center axis A <b> 6 of the sixth arm 16. Here, the center of the front end surface of the hand 102 is referred to as TCP (tool center point). In the present embodiment, it refers to the center of an area between two fingers of the hand 102.
Each of the first arm 11 to the sixth arm 16 is provided with a plurality of drive units 130 each having a motor such as a servo motor and a speed reducer. That is, as shown in FIG. 3, the robot 1 has the number of driving units 130 (six in this embodiment) corresponding to the first arm 11 to the sixth arm 16. The first arm 11 to the sixth arm 16 are each controlled by the control device 5 via a plurality (six in this embodiment) of motor drivers 120 electrically connected to the corresponding driving units 130. Yes.

Each drive unit 130 is provided with an angle sensor (not shown) such as an encoder or a rotary encoder. Thereby, the rotation angle of the rotating shaft of the motor or reduction gear which each drive part 130 has can be detected.
As shown in FIGS. 1 and 2, in this embodiment, the robot coordinate system (the coordinate system of the robot 1) used when controlling the robot 1 is an xr axis and yr parallel to the horizontal direction. A three-dimensional orthogonal coordinate system determined by an axis and a zr axis that is orthogonal to the horizontal direction and has a vertically upward direction as a positive direction is set. The translation component for the xr axis is “component xr”, the translation component for the yr axis is “component yr”, the translation component for the zr axis is “component zr”, and the rotation component around the zr axis is “component ur”. , The rotation component around the yr axis is referred to as “component vr”, and the rotation component around the xr axis is referred to as “component wr”. The unit of the length (size) of the component xr, the component yr, and the component zr is “mm”, and the unit of the angle (size) of the component ur, the component vr, and the component wr is “°”.

<Fixed camera>
The fixed camera 2 shown in FIGS. 1 and 2 has a function of capturing an object or the like.
As shown in FIG. 2, the fixed camera 2 includes an imaging element 21 configured by a CCD (Charge Coupled Device) image sensor having a plurality of pixels, and a lens 22 (optical system). The fixed camera 2 forms an image of light from an object or the like on the light receiving surface 211 (sensor surface) of the image sensor 21 by the lens 22, converts the light into an electric signal, and the electric signal is sent to the control device 5. Output. Here, the light receiving surface 211 is a surface of the image sensor 21 and is a surface on which light is imaged. In the present embodiment, the position advanced from the light receiving surface 211 by the focal length in the direction of the optical axis OA2 is defined as an “imaging reference point O2 of the fixed camera 2”.

Such a fixed camera 2 is fixed at a predetermined location in the work area 90 so as to capture an image in the vertical direction. In the present embodiment, the fixed camera 2 is mounted such that its optical axis OA2 (the optical axis of the lens 22) is substantially parallel to the vertical direction.
In this embodiment, the image coordinate system of the fixed camera 2 (the coordinate system of the image output from the fixed camera 2) is a two-dimensional determined by the xa axis and the ya axis that are parallel to the in-plane direction of the image. The Cartesian coordinate system is set. Further, the translation component with respect to the xa axis is referred to as “component xa”, the translation component with respect to the ya axis is referred to as “component ya”, and the rotation component around the normal line of the xa-ya plane is referred to as “component ua”. The unit of the length (size) of the component xa and the component ya is “pixel”, and the unit of the angle (size) of the component ua is “°”.

  The image coordinate system of the fixed camera 2 takes into account the optical characteristics (focal length, distortion, etc.) of the lens 22 and the number of pixels and the size of the image sensor 21 with respect to the three-dimensional orthogonal coordinates reflected in the camera field of view of the fixed camera 2. This is a two-dimensional orthogonal coordinate system obtained by nonlinear transformation.

<Mobile camera>
The mobile camera 3 shown in FIGS. 1 and 2 has a function of capturing an object or the like.
As shown in FIG. 2, the mobile camera 3 includes an imaging element 31 configured with a CCD (Charge Coupled Device) image sensor having a plurality of pixels, and a lens 32 (optical system). The mobile camera 3 forms an image of light from an object or the like on a light receiving surface 311 (sensor surface) of the image sensor 31 by a lens 32, converts the light into an electrical signal, and the electrical signal is sent to the control device 5. Output. Here, the light receiving surface 311 is the surface of the image sensor 31 and is a surface on which light is imaged. In the present embodiment, the position advanced from the light receiving surface 311 in the direction of the optical axis OA3 by the focal length is referred to as “imaging reference point O3 of the mobile camera 3”.

  Such a mobile camera 3 is attached to the sixth arm 16 so that the distal end side of the robot arm 10 can be imaged more than the sixth arm 16. In the present embodiment, the mobile camera 3 is designed so that the optical axis OA3 (the optical axis of the lens 32) is substantially parallel to the center axis A6 of the sixth arm 16 by design. It is attached. Since the mobile camera 3 is attached to the sixth arm 16, the posture of the mobile camera 3 can be changed together with the sixth arm 16 by driving the robot arm 10.

  In this embodiment, the image coordinate system of the mobile camera 3 (the coordinate system of the image output by the mobile camera 3) is a two-dimensional determined by an xb axis and a yb axis that are parallel to the in-plane direction of the image. The Cartesian coordinate system is set. Further, the translation component with respect to the xb axis is referred to as “component xb”, the translation component with respect to the yb axis is referred to as “component yb”, and the rotation component around the normal line of the xb-yb plane is referred to as “component ub”. The unit of length (size) of the component xb and the component yb is “pixel”, and the unit of angle (size) of the component ub is “°”.

  Note that the image coordinate system of the mobile camera 3 takes into account the optical characteristics (focal length, distortion, etc.) of the lens 32 and the number of pixels and the size of the image sensor 31 with respect to the three-dimensional orthogonal coordinates reflected in the camera field of view of the mobile camera 3. This is a two-dimensional orthogonal coordinate system obtained by nonlinear transformation.

<Control device>
A control device 5 shown in FIG. 1 controls each part of the robot 1, the fixed camera 2, and the mobile camera 3. The control device 5 can be configured by, for example, a personal computer (PC) with a built-in CPU (Central Processing Unit), ROM (read only memory), and RAM (Random Access Memory).
As illustrated in FIG. 3, the control device 5 includes a drive control unit 51, an information acquisition unit 52, a processing unit 53, and a storage unit 54.

  The drive control unit 51 controls the driving of each driving unit 130 responsible for driving the first arm 11 to the sixth arm 16 of the robot 1 and drives the first arm 11 to the sixth arm 16 independently. It can be stopped. For example, in order to move the hand 102 to the target position, the drive control unit 51 derives a target value of a motor included in each driving unit 130 provided in the first arm 11 to the sixth arm 16. Further, the drive control unit 51 performs feedback control of the robot 1 based on the rotation angle (detection result) output from the angle sensor included in each drive unit 130. Further, the drive control unit 51 controls the imaging of the fixed camera 2 and the mobile camera 3.

  The information acquisition unit 52 acquires detection results output from the robot 1, the fixed camera 2, and the mobile camera 3, respectively. As the detection result, for example, the rotation angle of the rotation axis of the motor or the speed reducer included in each driving unit 130 of the robot 1, images captured by the fixed camera 2 and the mobile camera 3, and the axis coordinate O6 in the robot coordinate system. Coordinates (components xr, yr, zr, ur, vr, wr: position and orientation) are included.

  The processing unit 53 performs various calculations and various determinations based on the detection result acquired by the information acquisition unit 52. For example, the processing unit 53 calculates the coordinates (components xa, ya, ua: position and orientation) of the imaging target in the image coordinate system of the fixed camera 2 based on the image captured by the fixed camera 2, or the mobile camera 3 The coordinates (components xb, yb, ub: position and orientation) of the imaging target in the image coordinate system of the mobile camera 3 are calculated on the basis of the image captured in (1). Further, for example, the processing unit 53 obtains correction parameters for converting the coordinates of the object in the image coordinate system of the fixed camera 2 into coordinates in the robot coordinate system, or coordinates of the object in the image coordinate system of the mobile camera 3. The correction parameter for converting to the coordinate in the robot coordinate system is obtained.

The storage unit 54 stores programs, data, and the like for the control device 5 to perform various processes. The storage unit 54 stores various detection results and the like.
As shown in FIGS. 1 and 3, a display device 41 and an operation device 42 are connected to the control device 5.

The display device 41 includes a monitor 411 configured with a display panel such as a liquid crystal display panel. An operator can check an image captured by the fixed camera 2 and the mobile camera 3, work performed by the robot 1, and the like via the monitor 411.
The operation device 42 is an input device configured with a keyboard, and outputs an operation signal corresponding to an operation by the operator to the control device 5. Therefore, the operator can instruct the control device 5 to perform various processes by operating the operation device 42.
The basic configuration of the robot system 100 has been briefly described above.

In the robot system 100 having such a configuration, for example, the following work can be performed.
First, under the control of the control device 5, the robot arm 10 is driven and the hand 102 holds the object. Thereafter, the robot arm 10 is driven to move the hand 102 onto the fixed camera 2. Next, the object is imaged by the fixed camera 2, and the control device 5 determines whether or not the object is accurately held by the hand 102 based on the image captured by the fixed camera 2. If it is grasped accurately, the hand 102 is moved onto the work table 61 by driving the robot arm 10. Then, based on the image picked up by the mobile camera 3, the object gripped by the hand 102 is incorporated into the object previously arranged on the work table 61.

In such an operation, the robot 1 performs an operation on the object based on the images of the object captured by the fixed camera 2 and the mobile camera 3 respectively.
In such an operation, in order for the robot 1 to accurately perform an operation on an object based on an image captured by the fixed camera 2, coordinates on the image of the fixed camera 2 (position and orientation in the image coordinate system). Needs to be corrected, that is, calibration of the fixed camera 2 (calibration). Similarly, in order for the robot 1 to accurately perform an operation on an object or the like based on an image captured by the mobile camera 3, coordinates on the image of the mobile camera 3 (position and posture in the image coordinate system). Needs to be corrected, that is, calibration of the mobile camera 3 (calibration).

  Hereinafter, a calibration method for the fixed camera 2 and a calibration method for the mobile camera 3 using the robot system 100 (hereinafter, these are collectively referred to as an “imaging unit calibration method (calibration method)”) will be described.

≪Imager calibration method (calibration method) ≫
FIG. 4 is a flowchart showing a method of calibrating the imaging unit using the robot system shown in FIG. FIG. 5 is a plan view of a calibration member used for calibration of the imaging unit shown in FIG. FIG. 6 is a flowchart for explaining calibration of the mobile camera shown in FIG. FIG. 7 is a schematic diagram of a robot for explaining calibration of the mobile camera shown in FIG. FIG. 8 is a flowchart for explaining the processing for obtaining the offset component shown in FIG. FIG. 9 is a diagram for explaining processing for obtaining the offset components Δu, Δv, Δw shown in FIG. FIG. 10 is a flowchart for explaining processing for obtaining the offset components Δx and Δy shown in FIG. 11, FIG. 12, FIG. 13, FIG. 14 and FIG. 15 are diagrams for explaining the processing for obtaining the offset components Δx and Δy shown in FIG. 8, respectively. FIG. 16 is a coordinate diagram for explaining processing for obtaining the offset components Δx and Δy shown in FIG. FIG. 17 is a diagram for explaining the processing for obtaining the offset component Δz shown in FIG.

As shown in FIG. 4, in the calibration method of the imaging unit of the present embodiment, the calibration of the mobile camera 3 (step S2) is performed after the calibration of the fixed camera 2 (step S1).
Calibration of the imaging unit is started when the operator instructs the control device 5 to start calibration of the imaging unit using the operation device 42. Then, as long as the operator instructs the control device 5 to start the calibration of the imaging unit, the calibration of the imaging unit is automatically performed by the robot system 100 thereafter. The imaging unit is calibrated each time the work content of the robot 1 is changed, for example.

Here, in the present embodiment, the imaging unit is calibrated using the calibration member 70 (calibration board) shown in FIG.
The calibration member 70 is a rectangular flat plate member, and a plurality of markers 75 are attached to the surface 701 of the calibration member 70. The plurality of markers 75 have the same circular shape (shape) and are substantially the same size. Further, the plurality of markers 75 are arranged such that the pitches (intervals) between the adjacent markers 75 are almost constant. The pitch between the markers 75 is measured in advance and is known.

  Among the plurality of markers 75, the marker 75 located on the upper side in FIG. 5, the marker 75 located at the center in FIG. 5 (the center of the surface 701), and the marker 75 located on the right in FIG. Each is further provided with a circle surrounding the marker 75. Of the concentric markers composed of these three markers 75 and a circle surrounding them, the marker located on the upper side in FIG. 5 is referred to as “first marker 71 (first reference point)” in FIG. The marker located in the center of the second is referred to as “second marker 72 (second reference point)”, and the marker located on the right side in FIG. 5 is referred to as “third marker 73 (third reference point)”. The first marker 71, the second marker 72, and the third marker 73 are used as reference markers in processing for specifying a reference plane for calibration of the imaging unit, which will be described later. Therefore, the first marker 71, the second marker 72, and the third marker 73 are at different positions, and the first marker 71, the second marker 72, and the third marker 73 are not on the same straight line.

  Note that the shapes of the plurality of markers 75, the first marker 71, the second marker 72, and the third marker 73 are not limited to the illustrated shapes, and may be any shape. Moreover, the marker 75, the 1st marker 71, the 2nd marker 72, and the 3rd marker 73 should just be a form which can be visually recognized, respectively, and may be what kind of color and the form which has an unevenness | corrugation. The plurality of markers 75, the first marker 71, the second marker 72, and the third marker 73 may have different forms. For example, the plurality of markers 75, the first marker 71, the second marker 72, and the third marker 73 may have different colors and shapes. However, since the 1st marker 71, the 2nd marker 72, and the 3rd marker 73 are used as a reference marker, it is preferable that it is a form which has discriminating power with other markers 75.

  The calibration member 70 having such a configuration is previously held by the robot 1 with the hand 102 before the operator gives an instruction to start calibration of the imaging unit. In the present embodiment, the calibration member 70 is gripped by the hand 102 so that the second marker 72 is positioned on the central axis A6 of the sixth arm 16. In this embodiment, the coordinates of the second marker 72 relative to the coordinates of the axis coordinate O6 in the robot coordinate system are specified, and the coordinates of the second marker 72 in the robot coordinate system can be obtained.

  In the present embodiment, the second marker 72 is located on the central axis A6 of the sixth arm 16, but if the coordinates of the second marker 72 in the robot coordinate system are set to be obtained, The second marker 72 may not be located on the central axis A6 of the sixth arm 16.

<Calibration of fixed camera (step S1)>
As shown in FIG. 4, when an instruction to start calibration of the imaging unit is given by the operator, the control device 5 first starts calibration of the fixed camera 2 (step S <b> 1).
In the calibration of the fixed camera 2 (step S1), after the process of specifying the reference plane is performed, the process of obtaining the relationship between the image coordinate system of the fixed camera 2 and the robot coordinate system is performed.

[Process to identify reference plane]
Hereinafter, processing for specifying the reference plane will be described.
When the process of specifying the reference plane is started, first, the control device 5 drives the robot arm 10 so that the calibration member 70 faces the fixed camera 2.
Next, the control device 5 drives the robot arm 10 to move the calibration member 70 so that the second marker 72 attached to the calibration member 70 is positioned at the center of the image of the fixed camera 2.

Next, the control device 5 causes the fixed camera 2 to image the second marker 72. At this time, the control device 5 moves the calibration member 70 by driving the robot arm 10 so that the focus of the fixed camera 2 is focused (focused) on the second marker 72 (focusing process). I do. This focusing process may be omitted.
Next, the control device 5 stores the image of the second marker 72 captured by the fixed camera 2 in the storage unit 54 as a “first image”, and the axis coordinate O6 in the robot coordinate system when the first image is captured. Are stored in the storage unit 54. Here, in the process of specifying the reference plane in the fixed camera 2, the second marker 72 when the first image is captured is referred to as a “first reference marker”.
Next, the control device 5 drives the robot arm 10 to move the calibration member 70 to the robot coordinate system so that the second marker 72 is positioned at a position different from the position moved on the image of the fixed camera 2. Is translated along the xr, yr, and zr axes.

Next, the control device 5 causes the fixed camera 2 to image the second marker 72.
Next, the shape and size of the second marker 72 in the image captured by the fixed camera 2 are compared with the shape and size of the second marker 72 in the first image stored in the storage unit 54. Then, it is determined whether or not the difference between the shape and size of the second marker 72 and the shape and size of the second marker 72 in the first image is within a predetermined threshold.
When it is determined that it is not within the predetermined threshold, the calibration member 70 is moved by driving the robot arm 10 so as to be within the predetermined threshold.
Next, when the control device 5 determines that the value is within the predetermined threshold, it stores the image of the second marker 72 captured by the fixed camera 2 in the storage unit 54 as a “second image (nth image)”, and The coordinates of the axis coordinate O6 in the robot coordinate system when the second image (nth image) is captured are stored in the storage unit 54. Here, in the process of specifying the reference plane in the fixed camera 2, the second marker 72 when the second image is captured is referred to as a “second reference marker”. When the second image is captured, the second marker 72 attached to the calibration member 70 held by the hand 102 is at a position different from the position at which the first image is captured.

Next, it is determined whether or not the number n of captured images is a predetermined number (where n is an integer and satisfies the relationship of 3 ≦ n).
Here, in the present embodiment, acquiring images until the number of images reaches three, that is, capturing three images (first image, second image, and third image) with the fixed camera 2 in advance. It is set. Here, in the process of specifying the reference plane in the fixed camera 2, the second marker 72 when the third image is captured is referred to as a “third reference marker”. When the third image is captured, the second marker 72 attached to the calibration member 70 held by the hand 102 is positioned when the first image is captured and when the second image is captured. They are not in the same straight line. Further, in the process of specifying the reference plane in the fixed camera 2, it can be understood that the second marker 72 also serves as “the first reference marker, the second reference marker, and the third reference marker”.

  Next, if it is determined that the number n of images is a predetermined number, the processing unit 53 is based on the coordinates of the axis coordinate O6 in the n (three in the present embodiment) robot coordinate system stored in the storage unit 54. Then, the origin of the reference surface 81 parallel to the image pickup device 21 (the plane passing through the second marker 72 arranged in three different locations) and the x-axis, y-axis, and z-axis directions are obtained. Then, the control device 5 defines the position and orientation of the reference surface 81 in the robot coordinate system, that is, the components xr, yr, zr, ur, vr, and wr of the reference surface 81.

[Process to obtain the relationship between the image coordinate system of the fixed camera and the robot coordinate system]
Next, processing for obtaining the relationship between the image coordinate system of the fixed camera and the robot coordinate system is performed.
First, the control device 5 drives the robot arm 10 to perform calibration so that the axial coordinates O6 are positioned at any nine reference points (virtual target points) arranged in a grid pattern in the reference plane 81, respectively. The working member 70 is moved. That is, the second marker 72 is moved to nine locations arranged in a lattice pattern. At this time, the control device 5 causes the fixed camera 2 to image the second marker 72 every time the calibration member 70 is moved.
Here, it is assumed that the nine reference points are all within the range of the image of the fixed camera 2 (in the imaging region), and the intervals between adjacent reference points are all equal.

Next, the control device 5 determines the coordinates (components xa, ya, ua) of the second marker 72 in the image coordinate system of the fixed camera 2 based on the nine images and the coordinates (component xr) of the reference plane 81 in the robot coordinate system. , Yr, ur), a correction parameter (coordinate conversion matrix) for converting the image coordinates of the fixed camera 2 into the coordinates of the reference plane 81 in the robot coordinate system is obtained.
Using the correction parameters obtained in this way, the position and orientation (specifically, components xa, ya, ua) of an object or the like imaged by the fixed camera 2 are converted into values in the robot coordinate system (specifically, Component xr, yr, ur). This correction parameter is a value that takes into account internal parameters of the fixed camera 2 such as distortion of the lens 22.

<Calibration of mobile camera (step S2)>
Next, the control device 5 starts calibration (step S2) of the mobile camera 3 shown in FIG.
As shown in FIG. 6, in the calibration (step S2) of the mobile camera 3, a process for specifying a reference plane (step S21), a process for obtaining an offset component (step S22), a process for specifying a work plane (step S23), Processing for teaching the position and orientation of the marker to the robot 1 (step S24) and processing for obtaining the relationship between the image coordinate system of the mobile camera 3 and the robot coordinate system (step S25) are performed in this order.

[Process for Specifying Reference Surface (Step S21)]
First, as shown in FIG. 7, the calibration member 70 gripped by the hand 102 is placed on the work surface 611 of the work table 61 in advance before performing the process of specifying the reference surface (step S <b> 21). Keep it. Then, the process (step S21) which specifies a reference plane is performed.
First, the control device 5 drives the robot arm 10 so that the mobile camera 3 faces the calibration member 70. Next, the control device 5 drives the robot arm 10 to move the mobile camera 3 so that the first marker 71 attached to the calibration member 70 is positioned at the center of the image of the mobile camera 3.

Next, the control device 5 causes the mobile camera 3 to image the first marker 71. At this time, the control device 5 performs a process of moving the mobile camera 3 (focusing process) by driving the robot arm 10 so that the mobile camera 3 is focused on (focused on) the first marker 71. . This focusing process may be omitted.
Next, the control device 5 stores the image of the first marker 71 captured by the mobile camera 3 in the storage unit 54 as a “first image”, and the axis coordinate O6 in the robot coordinate system when the first image is captured. Are stored in the storage unit 54. Here, in the process of specifying the reference plane in the mobile camera 3, the first marker 71 is set as a “first reference marker”.
Next, the control device 5 drives the robot arm 10 to translate the mobile camera 3 so that the second marker 72 is positioned at the center of the image of the mobile camera 3.
Next, the control device 5 causes the mobile camera 3 to image the second marker 72 (nth marker).

Next, the shape and size of the second marker 72 in the image captured by the mobile camera 3 are compared with the shape and size of the first marker 71 in the first image stored in the storage unit 54. Then, it is determined whether or not the difference between the shape and size of the second marker 72 and the shape and size of the first marker 71 is within a predetermined threshold.
If it is determined that it is not within the predetermined threshold, the mobile camera 3 is moved by driving the robot arm 10 so that it is within the predetermined threshold.
Next, when the control device 5 determines that the value is within the predetermined threshold, the image of the second marker 72 (n-th marker) captured by the mobile camera 3 is stored as a “second image (n-th image)” in the storage unit 54. And the coordinates of the axis coordinate O6 in the robot coordinate system when the second image (n-th image) is captured are stored in the storage unit 54. Here, in the process of identifying the reference plane in the mobile camera 3 (step S21), the second marker 72 is set as the “second reference marker”.

Next, it is determined whether or not the number n of captured images is a predetermined number (where n is an integer and satisfies the relationship of 3 ≦ n).
Here, in this embodiment, it is preset that the mobile camera 3 captures three images (first image, second image, and third image). Therefore, in this embodiment, after the second image is captured by the mobile camera 3, the image of the third marker 73 captured by the mobile camera 3 is stored in the storage unit 54 as a "third image" once more, And the coordinate of the axis coordinate O6 in the robot coordinate system at the time of imaging a 3rd image is memorize | stored in the memory | storage part 54. FIG. Here, in the process of specifying the reference plane in the mobile camera 3, the third marker 73 is referred to as a “third reference marker”.

  Next, if it is determined that the number n of images is a predetermined number, the processing unit 53 is based on the coordinates of the axis coordinate O6 in the n (three in the present embodiment) robot coordinate system stored in the storage unit 54. 7, the origin of the reference plane 82 parallel to the surface 701 (the plane passing through the first marker 71, the second marker 72, and the third marker 73) and the respective directions of the x-axis, y-axis, and z-axis are obtained. . Then, the control device 5 defines the position and orientation of the reference plane 82 in the robot coordinate system, that is, the components xr, yr, zr, ur, vr, and wr of the reference plane 82.

[Process for obtaining offset component (step S22)]
Next, the process for obtaining the offset component (step S22) will be described with reference to the flowchart shown in FIG.

  Here, as described above, the mobile camera 3 is offset and attached to the sixth arm 16 so that the optical axis OA3 is substantially parallel to the central axis A6 of the sixth arm 16 by design. ing. However, in practice, a deviation occurs from an offset component in the design (position and posture of the mobile camera 3 with respect to the sixth arm 16). This deviation occurs due to, for example, an assembly error of the mobile camera 3 or an assembly error of the image sensor 31 with respect to the housing of the mobile camera 3.

Therefore, an actual offset component (position and posture of the mobile camera 3 with respect to the sixth arm 16) is obtained in the process of obtaining the offset component (step S22).
In the processing for obtaining the following offset components (step S22), the offset components (Δx, Δx,...) Of the position of the imaging reference point O3 of the mobile camera 3 and the direction (posture) of the optical axis OA3 with respect to the axis coordinate O6 of the rotating shaft member 161. Δy, Δu, Δv, Δw) are obtained.

  In the present embodiment, the position of the imaging reference point O3 with respect to the axis coordinate O6 and the offset component in the direction of the optical axis OA3 are obtained, but the reference points for obtaining the offset component are the axis coordinate O6 and the imaging reference point O3. It is not limited and is arbitrary. For example, the reference part for obtaining the offset component may be a predetermined part of a tool such as TCP of the hand 102 attached to the robot arm 10.

As shown in FIG. 8, when the process for obtaining the offset (step S22) is started, the control device 5 first drives the robot arm 10 to cause the mobile camera 3 to detect the calibration member 70 (step S221).
Next, the control device 5 drives the robot arm 10 so that the light receiving surface 311 of the mobile camera 3 faces the surface 701 of the calibration member 70 (step S222).

Next, the control device 5 verifies the parallelism of the surface 701 of the calibration member 70 with respect to the light receiving surface 311 of the mobile camera 3 (step S223). Then, the control device 5 determines whether or not the parallelism is within a predetermined threshold (step S224).
As shown in FIG. 9, the parallelism is verified using a difference in pitch P between adjacent markers 75 attached to the surface 701 in the image. For example, as shown by the solid line in FIG. 9, if the difference between the pitches P1, P2, P3, and P4 between the adjacent markers 75 is substantially the same and the difference is within a predetermined threshold, the process goes to step S225. Transition. On the other hand, as shown by a two-dot chain line in FIG. 9, when the difference between the pitches P1 ′, P2 ′, P3 ′, and P4 ′ between the adjacent markers 75 is different and the difference exceeds a predetermined threshold value, Step S221 to Step S224 are repeated until the value is within the threshold value. Here, being within the predetermined threshold means that the above-described reference plane 82 and the optical axis OA3 are perpendicular to each other within the threshold.

  Next, when the control device 5 determines that it is within the predetermined threshold value, the components ur, vr, wr of the axis coordinate O6 in the robot coordinate system when it is determined to be within the predetermined threshold value, and the reference plane described above are used. Offset components Δu, Δv, Δw are obtained from the differences from the components ur, vr, wr of the reference plane 82 in the robot coordinate system obtained in the specifying process (step S21) (step S225). The offset components Δu, Δv, Δw correspond to the offset components Δu, Δv, Δw of the optical axis OA3 with respect to the axis coordinate O6.

  Next, as shown in FIG. 8, the control device 5 obtains offset components Δx and Δy of the imaging reference point O3 with respect to the axis coordinate O6 (step S226). Hereinafter, a method for obtaining the offset components Δx and Δy will be described with reference to FIGS. FIG. 10 is a flowchart of processing for obtaining the offset components Δx and Δy. FIGS. 11 to 16 schematically illustrate the mobile camera 3 and the sixth arm 16 when the robot 1 is viewed from above in the vertical direction, for example. It shows.

  Specifically, first, the control device 5 moves the marker to the center of the image (step S2261). As illustrated in FIG. 11, the control device 5 drives the robot arm 10 so that the second marker 72 is positioned at the center O30 (center of gravity) of the image 30 of the mobile camera 3, for example. The posture of the mobile camera 3 and the sixth arm 16 shown in FIG. 11 is referred to as a “first posture”, and this state is referred to as a “first state”. Here, the center O30 of the image 30 and the imaging reference point O3 coincide. Further, the position of the hand 102 attached to the sixth arm 16 shown in FIG.

  Next, the control device 5 changes the posture of the robot 1 (step S2262). The control device 5 rotates about 5 ° about the TCP as a rotation axis. Specifically, as shown in FIG. 12, the control device 5 drives the robot arm 10 in the first posture to rotate the sixth arm 16 around the central axis A6 at a predetermined angle. The predetermined angle at this time is set to a predetermined angle (for example, about 1 to 10 °) within a range where the second marker 72 does not appear outside the image 30 (within a range within the imaging region of the mobile camera 3).

  Next, the control device 5 moves the marker to the center of the image (step S2263). Specifically, as shown in FIG. 13, the control device 5 drives the robot arm 10 to move the mobile camera 3 and the sixth arm 16 in the robot coordinate system so that the second marker 72 coincides with the center O30. Translation is performed in a plane parallel to a plane including the xr axis and the yr axis (the xy plane of the reference plane 82). The posture of the mobile camera 3 and the sixth arm 16 shown in FIG. 13 is referred to as a “second posture”, and this state is referred to as a “second state”.

  Next, the control device 5 calculates the tool offset and sets it to the tool 1 (step S2264). The movements of the mobile camera 3 and the sixth arm 16 that change from the first state to the fourth state are as follows when the first state shown in FIG. 11 and the second state shown in FIG. It can be seen that this is equivalent to the rotation of the axis coordinate O6 (sixth arm 16) with the line segment passing through O3) as the rotation center axis. Accordingly, as shown in FIG. 11 to FIG. 13, the movement from the first state to the second state is performed by using the line segment passing through the center O30 (imaging reference point O3) as the rotation center axis and the axis coordinate O6 as the rotation angle θ10. Equivalent to moving with Therefore, the imaging reference in the robot coordinate system is based on the rotation angle θ10, the coordinate of the axis coordinate O6 in the robot coordinate system in the first state, and the coordinate of the axis coordinate O6 in the robot coordinate system in the second state. The coordinates of the point O3 are obtained. Then, based on the obtained coordinates of the imaging reference point O3 in the robot coordinate system and the coordinates of the axis coordinate O6 in either one of the first state and the third state, the provisional reference point O3 relative to the axis coordinate O6 is provisional. Offset components Δx ′ and Δy ′ (temporary tool set) are obtained.

Next, the control device 5 moves the hand 102 attached to the sixth arm 16 of the robot arm 10 (step S2265). Consider a case where an interference (obstacle) X1 such as another robot or device is disposed in the vicinity of the robot 1. In this case, when a certain robot 1 rotates the hand 102 attached to the sixth arm 16 of the robot arm 10 of the robot 1 at a predetermined angle around the central axis A6, the hand 102 becomes the interference object X1. It is assumed that it will touch.
Therefore, in the present embodiment, in the second state, when the hand 102 rotates around the central axis A6 at a predetermined angle, the hand 102 comes into contact with the interference X1, the control device 5 The hand 102 attached to the sixth arm 16 is moved from the first position to the second position where the hand 102 does not contact the interference X1 even if the hand 102 is rotated around the central axis A6 at a predetermined angle. Specifically, in this example, the control device 5 moves the hand 102 upward from the first position by a predetermined distance while maintaining the current posture of the mobile camera 3 in this example. Then, even if the hand 102 is rotated around the central axis A6 by a predetermined angle, the hand 102 is translated (relieved) to the second position where it does not contact the obstacle X1. The control device 5 may be configured to translate the hand 102 by a predetermined distance in the other direction instead of moving the hand 102 upward from the first position by a predetermined distance. The structure which moves to another position may be sufficient. When moving to another predetermined position, the posture of the mobile camera 3 must be the same in each of the first position and the second position. However, the posture of the mobile camera 3 during movement from the first position to the second position may change.

  Next, the control device 5 changes the posture of the robot (step S2266). Specifically, the control device 5 rotates 180 degrees with respect to the first posture with the tool tip as the rotation center. FIG. 14 shows the mobile camera 3 and the sixth arm 16 in a state in which the tool tip is rotated 180 ° around the tool tip. In the example illustrated in FIG. 14, the second marker 72 is included in the image 30 of the mobile camera 3 even after the control device 5 moves the hand 102 from the first position to the second position. In this state, the second marker 72 may not be included in the image 30 of the mobile camera 3.

Next, the control device 5 moves the position of the hand 102 attached to the sixth arm 16 to the first position (step S2267). In this case, the posture of the robot differs from the first posture by 180 °.
Next, the control device 5 moves the marker to the center of the image (step S2268). Specifically, the control device 5 drives the robot arm 10 based on the temporary offset components Δx ′ and Δy ′ so that the second marker 72 does not come out of the image 30, and the imaging reference point O3 (center O30). The axis coordinate O6 is rotated at a predetermined angle with the line segment passing through as the rotation center axis. As shown in FIG. 15, the control device 5 drives the robot arm 10 to move the mobile camera 3 and the sixth arm 16 to a plane including the xr axis and the yr axis in the robot coordinate system (the xy plane of the reference plane 82). The second marker 72 is positioned at the center O30 of the image 30 by translation in a parallel plane. The posture of the mobile camera 3 and the sixth arm 16 shown in FIG. 15 is referred to as a “third posture”, and this state is referred to as a “third state”.

Next, the control device 5 calculates the tool offset and sets it to the tool 1 (step S2269). The movement of the mobile camera 3 and the sixth arm 16 from the first state shown in FIG. 11 to the third state shown in FIG. 15 will be described when the first state shown in FIG. 11 and the third state shown in FIG. It can be seen that this is equivalent to rotating the axis coordinate O6 with the line segment passing through the center O30 (imaging reference point O3) as the rotation center axis. Therefore, as shown in FIG. 16, the movement from the first state of FIG. 11 to the third state of FIG. 15 is performed by rotating the axis coordinate O6 about the line segment passing through the center O30 (imaging reference point O3) as the rotation center axis. Equivalent to moving at a moving angle θ1. Therefore, the imaging reference in the robot coordinate system is based on the rotation angle θ1, the coordinate of the axis coordinate O6 in the robot coordinate system in the first state, and the coordinate of the axis coordinate O6 in the robot coordinate system in the third state. The coordinates of the point O3 are obtained. Then, the offset of the imaging reference point O3 with respect to the axis coordinate O6 from the coordinates of the imaging reference point O3 in the obtained robot coordinate system and the coordinates of the axis coordinate O6 in one of the robot coordinate systems in the first state and the third state. Components Δx and Δy (tool set) are obtained.
Next, the control device 5 moves the hand 102 attached to the sixth arm 16 of the robot arm 10 to a place where there is no interference (step S2270).
Next, the control device 5 returns the robot to its original posture (step S2271). Specifically, the control device 5 rotates −180 ° about the tool tip as the rotation center.
Next, the control device 5 moves the position of the hand 102 attached to the sixth arm 16 to the first position (step S2272).

According to such processing, the offset components Δx and Δy of the imaging reference point O3 with respect to the axis coordinate O6 can be obtained easily and with high accuracy.
Further, as described above, in the present embodiment, the temporary offset components Δx ′ and Δy ′ are calculated by performing the process of shifting from the first state to the second state. That is, the temporary offset is performed by performing the process of rotating the sixth arm 16 about the central axis A6 at a minute angle within a range in which the second marker 72 is within the image 30 (in the imaging region) of the mobile camera 3. Components Δx ′ and Δy ′ are calculated. By using the information of the temporary offset components Δx ′ and Δy ′ as described above, the second marker 72 can be reliably copied in the image 30 by performing the process of step S2268.

  Next, as shown in FIG. 8, the control device 5 obtains an offset component Δz of the imaging reference point O3 with respect to the axis coordinate O6 (step S227). Hereinafter, a method for obtaining the offset component Δz will be described with reference to FIG. FIG. 17 illustrates the process of the mobile camera 3 and the sixth arm 16 when obtaining the offset component Δz. For convenience of explanation, the mobile camera 3 indicated by the solid line in FIG. The position of the camera 3 ”is shown, and the mobile camera 3 ′ indicated by the dotted line in FIG. 17 indicates the position of the“ actual mobile camera 3 ”.

As shown in FIG. 17, first, the control device 5 drives the robot arm 10 so that the second marker 72 appears in the center of the image of the mobile camera 3 ′, for example, and enters the state A shown in FIG. 16. Next, the control device 5 captures an image of the second marker 72 with the mobile camera 3 ′, and obtains a distance H between the light receiving surface 311 of the mobile camera 3 and the second marker 72 shown in FIG.
Here, in the present embodiment, the focal length of the mobile camera 3 is obtained in advance and is known. Therefore, the distance H is calculated from, for example, the focal length of the mobile camera 3, the length “pixel” of the pitch between the markers 75 in the image of the mobile camera 3, and the pitch “mm” between the actual markers 75. be able to.

The focal length of the mobile camera 3 is, for example, on the image before and after the operation when the mobile camera 3 is moved by a minute amount in the optical axis OA3 direction (zr direction) while copying the marker 75 of the calibration member 70. It is also possible to obtain from the pitch length “pixel” between the markers 75 and the actual pitch “mm” between the markers 75.
Next, as shown in state B in FIG. 16, the control device 5 drives the robot arm 10 to tilt the mobile camera 3 ′ by an angle θ2 based on the designed offset component Δz.

  Next, as shown in state C in FIG. 16, the control device 5 drives the robot arm 10 to keep the posture of the mobile camera 3 ′ in the state B while the second marker 72 is in the mobile camera 3 ′. The mobile camera 3 ′ is translated in a plane parallel to a plane including the xr axis and the yr axis (xy plane of the reference plane 82) in the robot coordinate system so as to appear in the center of the image. The control device 5 then moves the movement distance X ′ of the axis coordinate O6 in the robot coordinate system at this time (specifically, in the plane parallel to the xy plane of the reference plane 82 based on the designed offset component Δz). (Movement distance of imaging reference point O3) at (1).

Next, the control device 5 obtains a correction amount ΔH for obtaining the actual offset component Δz of the mobile camera 3 ′ from the following equation (1).
ΔH = (X ′ / tan θ2) −H (1)
Next, the control device 5 obtains the actual offset component Δz based on the correction amount ΔH and the designed offset component Δz.
In this way, the offset component Δz can be obtained. According to such processing, the offset component Δz can be easily calculated.

Next, as shown in FIG. 8, the control device 5 updates the data from the designed offset component to the actual offset components Δx, Δy, Δz, Δu, Δv, and Δw that are obtained (step S228).
Thus, the process for obtaining the offset shown in FIG. 6 (step S22) is completed.

[Process for Specifying Work Surface (Step S23)]
Next, as shown in FIG. 6, a process of specifying a work surface (step S23) is performed. The process for identifying the work surface (step S23) is a process for obtaining the position and orientation of the work surface 611 in the robot coordinate system, that is, the components xr, yr, zr, ur, vr, and wr of the work surface 611.

  Here, the work surface 611 is parallel to the reference surface 82 and is at a position offset in the normal direction (zr direction) of the reference surface 82. For this reason, in the process of identifying the work surface (step S23), the offset amount in the normal direction (zr direction) of the work surface 611 with respect to the reference surface 82 is determined to thereby determine the components xr, yr, zr of the work surface 611. , Ur, vr, wr can be obtained.

  The offset amount in the normal direction (zr direction) of the work surface 611 with respect to the reference surface 82 is relative to the focal length of the mobile camera 3 obtained in advance and the pitch value (actual size) of the adjacent marker 75 of the calibration member 70. It can be obtained based on the number of pixels of the mobile camera 3 and the actual offset component described above.

  By obtaining the position and orientation of the work surface 611 in the robot coordinate system in this way, the robot 1 can work with high accuracy on the object placed on the work surface 611.

[Process of Teaching Robot to Position and Posture of Marker (Step S24)]
Next, as shown in FIG. 6, processing for teaching the position and posture of the marker to the robot 1 is performed (step S24).
Here, for example, the robot 1 is taught the robot coordinates of the second marker 72 in the xy plane of the reference plane 82 (or the work plane 611).

  Specifically, first, the control device 5 is based on the offset component in the direction of the optical axis OA3 and the position of the imaging reference point O3 with respect to the axis coordinate O6 calculated in the above-described processing for obtaining the offset component (step S22). The optical axis OA2 of the mobile camera 3 is obtained on the z axis of the reference plane 82. Thereafter, the control device 5 translates the mobile camera 3 in a plane parallel to the xy plane of the reference plane 82 by driving the robot arm 10 and aligns the second marker 72 with the image center of the mobile camera 3. Then, the control device 5 teaches the position of the imaging reference point O3 of the mobile camera 3 when the second marker 72 is aligned with the image center of the mobile camera 3 as the robot coordinates of the second marker 72.

  For example, the robot 1 may be taught the position and orientation of the marker 75 by bringing a dedicated tool (touch-up hand) whose offset at the axis coordinate O6 is known into contact with the second marker 72. However, when the mobile camera 3 captures an image of the second marker 72, the second marker 72 is taught to the robot 1 regardless of the material of the calibration member 70, for example. 72 is preferable because it can be taught with high accuracy.

[Process for obtaining relationship between image coordinate system of mobile camera and robot coordinate system (step S25)]
Next, as shown in FIG. 6, a process of obtaining the relationship between the image coordinate system of the mobile camera and the robot coordinate system (step S25).
When the process of obtaining the relationship between the image coordinate system of the mobile camera 3 and the robot coordinate system (step S25) ends, the coordinates (components xb, yb) of the second marker 72 in the image coordinate system of the mobile camera 3 based on the nine images. Ub) and the coordinates of the reference plane 82 in the robot coordinate system (components xr, yr, ur) obtained in step S21 described above, the image coordinates of the fixed camera 2 are converted to the coordinates of the reference plane 82 in the robot coordinate system. A correction parameter (coordinate transformation matrix) to be converted into can be obtained.

Using the correction parameters obtained in this way, the position and orientation (specifically, components xb, yb, ub) of the object and the like imaged by the mobile camera 3 are converted into values in the robot coordinate system (specifically, Component xr, yr, ur).
Thus, the calibration of the imaging unit shown in FIG. 4 is completed.
According to such a calibration method of the imaging unit, the postures of the reference planes 81 and 82 can be obtained based on the images captured by the fixed camera 2 and the mobile camera 3, respectively. Can be omitted. For this reason, it is possible to reduce an artificial error and a variation by an operator, and thus a highly accurate calibration can be performed.

  The robot, the control device, and the robot system of the present invention have been described based on the illustrated embodiment. However, the present invention is not limited to this, and the configuration of each unit is an arbitrary configuration having the same function. Can be substituted. Moreover, other arbitrary components may be added. Further, the present invention may be a combination of any two or more configurations (features) of the above embodiments.

  In the above embodiment, a case where a 6-axis vertical articulated robot is used has been described as an example. However, the robot in the present invention may be a robot other than the vertical articulated robot, for example, a horizontal articulated robot. . The horizontal articulated robot is, for example, a base, a first arm connected to the base and extending in the horizontal direction, and a second arm connected to the first arm and having a portion extending in the horizontal direction. It is the structure which has. Further, when the robot according to the present invention is a horizontal articulated robot, by performing the calibration as described above, for example, whether the robot is installed in parallel to the work surface or whether the optical axis of the fixed camera is It can be ascertained whether or not the fixed camera is installed so as to be perpendicular to the plane including the xr axis and the yr axis in the robot coordinate system.

  In the embodiment, the number of rotation axes of the robot arm included in the robot is six. However, the present invention is not limited to this, and the number of rotation axes of the robot arm is, for example, two. There may be three, four, five or more than seven. Moreover, in the said embodiment, the number of arms which a robot has is six, However, In this invention, it is not limited to this, For example, the number of arms which a robot has is 2, 3, 4, Five or seven or more may be used.

Moreover, in the said embodiment, although the number of the robot arms which a robot has is one, in this invention, it is not limited to this, For example, the number of the robot arms which a robot has may be two or more. That is, the robot may be a multi-arm robot such as a double-arm robot.
In the above-described embodiment, the fixed camera and the mobile camera as the imaging unit have an imaging element and a lens, respectively. However, the imaging unit according to the present invention includes a first marker, a second marker, and a third marker. Any configuration may be used as long as it can capture an image of the image.

In the above embodiment, the calibration of the fixed camera is performed using the calibration member. However, the calibration member may not be used in the calibration of the fixed camera. When the calibration member is not used, for example, one marker may be attached to the tip (axis coordinate) of the robot arm, and the one marker may be used as the reference marker. In this case, one marker is regarded as “a first reference marker, a second reference marker, and a third reference marker”.
In the above embodiment, a hand is applied as a tool attached to the robot arm 10, but a dispenser may be applied as a tool attached to the robot arm 10.

  As described above, the robot (in this example, the robot 1) in the present embodiment has the imaging unit (in this example, the first state) and the third state (in this example, the third state) in the imaging unit ( In this example, an offset with respect to an arm (in this example, the robot arm 10) of a predetermined part (in this example, the TCP of the hand 102) can be set based on an image captured by the mobile camera 3). As a result, the robot can reduce human error and variations due to the operator, rather than setting the tool offset while teaching the position of the reference point by the operator operating the arm.

  Further, the robot can rotate the tool around a rotation axis parallel to the optical axis of the imaging unit through a predetermined portion at a position where there is no obstacle. As a result, the robot can avoid the collision with the obstacle and rotate the tool around the rotation axis parallel to the optical axis of the imaging unit through the predetermined part. Even when there is an object, the offset can be calculated with high accuracy.

  Moreover, the control apparatus can set the offset with respect to the arm of a predetermined site | part to a robot based on the image imaged by the imaging part in the 1st state and the 3rd state. Thus, the control device can set the offset with higher accuracy than setting the offset of the tool in the robot while teaching the position of the reference point by operating the arm by the operator.

  Further, in the robot system, the control device can set an offset for the arm of the predetermined part in the robot based on the images captured by the imaging unit in the first state and the third state. Thereby, in the robot system, the control device can set the offset with higher accuracy than setting the offset of the tool in the robot while teaching the position of the reference point by operating the arm by the operator.

The embodiment of the present invention has been described in detail with reference to the drawings. However, the specific configuration is not limited to this embodiment, and changes, substitutions, deletions, and the like are possible without departing from the gist of the present invention. May be.
Further, a program for realizing the function of an arbitrary component in the above-described apparatus (for example, a control apparatus) is recorded on a computer-readable recording medium, and the program is read by a computer system and executed. It may be. Here, the “computer system” includes hardware such as an OS (Operating System) and peripheral devices. The “computer-readable recording medium” refers to a portable medium such as a flexible disk, a magneto-optical disk, a ROM, a CD (Compact Disk) -ROM, or a storage device such as a hard disk built in the computer system. . Furthermore, “computer-readable recording medium” means a volatile memory (RAM) inside a computer system that becomes a server or client when a program is transmitted via a network such as the Internet or a communication line such as a telephone line. In addition, those holding programs for a certain period of time are also included.

In addition, the above program may be transmitted from a computer system storing the program in a storage device or the like to another computer system via a transmission medium or by a transmission wave in the transmission medium. Here, the “transmission medium” for transmitting the program refers to a medium having a function of transmitting information, such as a network (communication network) such as the Internet or a communication line (communication line) such as a telephone line.
Further, the above program may be for realizing a part of the functions described above. Further, the program may be a so-called difference file (difference program) that can realize the above-described functions in combination with a program already recorded in the computer system.

DESCRIPTION OF SYMBOLS 1 ... Robot, 2 ... Fixed camera, 3 ... Mobile camera, 3 '... Mobile camera, 5 ... Control apparatus, 10 ... Robot arm, 11 ... 1st arm, 12 ... 2nd arm, 13 ... 3rd arm, 14 ... 4th arm, 15 ... 5th arm, 16 ... 6th arm, 21 ... Imaging device, 22 ... Lens, 30 ... Image, 31 ... Imaging device, 32 ... Lens, 41 ... Display device, 42 ... Operating device, 51 ... Drive control unit 52 ... Information acquisition unit 53 ... Processing unit 54 ... Storage unit 61 ... Working table 62 ... Supply table 70 ... Calibration member 71 ... First marker 72 ... Second marker 73 ... 3rd marker, 75 ... marker, 81 ... reference plane, 82 ... reference plane, 90 ... work area, 100 ... robot system, 101 ... base, 102 ... hand, 111 ... rotating shaft member, 120 ... motor driver, 121 ... Rotating shaft Numeral 130, drive unit 131, rotation shaft member, 141 rotation shaft member, 151 rotation shaft member, 161 rotation shaft member, 211 light receiving surface, 311 light receiving surface, 411 monitor, 611 ... work surface, 701 ... surface, A ... state, A6 ... central axis, B ... state, C ... state, H ... distance, O2 ... imaging reference point, O3 ... imaging reference point, O30 ... center, O6 ... axis coordinate, OA2 ... optical axis, OA3 ... optical axis, P ... pitch, P1, P2, P3, P4 ... pitch, P1 ', P2', P3 ', P4' ... pitch, TCP ... tool center point, X '... moving distance, ΔH: correction amount, θ1: rotation angle, θ10: rotation angle, θ2: angle

Claims (4)

With an arm,
The arm is
In a second state in which the predetermined part of the tool attached to the arm is moved from the first state in which the predetermined part can be imaged by the imaging unit, a rotation axis that passes through the predetermined part and is parallel to the optical axis of the imaging unit The tool is rotated to the center, and the predetermined part is moved to a third state that is the same position as the position in the first state while maintaining a change in posture due to the rotation,
It is controlled using an offset for the arm of the predetermined part set based on images captured by the imaging unit in the first state and the third state.
robot. The arm is
When there is an obstacle in the first state, the tool is moved from the first state to a second state without the obstacle;
The robot according to claim 1. Controlling the robot according to claim 1 or 2,
Control device. The robot according to claim 1 or 2,
A control device according to claim 3;
A robot system comprising:

Images