JP2002018754A - Robot apparatus and control method therefor - Google Patents

Abstract

PROBLEM TO BE SOLVED: To save labor for troublesome teaching work and calibration and allow easy work for objects, as predetermined. SOLUTION: In teaching work, a robot controller 5 stores a position Pd of a gripper 3 in the state of gripping the object 11 and further stores any position (Ps) after the object 11 is released. In clamping work, the robot controller 5 makes the gripper 3 follow the object 11 being carried, go to the object 11 in accordance with the positions Ps, Pd and grip the object 11.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a robot apparatus and a control method therefor, and more particularly to a robot apparatus for gripping a moving object and a control method therefor.

[0002]

2. Description of the Related Art When performing an operation (for example, grasping an object) on an object using a robot, an example in which information such as the position and orientation of the object is acquired by a sensor and the operation is performed based on the sensor information. Has been put to practical use. Such an apparatus configuration is roughly classified into the following two systems.

[0003] For example, as described in Japanese Patent Application Laid-Open No. 8-63214, an object to be conveyed on a conveyor is imaged by a fixed camera, and the position and orientation of the object are detected from the obtained image. In some cases, the information is transmitted to the robot. This robot performs operations such as grasping based on position information of an object.

Further, a sensor such as a camera is attached to the hand of the robot, and the position and orientation of the object are detected from the image from the camera as described above. Some do work.

[0005]

However, in the case of realizing work such as grasping of an object by using a camera installed separately from the robot as in the former case, it is necessary to accurately determine the installation positions of the robot and the camera in advance. Is required. Further, it is necessary to perform camera calibration in which the focal length of the camera, the lens distortion coefficient, and the like are calculated in advance.

In addition, in the example of Japanese Patent Application Laid-Open No. 8-63214, the optical axis of the camera is generally perpendicular to the plane of the conveyor, and the x-axis or y-axis in the screen is parallel to the direction of conveyance of the conveyor. It is necessary to set it up. Further, since the object is moving on a belt conveyor or the like, in addition to the position and orientation information of the object detected from the image from the camera, speed information of the conveyor is detected by another means (for example, an encoder of the conveyor). It is necessary to generate a command value to the robot based on the information of the robot. Further, it is necessary to generate a trajectory when the robot grasps the object from the installation position information of the camera and the robot, the position information of the object, and the speed information of the conveyor. Therefore,
There is a problem that the teaching procedure and the processing procedure for trajectory generation become very complicated.

The same problem can be said for the latter example in which a camera is installed at the hand of a robot in the prior art. In other words, when gripping an object with a gripper or the like attached to the robot's hand based on the position and orientation information of the object detected from the image from the camera, the position and orientation information of the object obtained from the camera is used as the reference coordinates of the robot. It is necessary to convert it into position and orientation information from the viewpoint of the system, and the positional relationship between the robot hand and the camera is determined in advance precisely,
So-called hand-eye calibration is required.
In particular, when a sensor such as a camera is installed at the robot hand, the positional relationship between the robot hand and the camera frequently changes due to a malfunction during the work or the like. In such a case, the calibration work needs to be performed again. Also, when the target is moving as in the former example, since the position and orientation information of the target viewed from the robot reference coordinates changes every moment,
Trajectory generation for gripping becomes very complicated.

Conventionally, the former method and the latter method can be used even if the constituent devices (conveyor installation status and transfer speed, or the positional relationship between the camera and the robot) and the shape of the object to be grasped do not change. Each time the robot and sensor to be changed, the high-precision calibration described above,
It was necessary to perform complicated teaching work.

The present invention has been proposed in order to solve the above-mentioned problems, and can save a complicated teaching work and a labor of performing calibration, and can easily perform a predetermined work on an object. It is an object to provide a robot device and a control method thereof.

[0010]

According to the first aspect of the present invention,
A gripping mechanism having a gripping mechanism for gripping an object, wherein the gripping mechanism is configured to be movable; an imaging means fixed to be movable together with the gripping mechanism of the gripping means; And a feature amount extracting means for extracting coordinates of one or more feature amounts from an image of the object captured by the camera, and storing a position of the gripping mechanism when the gripping mechanism grips the object as a first hand position. Storing the position of the gripping mechanism when the gripping mechanism is moved within a range in which the object can be opened and the imaging unit can image the object, as a second hand position; Storage means for storing the coordinates of each feature amount of the object extracted by the imaging means and extracted by the feature amount extraction means at the hand position, and moving from the second hand position to the first hand position Move for Calculating means for calculating a sequence, and the gripping so that the coordinates of each feature amount of the moving object extracted by the feature amount extracting means match the coordinates of each feature amount stored in the storage means. By moving the means, follow-up control means for controlling the gripping mechanism to follow the moving object, and the calculating means calculates when the gripping mechanism is following the moving object. Gripping control means for controlling the gripping means so as to bring the gripping mechanism closer to the moving object based on the movement matrix and to grip the object.

According to a first aspect of the present invention, there is provided a robot apparatus for gripping an object moving relatively to the robot apparatus. Therefore, not only when the robot apparatus is fixed and the object moves, but also when the robot apparatus moves and the object is fixed, or when the robot apparatus and the object move at different speeds. It can also be applied if Further, the object is a linear motion at a constant speed relative to the robot device,
It may be a constant velocity circular motion or a constant acceleration linear motion, but may be relatively stationary.

First, the operator performs a teaching operation as follows.
You need to do two things: First, the gripping mechanism is moved to a first hand position, where the object is gripped. Second, the object is released from the gripping mechanism, and the gripping mechanism is moved to the second hand position within a range where the imaging unit can image the object. Such teaching work is performed, and the first and second hand positions are stored in the storage means. Further, at the second hand position, the coordinates of one or more feature amounts extracted from the image captured by the imaging unit are also stored in the storage unit. The imaging means serves as a sensor of the gripping mechanism, and is fixed so as to move together with the gripping means. Further, a movement matrix for moving from the second hand position to the first hand position is calculated.

When such a teaching operation is completed, a gripping operation for gripping a moving object can be performed. The gripping mechanism is moved so that the coordinates of each feature amount of the moving object coincide with the coordinates of each feature amount stored in the storage unit. That is, so-called visual servo is performed so that the gripping mechanism follows the moving object.
The relative positional relationship between the target and the gripping mechanism at this time is the same as the relative positional relationship between the target and the gripping mechanism during the teaching operation. When the gripping mechanism follows a moving object,
The gripping mechanism is moved closer to the moving object based on the first movement matrix calculated by the calculating means, and the object is gripped.

When performing visual servoing,
It is desirable that the gripping mechanism completely follows the moving object. However, the gripping mechanism may follow a moving object at a predetermined distance. In such a case, it is preferable that the re-teaching operation as described in claim 2 is performed. Specifically, the storage means stores the object extracted by the feature amount extraction means when the gripping mechanism is following the moving object by a predetermined distance by the tracking control means. The coordinates of each feature amount of the object are newly stored, and the following control unit causes the gripping mechanism to completely follow the object from the second hand position using the newly stored coordinates of the feature amounts. What is necessary is just to store the position of the gripping mechanism at the time of pressing as a new second hand position.

According to a third aspect of the present invention, there is provided a control method of a robot apparatus for gripping an object with a movable gripping mechanism, the position of the gripping mechanism when the gripping mechanism grips the object. Is stored as a first hand position, the object is opened, and the position of the gripping mechanism when the gripping mechanism is moved within a range in which the object can be imaged is defined as a second hand position. Storing the object at the second hand position, extracting coordinates of one or more feature amounts from the captured image of the object, storing the coordinates of each extracted feature amount, A movement matrix for moving from the second hand position to the first hand position is calculated, and in a state in which it is movably fixed together with the gripping mechanism,
The moving object is imaged, and coordinates of one or more feature values are extracted from the captured image of the moving object, and the coordinates of each of the extracted feature values of the moving object are stored in the memory. By moving the gripping mechanism so as to match the coordinates of each feature amount being performed, the gripping mechanism follows the moving object, and the gripping mechanism follows the moving object. At this time, the gripping mechanism is brought closer to the moving object based on the calculated movement matrix, and the gripping mechanism is caused to grip the object.

According to the present invention, there is provided a method for controlling a robot apparatus according to the first aspect of the present invention, in which an operator only needs to perform the above-described two things as a teaching operation. That is, the operator can make the gripping mechanism follow the moving object and grip the object only by performing two simple teaching operations.

When performing visual servoing,
It is desirable that the gripping mechanism completely follows the moving object. However, the gripping mechanism may follow a moving object at a predetermined distance. In such a case, it is preferable to perform a re-teaching operation as described in claim 4. Specifically, the gripping mechanism is made to follow the moving object, and is movable together with the gripping mechanism when the gripping mechanism is following the moving object at a predetermined distance. In the state fixed to, the moving object is imaged, coordinates of one or more feature amounts are extracted from the captured image of the moving object, and the coordinates of each extracted feature amount of the object are extracted. Is newly stored, and using the coordinates of each of the newly stored feature amounts, the gripping mechanism is moved so that the gripping mechanism completely follows the target from the second hand position, and completely When following, the position of the gripping mechanism may be stored as a new second hand position.

[0018]

Embodiments of the present invention will be described below in detail with reference to the drawings.

(First Embodiment) As shown in FIG.
The robot apparatus according to the present embodiment grips an object 11 being conveyed by a conveyor 10 in a fixed state. Note that the present invention is not limited to such an embodiment, and can be applied as long as the object 11 is relatively moving with respect to the robot device. That is, the case where the robot apparatus is moving and the object 11 is stationary, or the case where the robot apparatus and the object 11 are moving at different speeds may be used. Further, the same applies to the second and third embodiments.

The robot apparatus has a C for imaging the object 11
A CD camera 1; a sensor controller 2 for extracting a feature amount from an image captured by the CCD camera 1; a gripper 3 for gripping an object 11; a robot arm 4 for moving the gripper 3 in a three-dimensional direction; A robot controller 5 that performs trajectory generation and the like, includes a servo circuit (not shown), an amplifier, and the like, and controls operations of the gripper 3 and the robot arm 4 is provided.

The CCD camera 1 is fixed near the gripper 3 which is the tip of the robot arm 4. The robot controller 5 calculates an error vector E (t) and a speed command value vector V (t) based on the image feature amount Fi obtained by the sensor controller 2, and moves the robot arm 4 according to these values. Object 1 by gripper 3
1 is controlled. In addition, the conveyor 10 is the object 1
1 is transported at a constant speed.

The robot apparatus having the above-described configuration is capable of moving the object 11 before actually gripping the object 11 being transported.
Is performed for grasping the object.

(Teaching operation) First, the operator causes the robot apparatus to grip the target object 11 and
Teach. The position to be gripped may be limited by the gripping mechanism such as the gripper 3 and the shape of the object 11. In addition, a position to be gripped may be defined in advance for a later process (for example, assembling another component to the object 11 or putting it into a component box).

Here, as shown in FIG. 2, for example, the object 11 is formed in the shape of a hexagonal prism and has seven side surfaces 111 to 1.
17 is provided. Six mounting holes (for example, screw holes) for mounting components (not shown) are formed on the upper surface 118 of the object 11. It should be noted that such an assembling hole is not formed in the lower surface 119 of the object 11. The gripped portion of the object 11 is determined by considering the stability when gripped,
1, 115 or the side surfaces 113, 116 are preferred. In the present embodiment, the object 1 having the above shape is used.
Although 1 is used, the present invention is not limited to this shape, and the object 11 may have another shape. And
The teaching operation is specifically performed according to the processing of steps ST1 to ST6 shown in FIG.

In step ST1, the operator
Using a teaching pendant (not shown) or the like, the gripper 3
The process proceeds to step ST2 so as to hold the members 3 and 116.

In step ST2, the robot controller 5 moves the gripper 3 to the object 11 as shown in FIG.
With the held, the position of the gripper 3 in the robot reference coordinate system, that is, the robot hand position Pd is stored, and the process proceeds to step ST3. "Position" here
Is represented by parameters of six degrees of freedom including the posture. Hereinafter, similarly, a parameter having six degrees of freedom including a posture is referred to as “position”.

In step ST3, the robot controller 5 releases the object 11 gripped by the gripper 3, and moves the gripper 3 to an arbitrary position.
The process proceeds to step ST4.

The position at which the gripper 3 moves must be within the field of view of the CCD camera 1. That is, CC
The position is such that the feature amount of the object 11 is extracted from the image captured by the D camera 1. Specifically, the position where the object 11 is not within the field of view of the CCD camera 1,
This is not a position where the D camera 1 captures the image of the object 11 from the side, but a position where a feature amount to be extracted is preferably obtained from the image.

In step ST4, the robot controller 5 determines the position of the gripper 3 in the current robot reference coordinate system, that is, the robot hand position P as shown in FIG.
s is stored, and the routine goes to Step ST5.

Here, the robot hand position changes from Ps to Pd.
Is the position and orientation conversion matrix for _OThen,
The formula holds.

[0031] _{Ps · P O = Pd ∴P O} = Ps -1 · Pd robot controller 5, this time to calculate the position and orientation transformation matrix P _O, and stored. Note that the robot controller 5 may calculate the position and orientation conversion matrix P _O during a gripping operation described later.

In step ST5, the CCD camera 1
Captures the object 11 at the robot hand position Ps, and supplies this image to the sensor controller 2. The sensor controller 2 extracts an image feature amount based on the image from the CCD camera 1, and proceeds to step ST6. Here, the CCD camera 1 obtains an image of the object 11 as shown in FIG. 6, for example.

At step ST 6, the sensor controller 2 processes the upper surface 118 of the object 11
The three mounting holes are extracted as image feature amounts Fi (i = 1 to 6), and the coordinate values (Xi, Y) of the center position of each image feature amount Fi are extracted.
i) Memorize and end the teaching work process.

As a method of measuring the center position of the mounting hole in the image, a known method such as binarized image processing or edge processing may be used. Although the center position of the assembling hole is set as the image feature amount Fi here, other image feature amounts such as a line segment end point, a corner, and an area centroid of an isolated region of the binarized image may be used. Further, at the time of teaching, the operator may artificially define the image feature amount Fi using an input unit (not shown) such as a mouse or a keyboard.

(Gripping operation) When the teaching operation is completed, the robot device grips the object 11 being conveyed by using the coordinates of the image feature amount Fi and the robot hand positions Pd and Ps stored in the operation. I do. In this gripping operation, processing from step ST11 to step ST18 shown in FIG. 7 is executed.

In step ST11, as shown in FIG. 8, the CCD camera 1 captures an image of the transport region of the object 11, that is, on the conveyor 10, and proceeds to step ST12.

In step ST12, the sensor controller 2 determines whether the object 11 has been detected. In order to detect the target object 11 and extract the feature amount from the image, the entire target object 11 needs to be within the range of the field of view of the CCD camera 1. Here, the luminance of the image is used for detecting the target object 11. For example, when the object 11 is located upstream of the conveyor 10 and not within the field of view of the CCD camera 1, the brightness of the image obtained by the CCD camera 1 hardly changes. On the other hand, when the object 11 is conveyed by the conveyor and enters the range of the field of view of the CCD camera 1, its brightness changes. Therefore, the sensor controller 2
A window (not shown) is provided at an appropriate position in the image. If the luminance in the window is equal to or higher than a predetermined threshold, the object 11 is determined to be within the visual field range. Is determined not to be within the range. When the sensor controller 2 detects the target object 11, the process proceeds to step ST13, and when not detected, returns to step ST11.

In step ST13, the sensor controller 2 determines the time T from the image captured by the CCD camera 1.
Are extracted, and the process proceeds to step ST14. Here, FIG.
FIG. 4 is a diagram illustrating an image captured during execution of a gripping operation. At the time of the gripping operation, it is necessary to extract the image feature amount fi (t) from the image captured at time t. However, it should not be extracted for some reason.
3 (t) may be extracted. Therefore, the feature amount f1
It is necessary to perform a corresponding point search to determine which of (t) to f7 (t) corresponds to F1 to F6 shown in FIG.

Various methods have been proposed for the corresponding point search. For example, a so-called collation method such as a dynamic programming method and a maximum faction method described in “Robot Vision” (written by Masahiko Yauchida, Shokodo, 1990). Can be performed. It should be noted that once those corresponding to F1 to F6 are obtained from f1 to f7, the next image feature can be easily obtained.

Specifically, the cycle time of the repetition processing from step ST13 to step ST16 is Δt
Then, the image feature value fi (t + Δt) at the time (t + Δt) following the time t is the image feature value fi in the previous image.
What is necessary is just to extract from the vicinity of (t). As a result, the stability of feature amount extraction is increased, and the load of calculation processing can be reduced.

In step ST14, the sensor controller 2 determines whether the error vector E (t) at the time t is sufficiently small, specifically, satisfies || E (t) || <ε. Step ST if sufficiently small
The process proceeds to step ST15 if it is not sufficiently small.
Move to

Here, the error vector E (t) is obtained based on the image feature amount fi (t) extracted from the current image of the CCD camera 1 and the image feature amount Fi stored in step ST6 described above. .

In the present invention, reference is made to B. Espiau et al., A New Appr.
oach to Visual Servoing in Robotics, IEEE Trans.Rob
otics and Automation, 8 (3), 313-326, 1992, and literature F. Be
nsalha et al., Real Time Visual Tracking using the Gene
ralized Likelihood Ratio Test, Int.Conf.Automation,
Based on the method described in Robotics and Computer Vision, 1379-1383, 1994, an error vector E (t)
A later-described speed command value vector V (t) is calculated.

The image feature value Fi = (Xi, Yi) during teaching and the image feature value fi obtained from the image captured at time t.
Using (t) = (Xi (t), Yi (t)), a feature quantity deviation vector e (t) is obtained by the following equation (1).

[0045]

(Equation 1)

(Xi, yi) and (xi (t), yi
(T)) are represented by pixel values (Xi, Yi) and (Xi, Yi).
i (t), Yi (t)) are converted into values in the camera coordinate system. N represents the number of image features.

The error vector E (t) is represented by the following equation (2) using the feature amount deviation vector e (t).

[0048]

(Equation 2)

Here, L ^{T +} is the image feature amount F, F (t).
It is a pseudo inverse matrix of the image Jacobian L ^T obtained from.

Further, the sensor controller 2 has || E
(T) When the criterion of || <ε is satisfied, it is determined that the object 11 that has moved is followed. ε is a scalar quantity given in advance by the operator. Alternatively, if E = (E1 E2 E3 E4 E5 E6) ^T , the criterion shown in equation (3) may be used.

[0051]

(Equation 3)

In step ST15, the robot controller
The controller 5 calculates a speed command value vector V (t),
The process moves to step ST16. Note that the speed command value vector
V is the translation speed of the robot hand position P (t_x, T_y, T
_z) And rotational speed (ω_x, Ω _y, Ω_z)
V = (t_x t_y t_z ω_x ω_y ω_z)^T It is expressed as Then, the speed finger at the time (t + Δt)
The command value vector V (t + Δt) is obtained from equation (4).
You.

[0053]

(Equation 4)

Here, λ is a speed gain. The rightmost term on the right side of the equation (4) is an estimated value of the motion speed of the object 11 and is obtained from the equation (5).

[0055]

(Equation 5)

As described above, the robot controller 5 sets the error vector E (t) and the speed command value vector V
(T) is calculated.

In step ST16, the robot controller 5 calculates the speed command value vector V (t)
, A trajectory generation, a joint angular velocity / angular acceleration are generated, the robot arm 4 is controlled according to these, and the process returns to step ST13.

At step ST17 when it is determined in step ST14 that E (t) is sufficiently small, the robot controller 5 determines that the gripper 3 has followed the object 11, and approaches the object 11. Start. In the following description, causing the gripper 3 to follow the target by performing the processing of the flowchart shown in FIG. 7 is referred to as visual servoing.

[0059] Here, when the time t ₁ the migrated time in step ST14, for example, as shown in FIG. 10, the speed command value vector V _{actual are} (t _1), the position vector of the robot hand position P (t ₁ ).

Time t₁, The robot hand position P
Is following the target object 11, so the speed command at that time is
Value vector V_actual(T₁) Is the velocity vector of the object 11
Equal to Therefore, the robot controller 5
Hand position P (t₁) To P_OSpeed for moving by
In the turn, the speed command value vector V
_actual(T ₁) Is generated.
Note that P_OIs the position and orientation transformation matrix shown in the teaching work.
You. Also, P_OThe speed pattern for moving by the minute
Calculated by conventional methods such as trapezoidal speed patterns
be able to.

[0061] Further, the robot controller 5, as it takes time T _O to move P _O content relative to calculate the P _a according to the following equation.

[0062] _{P a = P (t 1)} + P O + V actual (t 1) · T O The robot controller 5 controls the robot arm 4 on the basis of the calculated P _a. As a result, FIG.
As shown in 1, it is possible to position P _a to be gripped an object 11 from the hand position P (t ₁₎ moves the robot hand (gripper 3). When such an approach ends, the process moves to step ST18.

In step ST18, the robot controller 5 holds the object 11 with the gripper 3, and ends the processing.

As described above, the robot apparatus stores the robot hand positions Ps and Pd during the teaching operation, follows the object 11 being conveyed by the visual servo, and uses the robot hand positions Ps and Pd to store the object. The work of gripping the object 11 can be performed. Accordingly, the robot apparatus can easily grip the transported object 11 without being restrained by the posture position of the transported object 11 and without stopping the conveyor 10.

Further, the robot apparatus does not need to previously determine the installation position and the positional relationship among the CCD camera 1, the gripper 3, and the object 11, so that, for example, when the installation conditions of the CCD camera 1 change. Even
The object 11 being conveyed can be gripped only by the simple teaching operation as described above. Furthermore, even when the same work is to be realized by different robot devices, the CCD camera 1 and the gripper 3, or when the object 11 to be gripped is different, only a simple teaching work is required.

As a result, the operator can greatly reduce the load on the conventional teaching work and programming. Furthermore, even if changes occur in the CCD camera 1, the gripper 3, and the object 11, it is possible to flexibly respond.

In the above-described embodiment, the robot device is used to move the object 1 conveyed by the conveyor 10.
Although the example of gripping the object 1 has been described, the same applies to the case of gripping the stationary target object 11. The teaching operation is the same as described above.

In the gripping operation, step ST1 shown in FIG.
4, when the error vector E (t) becomes sufficiently small, the speed command value vector V (t) becomes zero,
Similarly, the speed V of the actual robot hand position
_actual (t ₁ ) also becomes zero.

Therefore, in step ST17,
The robot controller 5 calculates the current robot hand position P
The object 11 can be gripped using only (t ₁ ) and the position change P _O during teaching. That is, the position P _a to be gripped is determined by the following equation.

P _a = P (t ₁ ) + P _{O The} robot controller 5 controls the robot arm 4 based on the calculated P _a, and the stationary object 11 can be gripped by the gripper 3. it can.

(Second Embodiment) Next, a second embodiment of the present invention will be described. The same parts as those in the first embodiment are denoted by the same reference numerals, and the description thereof will not be repeated.

In the first embodiment, a complete follow-up state in which the relative positional relationship between the object being conveyed and the robot hand position P is substantially the same as their relative positional relationship during the teaching operation is described. explained. On the other hand, in the second embodiment, a case will be described in which the robot apparatus grips the target object when the robot apparatus is not in the perfect following state.

Here, the following state is defined as a state in which the relative positional relationship is kept constant. For example, if the object is at a speed V ₀
When the robot is exercising at the speed V
It is a state of exercising at ₀ . Next, as the following state,
Two "perfect following states" and "incomplete following states" are defined.

[0074] The "perfect tracking state", a state where the relative positional relationship between the robot hand position P and the object is continued for a certain time remains constant value P _O, i.e. their relative during teaching operation It is a state where the same positional relationship is obtained.
The "Incomplete follow-up state", a state where the relative positional relationship between the robot hand position P and the object is continued for a certain time remains constant value _{P '0 (≠ P O)} , i.e. the robot hand position P Although there is no change in the relative position between the object and the object, it means a state different from the relative positional relationship during the teaching operation. Table 1 shows the image error, the robot hand position P, the speed of the robot hand position P, and the acceleration of the robot hand position P for these two states.

[0075]

[Table 1]

In the “complete following state”, the robot hand position
The relative positional relationship between P and the object is P _OAnd
The robot hand position P moves at the same speed as the object. This
Therefore, the approaching motion is the approaching movement while maintaining the following motion V
P_OBy performing the movement of
You.

On the other hand, even in the "incomplete follow-up state", the robot hand position P is moving at the same speed as the object. However, the relative positional relationship P _'0 with the robot hand position P and the object is different from the positional relationship between P _O during teaching operation. For example, as shown in FIG. 12A, the initial image of the target 11 at the time of starting following does not match the target image. At this time, the speed of the robot hand position Ps is increased as shown in FIG. As shown in FIG. 12B, the image of the target object 11 in the following state is in a state of maintaining a predetermined shift from the target image. At this time, the speed of the robot hand position Ps is V _actual (t ₁ ), as shown in FIG.

[0078] Therefore, if it is possible to teach the relative positional relationship between P _'0, as shown in FIG. 12 (C), it is possible to obtain exactly the same effect as "complete follow-up state". In addition,
The positional relationship P ′ ₀ in the “incomplete tracking state” may change due to a change in the moving speed of the target object, a change in the gain of the visual servo, or the like, and it is necessary to re-teach each time.
In the present embodiment, a re-teaching operation is performed so as to be able to cope with the “incomplete following state”.

(Re-teaching operation and gripping operation) In the re-teaching operation, an object is first followed by using a visual servo. At this time, the robot controller 5 extracts the image feature value F′i of the target object and stores the coordinates of each image feature value F′i. The process of extracting the image feature value F′i is the same as the process of step ST6 shown in FIG.

Next, the processing from step ST1 to step ST3 shown in FIG. 3 is performed, and the gripper 3 is moved to an arbitrary position (robot hand position Ps). The robot controller 5 further performs visual servo using the image feature amount Fi ′, and sets the position of the gripper 3 that has reached the following state as the robot hand position P ′s. Then, the robot controller 5 calculates a position / posture conversion matrix P ′ _O (= P ′ _S ⁻¹ Pd) in the following state, and stores the position / posture conversion matrix P ′ _O.

In the gripping operation, as in the first embodiment, steps ST11 to ST18 shown in FIG.
The processing up to is performed. In this case, instead of the image feature quantity Fi and the position and orientation transformation P _O obtained in the first embodiment may be used an image feature amount F'i and the position and orientation transformation industry column P _'O.

As described above, according to the robot apparatus according to the second embodiment, the robot hand position P is
Even if the object 11 is not completely followed, the object 11 can be grasped only by performing the simple teaching operation described above without performing complicated adjustment.

(Third Embodiment) In the first and second embodiments, the case of stationary motion and constant velocity linear motion has been described as an example. It can be extended to circular motion. Then, in this gripping operation as well, the processing from step ST11 to step ST17 shown in FIG. 7 is performed as in the first embodiment.

Specifically, in the case of the linear motion with the uniform acceleration, the robot controller 5 determines at time t ₁ when the error vector E (t) becomes sufficiently small, the robot hand position P
Determines that the position has been controlled to the position of P _O or P ′ _O relative to the object 11, and V at this time t ₁
and _actual (t _1), and hand acceleration acc _{actual are} velocity components V _estimate generated from (t ₁₎ (t), the speed command value obtained by superimposing the speed command value generated from the P _O or P _'O Generated and the robot hand position P at time t ₁
From (t ₁ ), the target 11 is approached and the gripper 3 grips the target 11.

[0085] That is, the robot controller 5, as shown in FIG. 14, controls the robot hand position Ps to move P _O amount corresponding gripper 3 when it is follow-up state. At this time, the robot controller 5
As _O partial takes time T _'O to move relative to calculate the P _a according to the following equation.

[0086]

(Equation 6)

In the above-described embodiment, the case where the object is gripped has been described as an example, but the present invention is not limited to this. For example, the present invention can be similarly applied to a case where components are attached to the object 11 or the like.

[0088]

According to the robot apparatus and the control method thereof according to the present invention, the operator can perform the following two tasks as a teaching operation.
You only need to do one thing. First, the gripping mechanism is moved to a first hand position, where the object is gripped. Second
Then, the object is released from the gripping mechanism, and the gripping mechanism is moved to the second hand position within the imaging range. The coordinates of the feature amounts of the object at the first and second hand positions and the second hand position which are stored at this time are used to follow the object and grasp it. That is, the operator can cause the robot apparatus to grasp the moving object only by performing a simple teaching operation.

[Brief description of the drawings]

FIG. 1 is a diagram showing a configuration of a robot device according to a first embodiment of the present invention.

FIG. 2 is an external perspective view of an object gripped by a robot device.

FIG. 3 is a flowchart showing processing contents in a teaching operation of the robot apparatus.

FIG. 4 is a diagram showing a schematic configuration showing a robot hand position Pd in a state where an object is gripped in a teaching operation.

FIG. 5 is a diagram showing a schematic configuration showing a robot hand position Ps when an object is released in a teaching operation.

FIG. 6 is a diagram showing a captured image of a camera in a teaching operation.

FIG. 7 is a flowchart showing processing contents in a gripping operation of the robot device.

FIG. 8 is a diagram for explaining that a camera captures an image of a conveyor that conveys an object in a gripping operation.

FIG. 9 is a diagram showing a captured image of a camera in a gripping operation.

FIG. 10 shows a robot hand position P (t ₁ ) at time t ₁ .
FIG. 4 is a diagram for explaining a speed command value vector V _actual (t ₁ ).

FIG. 11 is a diagram illustrating a state in which an object is gripped.

FIG. 12 is a diagram illustrating an image of an object in an initial state, a following state, and a re-teaching state in the robot apparatus according to the second embodiment.

FIG. 13 shows a speed command value vector V of the robot hand position P.
It is a figure explaining (t).

FIG. 14 is a diagram illustrating a state in which the robot apparatus according to the third embodiment of the present invention has gripped an object being transported.

[Explanation of symbols]

 DESCRIPTION OF SYMBOLS 1 Camera 2 Sensor controller 3 Gripper 4 Robot arm 5 Robot controller

Continued on the front page F term (reference) 3F059 BC07 BC09 CA05 DA02 DA05 DA08 DB04 DB09 DD01 DE02 FA01 FA03 FA05 FA10 FB01 FB16 FB22 FC04 FC13 FC14 5H269 AB21 AB33 CC09 EE10 JJ09

Claims (4)

[Claims] 1. A robot apparatus for gripping a relatively moving object, comprising: a gripping mechanism having a gripping mechanism for gripping the object, wherein the gripping mechanism is configured to be movable; and gripping the gripping means. An imaging unit that is movably fixed together with a mechanism and captures an image of an object; a feature amount extraction unit that extracts coordinates of one or more feature amounts from an image of the object captured by the imaging unit; The position of the gripping mechanism at the time of gripping the object is stored as a first hand position, and the object is released and the gripping mechanism is moved within a range where the imaging unit can image the object. The position of the gripping mechanism at the time is stored as a second hand position,
Storage means for storing the coordinates of each feature amount of the object extracted by the imaging means and extracted by the feature amount extraction means at the hand position, and moving from the second hand position to the first hand position Calculation means for calculating a movement matrix for the moving object, and the coordinates of each feature quantity of the moving object extracted by the feature quantity extraction means coincide with the coordinates of each feature quantity stored in the storage means. Moving the gripping means to follow the gripping mechanism with the moving object; andfollowing control means that controls the gripping mechanism to follow the moving object. A gripping control unit that controls the gripping unit so that the gripping mechanism approaches the moving object based on the movement matrix calculated by the unit and grips the object. 2. The object extracted by the feature amount extracting means when the gripping mechanism is following the moving object by a predetermined distance by the following control means. The coordinates of each feature amount of the object are newly stored, and the following control unit causes the gripping mechanism to completely follow the target object from the second hand position using the newly stored coordinates of each feature amount. The robot apparatus according to claim 1, wherein the position of the gripping mechanism when stored is stored as a new second hand position. 3. A control method for a robot apparatus for gripping a relatively moving target by a gripping mechanism configured to be movable, wherein a position of the gripping mechanism when the gripping mechanism grips the target is set to a first position. The position of the gripping mechanism when the object is opened and the gripping mechanism is moved within a range in which the object can be imaged is stored as a second hand position. Capturing the object at the second hand position, extracting coordinates of one or more feature amounts from the captured image of the object, storing the coordinates of the extracted feature amounts, A movement matrix for moving from the hand position to the one hand position is calculated, and the moving object is imaged in a state in which the moving object is fixed together with the gripping mechanism. 1 or less from the image By extracting the coordinates of the feature amount of the extracted, by moving the gripping mechanism so that the coordinates of each of the extracted feature amounts of the moving object match the coordinates of each of the stored feature amounts, The gripping mechanism is made to follow the moving object, and when the gripping mechanism is following the moving object, the gripping mechanism is moved to the moving object based on the calculated movement matrix. A control method of a robot apparatus that causes the gripping mechanism to approach the target object. 4. The apparatus according to claim 1, wherein the gripping mechanism follows the moving object, and is movable with the gripping mechanism when the gripping mechanism follows the moving object at a predetermined distance. In the state fixed to, an image of the moving object is imaged, coordinates of one or more feature amounts are extracted from the captured image of the moving object, and the coordinates of each of the extracted feature amounts of the object are Is newly stored, and the gripping mechanism is moved so that the gripping mechanism completely follows the object from the second hand position by using the coordinates of the newly stored feature amounts. When following, the position of the gripping mechanism is changed to a new second position.
The control method for a robot device according to claim 3, wherein the information is stored as a hand position of the robot.