JP2006224291A - Robot system - Google Patents

Abstract

PROBLEM TO BE SOLVED: To detect a three-dimensional position and posture of an object with high accuracy.
A robot includes a robot, a robot control device that controls the robot, an imaging device provided at the tip of the robot arm, and an image processing device that processes an image captured by the imaging device. In the robot system that operates the object 100, the image processing apparatus 20 performs image processing on the model image registration unit 1 that registers two or more reference images that are characteristics of the object 100 and the reference image registered in the model image registration unit. A detection processing unit 3 that detects positions on the camera coordinate system of three points of the object in the captured image, and a three-dimensional position conversion unit 4 that converts the detection points on the camera coordinate system into a three-dimensional position. .
[Selection] Figure 1

Description

  The present invention relates to a robot system that detects the position of an object with an image processing apparatus and operates the object.

A conventional image processing apparatus can detect the position and orientation of a target object by imaging a piled target object or a target object stored in a predetermined area at an arbitrary position and orientation. (For example, refer to Patent Document 1). Specifically, the position of the object is detected as follows.
First, the camera attached to the hand of the robot is moved to sequentially image the workpiece, and each model is registered. Then, the relative position and orientation of the camera and the workpiece are obtained and stored. At that time, the robot is operated so that the workpiece enters the field of view, and then a plurality of model images are sequentially matched, and the relative positional relationship of the camera is determined by the model image having the highest matching degree among the matched model images.
As described above, the conventional image processing apparatus captures a model of one workpiece from a plurality of camera positions and postures, stores them in association with relative positions, and then sequentially matches the models to obtain the highest matching degree. A high model is obtained, and the three-dimensional position and posture of the object are obtained from the three-dimensional position and posture associated with the model.
Japanese Patent No. 3377465 (page 4-6, FIGS. 5 and 6)

However, the conventional image processing apparatus stores a plurality of model images of the same object for each of a plurality of different camera positions and orientations, and matches the imaging with the plurality of model images. The accuracy depends on the accuracy of the camera position and orientation registered in advance.
For example, if the camera position is registered in increments of 10 °, the detection accuracy is also a resolution in increments of 10 °.
For this reason, the hand should be adjusted so that the clearance (allowable accuracy) between the tool attached to the tip of the robot (a hand device for manipulating the object, such as a hand for handling the object) and the object is increased. There was a problem that the control accuracy of the robot was lowered because it had to be manufactured.
The present invention has been made in view of such problems, and an object of the present invention is to provide a robot system including an image processing apparatus that detects a three-dimensional position and posture of an object with high accuracy.

In order to solve the above-described problem, the first aspect of the present invention provides a robot, a robot control device that controls the robot, an imaging device provided at an arm tip of the robot, and an image captured by the imaging device. In a robot system that operates an object, the image processing apparatus includes: a model image registration unit that registers two or more reference images that are characteristics of the object; and a model image registration unit A registered reference image is subjected to image processing, and a detection processing unit that detects positions on the camera coordinate system for the target in the captured image, and detection points on the camera coordinate system are set to a three-dimensional position. And a three-dimensional position conversion unit for conversion.
According to a second aspect of the present invention, the image processing apparatus includes a model distance registration unit that registers a distance between two or more registered reference images.
The third aspect of the present invention is characterized in that the number of the reference images is three.
According to a fourth aspect of the present invention, a coordinate system is generated from the positions of the three points converted by the three-dimensional position conversion unit.
The fifth aspect of the present invention is characterized in that the reference image is two or more with respect to one characteristic portion of the object.

According to the first aspect, the image processing apparatus performs image processing on the reference image registered in the model image registration unit and the model image registration unit that registers two or more reference images that are the characteristics of the target object. A detection processing unit that detects the position of three points on the camera coordinate system and a three-dimensional position conversion unit that converts the detection point on the camera coordinate system into a three-dimensional position. Thus, it is possible to provide a robot system that detects the three-dimensional position and posture of an object.
According to the second invention, since the image processing apparatus includes the model distance registration unit that registers the distance between two or more registered reference images, even when the reference image is 2, the three-dimensional object There is an effect that the position and orientation can be detected.
According to the third invention, since the number of reference images is 3, there is an effect that the operator can easily operate objectively when there are three different features in one object.
According to the fourth invention, since the coordinate system is generated from the positions of the three points converted by the three-dimensional position conversion unit, there is an effect that the robot locus registered in advance can be easily corrected.
According to the fifth invention, since the reference image is 2 or more with respect to a part that is one feature of the object, there is an effect that the three-dimensional position and orientation of the object can be detected with higher accuracy.

  Hereinafter, embodiments of the present invention will be described with reference to the drawings.

  FIG. 1 is a configuration diagram of a robot system of the present invention provided with an image processing apparatus. In the figure, the robot 10 is controlled by a robot control device 21, and two cameras 20 are mounted on the hand portion. The camera 20 and the image processing apparatus 30 are connected by a power line and a signal line, and the camera 20 is controlled by the image processing apparatus 30. The objects 100 are randomly piled as shown in the figure, and the robot 10 is equipped with a hand for handling the objects.

  Two cameras 20 are used to image the same object, which is generally known as stereo binocular vision. When the same point of the same object is photographed by two cameras by this stereo binocular vision, it is perspective-transformed based on the two-dimensional position of the camera coordinate system and converted to a three-dimensional position. In this conversion, a plurality of camera parameters are used, and the camera parameters are divided into internal parameters and external parameters. The internal parameters include parameters that define the camera coordinate system and the geometric correction coefficient of the lens, and the external parameters include information on the three-dimensional position and orientation of the camera in the world coordinate system.

  The image processing apparatus 30 includes a model image registration unit 1, a model distance registration unit 2, a detection processing unit 3, a three-dimensional position conversion unit 4, and a communication unit 5. A monitor to display and a pendant to move the cursor are also included.

In the model image registration unit 1, feature points of the object 100 to be detected are registered in advance, and the feature points include image data and various parameters necessary for image processing. FIG. 2 shows an example thereof, in which 300 is a main reference image, and 301 and 302 are sub-reference images.
The model distance registration unit 2 registers a three-dimensional distance between the detection positions.
The detection processing unit 3 performs image processing on the images captured by the two cameras 20 and the model of the model image registration unit 1, and on the camera coordinate system of the main reference image 300 and the sub-reference images 301 and 302 of the object 100. A two-dimensional position is detected.
The three-dimensional position conversion unit 4 converts each reference image into a three-dimensional position according to the positions detected by the two cameras 20.

Next, the operation of the image processing apparatus 30 will be described in order along the flowchart of FIG. First, processing at the time of registration, that is, processing at the time of teaching will be described.
(Step A) One of the objects 100 is imaged by the two cameras 20.
(Step B) As shown in FIG. 2, registration is made as a main reference image 300 and sub-reference images 301 and 302.
(Step C) Using the position of the main reference image 300 as the origin, the direction of the slave reference image 301 viewed from the origin as the X-axis direction, and the plane including the slave reference image as the XY plane, as shown in FIG. Create This coordinate system M is a coordinate system on the coordinate system of the robot.

Next, processing at the time of execution will be described in order.
(Step D) The robot controller 21 moves the camera 20 to a position where the object 100 can be imaged.
(Step E) The robot control device 21 outputs a detection start command to the image processing device 30.
(Step F) The robot control device 21 detects a position matching the main reference image 300 and obtains a three-dimensional position.
(Step G) The robot controller 21 detects a position that matches the sub-reference images 301 and 302, and obtains a three-dimensional position of the two points.
(Step H) The coordinate system N is created by the same method as in Step C from the three-dimensional position obtained by the robot controller 21 in Steps F and G.
(Step I) The change amount of the three-dimensional position and orientation of the object 100 is calculated from the coordinate systems M and N. The image processing device 30 transmits the change amount obtained in step I to the communication unit 6 of the robot control device 21 through the communication unit 5.
(Step J) After the communication unit 6 of the robot control device 21 receives the change amount of the position and posture, the change command is sent to the operation command unit 7 and the robot 10 is operated to thereby move the position and posture of the robot 10. To change.

  By executing the above steps, the robot 10 can recognize the three-dimensional position and posture of the object 100. And based on this position and attitude | position, it becomes possible to operate the target object 100 by correcting the robot locus registered beforehand.

As a second embodiment, a case where there are two images that are characteristic of the object 100 will be described.
As shown in FIG. 3, only two types of characteristic images, a main reference image 300 and a sub reference image 310, are registered in the object 100. In this case, the model distance registration unit 2 calculates the distance between each of the three-dimensional position of the main reference image 300 and the three-dimensional position of the sub-reference image 310, and creates a data table 400 as shown in FIG. The distance L1 is a distance between points P0-P1, and the distance L2 is a distance between distances P0-P2. The coordinate system M is created with the position of the main reference image as P0, P0 as the origin, P1 as the X-axis direction, and P2 as a point on the XY plane.

At the time of execution, the three-dimensional position is obtained after detecting the main reference image 300, and the three-dimensional position is obtained after detecting the sub-reference image 310. Then, each distance is calculated from the three-dimensional position of the sub-reference image 310 and the three-dimensional position of the main reference image, and a three-dimensional position that gives the same distance as the registered points P1 and P2 is obtained from each distance. Find out. Next, a coordinate system N is created with the detection position of the main reference image as Q0, the point corresponding to P1 as Q1, and the point corresponding to P2 as Q2. Based on the coordinate systems M and N, the amount of change in the three-dimensional position and orientation is obtained.
Since the posture and the amount of change are obtained in this way, it is possible to detect even when there is one type of sub-reference image different from the main reference image.

Next, Example 3 will be described with reference to FIG. In this example, the model image registration unit 1 registers two models for one characteristic portion with 320 as a main reference macro image and 321 as a main reference micro image. Macro images are used, and reference numerals 331 and 341 are sub-standard micro images.
The operation of the detection processing unit 3 in this example will be described with reference to the flowchart of FIG.
(Step K) The image captured by the two cameras 20 and the main reference macro image 320 are subjected to image processing to detect a two-dimensional position on the camera coordinate system.
(Step L) From the result of Step K, for example, a detection condition such as a processing range is reset.
(Step M) Under the detection conditions reset in Step L, the main reference micro image 321 is subjected to image processing, and a two-dimensional position on the camera coordinate system is detected.
Since the detection is performed in such a procedure, even when the target image 100 as shown in FIG. 9 is inclined and the input image is different from the reference image, the detection can be performed with high accuracy.

  The present invention is useful for a robot system that recognizes a three-dimensional position of an object by a robot.

The figure which shows the structure of the robot system of this invention Illustration explaining the object The figure explaining the 2nd subject Diagram explaining the coordinate system of the object Flowchart explaining operation procedure The figure explaining the 2nd subject and its data structure The figure explaining the 3rd subject Flowchart explaining second operation procedure The figure explaining the 3rd subject

Explanation of symbols

1 model registration unit, 2 model distance registration unit, 3 detection processing unit,
4 3D position conversion unit, 5 communication unit, 6 communication unit,
7 Operation command section, 10 robot, 20 camera,
21 robot control device, 22 hand, 30 image processing device,
100 objects

Claims (5)

A robot that operates a target, comprising: a robot; a robot control device that controls the robot; an imaging device provided at an arm tip of the robot; and an image processing device that processes an image captured by the imaging device In the system,
The image processing apparatus includes:
A model image registration unit for registering two or more reference images as features of the object;
A detection processing unit that performs image processing on a reference image registered in the model image registration unit, and detects positions on the camera coordinate system of three points for the object in the captured image;
A three-dimensional position conversion unit that converts detection points on the camera coordinate system into three-dimensional positions;
A robot system comprising:   The robot system according to claim 1, wherein the image processing apparatus includes a model distance registration unit that registers a distance between two or more registered reference images.   The robot system according to claim 1, wherein the number of the reference images is three.   The robot system according to claim 1, wherein a coordinate system is generated from the positions of the three points converted by the three-dimensional position conversion unit.   The robot system according to claim 1, wherein the reference image is two or more for a part that is one characteristic of the object.

Images