JP2019018339A - Robot system - Google Patents

Abstract

To provide a robot system that detects at least one of a conveyance speed and a position of an object being conveyed by a conveyance device and also detects a position and an attitude of the object with high precision, and can perform a proper operation on the object.SOLUTION: A robot system 1 comprises: a conveyance device 2 which conveys an object O; a robot 3 which performs processing on the object O being conveyed; a visual sensor 4 which acquires visual information on the object O having been conveyed by the conveyance device 2; a high-period processing part which processes the visual information acquired by the visual sensor 4 in a first period and detects at least one of a conveyance speed and a position of the object O being conveyed by the conveyance device 2; a low-period processing part which processes the visual information acquired by the visual sensor 4 in a second period shorter than the first period and detects a position of the object O; and a control part 6 which controls the robot based upon at least one of the conveyance speed and position of the object O detected by the high-period processing part and the position of the object O detected by the low-period processing part.SELECTED DRAWING: Figure 1

Description

  The present invention relates to a robot system.

  Conventionally, images of the articles conveyed by the conveyor and marks attached to the conveyor are imaged, and the movement speed of the articles conveyed by the conveyor is detected from the obtained images, and the position of the robot hand is synchronized with the movement of the articles. There is known a robot system that grips a moving article with a robot hand by controlling (for example, see Patent Document 1).

  In addition, the workpiece on the conveyor is identified by the image acquired by the visual sensor in synchronization with the signal from the encoder that detects the amount of movement in the conveyor transport path, and the robot's tip hand is positioned on the identified workpiece. There is known a robot system that is handled (see, for example, Patent Document 2).

JP 2009-28818 A JP 2012-192466 A

  However, in order to detect the amount of movement of the conveyor from the image acquired by the visual sensor with a resolution comparable to that of the encoder, it is necessary to detect the position of the article at a high cycle of about 1 ms, while being conveyed by the conveyor. In order to recognize the shape and posture of the coming article with high accuracy, enormous calculation is required for processing with a large load such as pattern matching, and there is an inconvenience that processing cannot be performed at a high cycle.

  The present invention has been made in view of the above-described circumstances, and based on visual information detected by a visual sensor, detection of at least one of a transport speed and a position of a target being transported by a transport device, An object of the present invention is to provide a robot system that can detect a position with high accuracy and perform an appropriate operation by a robot on an object being conveyed by a conveying device.

In order to achieve the above object, the present invention provides the following means.
One embodiment of the present invention acquires a transport device that transports a target, a robot that performs processing on the target transported by the transport device, and visual information of the target transported by the transport device. A visual sensor, and a high-cycle processing unit that processes the visual information acquired by the visual sensor in a first cycle to detect at least one of a conveyance speed and a position of the object being conveyed by the conveyance device; A low-cycle processing unit that detects the position of the target by processing the visual information acquired by a visual sensor in a second cycle longer than the first cycle, and the conveyance of the target detected by the high-cycle processing unit A robot that includes at least one of a speed and a position, and a control unit that controls the robot based on the position of the target detected by the low-cycle processing unit. To provide a door system.

  According to this aspect, when a plurality of targets are transported in one direction by the transport device, the visual information of the target is acquired by the visual sensor, and the acquired plurality of visual information is the high cycle processing unit and the low cycle processing unit. Sent to. In the high cycle processing unit, the visual information acquired by the visual sensor is processed in the first cycle, so that at least one of the conveyance speed and position of the target that can be calculated with a relatively small processing load is the first in the high cycle. Detected with a period. Thereby, at least one of the conveyance speed and the position of the article can be detected with a resolution equivalent to that of the encoder.

On the other hand, in the low cycle processing unit, visual information is processed in a second cycle longer than the first cycle, so that it is possible to perform a process that requires enormous calculation, and the target position is detected with high accuracy. be able to.
As a result, the robot is moved based on at least one of the conveyance speed and position of the target detected by the high cycle processing unit so that the robot follows the target being conveyed by the conveyance device, and is detected by the low cycle processing unit. Based on the position of the target, the robot can accurately process the target being transported by the transport device.

In the above aspect, the high cycle processing unit may process a part of the visual information acquired by the visual sensor in the first cycle.
By doing in this way, the amount of visual information processed by the high cycle processing unit can be reduced, and at least one of the moving speed and position of the target can be easily detected at a high cycle.

In the above aspect, the visual sensor may acquire the target image on the transport device as the visual information.
In this way, the target image acquired at different times by the visual sensor can be processed to detect the conveyance speed of the target in the first period, and pattern matching is performed on the target image acquired by the visual sensor. It is possible to detect the position of the target with high accuracy by performing a process that requires enormous calculation such as in the second period.

Moreover, in the said aspect, the said conveying apparatus may be provided with the mark moved at the same speed as the said object.
By doing in this way, even if the target is not transported by the transport device, it is possible to accurately detect the transport speed of the target by acquiring the visual information of the mark moving by the transport device.

Moreover, in the said aspect, the said high cycle process part may detect the said conveyance speed and the rough position and attitude | position of the said object.
By doing in this way, a high cycle processing part detects a rough position and posture in addition to a target conveyance speed. Since the high-cycle processing unit processes visual information at a high cycle, it is difficult to detect a highly accurate position and orientation, but it is possible to detect a rough position and orientation, and to process the target. It can be used as information for the initial motion of the robot that performs the operation efficiency.

Moreover, in the said aspect, the said robot may work while chasing the said object currently conveyed by the said conveying apparatus.
By doing so, the robot is moved so as to follow the object being transported using the transport speed of the object detected by the high-cycle processing unit, and the target detected accurately by the low-cycle processing unit Using the position, the object being transported can be removed from the transport device without error.

In the above aspect, the visual sensor may output the visual information immediately before or immediately after receiving the trigger by an external trigger.
In this way, visual information is output as required by a trigger from a high-cycle processor, low-cycle processor, or other equipment outside the visual sensor, and highly accurate position / posture detection, visual It can be used for sensor adjustment and execution status confirmation.

  According to the present invention, based on the visual information detected by the visual sensor, detection of at least one of the conveyance speed and position of the object being conveyed by the conveyance device and high-precision detection of the position of the object are performed. There is an effect that an appropriate work can be performed by the robot on the object being transported by the transport device.

1 is an overall configuration diagram showing a robot system according to an embodiment of the present invention. It is a block diagram which shows the robot system of FIG. It is a figure which shows the time change and conveyance speed of the image which are acquired with the camera of the robot system of FIG. FIG. 4 is a view similar to FIG. 3 showing a case where a plurality of articles are photographed within the same field of view in the robot system of FIG. 1. It is a whole block diagram which shows the 1st modification of the robot system of FIG. It is a whole block diagram which shows the 2nd modification of the robot system of FIG. It is a whole block diagram which shows the 3rd modification of the robot system of FIG.

A robot system 1 according to an embodiment of the present invention will be described below with reference to the drawings.
As shown in FIG. 1, the robot system 1 according to the present embodiment includes a conveyor (conveying device) 2 that conveys the target article O, a robot 3 installed near the conveyor 2, and a robot 3. A two-dimensional camera (visual sensor) 4 installed downward above the conveyor 2 on the upstream side in the conveying direction, an image processing unit 5 for processing an image acquired by the two-dimensional camera 4, and processing by the image processing unit 5 And a control unit 6 for controlling the robot 3 based on the result.

The conveyor 2 is, for example, a belt conveyor, and includes a belt 7 that carries the article O and conveys it in one direction. The belt 7 is driven by a motor (not shown).
The robot 3 may be of any type such as a flat type or a ceiling type. For example, the robot 3 includes a robot hand 8 that can hold the article O at the tip of the wrist.

  The two-dimensional camera 4 has a field of view fixed in a partial region in the conveyance direction on the conveyor 2 and acquires a two-dimensional image of the article O conveyed on the conveyor 2. The two-dimensional camera 4 acquires, for example, a two-dimensional image (visual information) at a high period (first period) of 1 ms and outputs the two-dimensional image to the image processing unit 5 and two-dimensionally according to a trigger input from the outside. An image is output.

As shown in FIG. 2, the image processing unit 5 is acquired in a high cycle processing unit 9 that performs image processing on a part of the two-dimensional image acquired in the first cycle, and in a second cycle longer than the first cycle. And a low-cycle processing unit 10 that performs image processing on the two-dimensional image.
The high-cycle processing unit 9 calculates the gravity center position of the article O included in each two-dimensional image sent in the first period, and the displacement of the gravity center position of the article O in the two-dimensional image acquired at different times. Based on the above, the conveying speed of the article O by the conveyor 2 is calculated.

  The low-cycle processing unit 10 performs image processing such as pattern matching on the two-dimensional image output from the two-dimensional camera 4 so as to detect the position and orientation of the article O included in the two-dimensional image. It has become.

  The control unit 6 detects the conveyance speed of the article O by the conveyor 2 calculated from the position of the same article O recognized by the high cycle processing unit 9 based on the images acquired at different times, and detects by the low cycle processing unit 10. A drive signal (drive command) for controlling the robot 3 is generated based on the position and orientation of the article O that has been set.

For example, the control unit 6 uses the conveyance speed of the article O by the conveyor 2 calculated by the high cycle processing unit 9 from the time when the position and orientation of the article O are detected by the low cycle processing unit 10, and the control unit 6 The amount of movement is sequentially accumulated to determine the current position of the article O, and a drive signal corresponding to the current position of the article O is output to the robot 3.
If the article O is temporarily not recognized, the moving amount of the conveyor 2 is calculated using the transport speed of the article O by the immediately preceding conveyor 2.

  The control unit 6 operates the robot 3 in accordance with an operation program taught in advance, and performs tracking to follow the article O on the conveyor 2 based on the conveyance speed calculated in the high cycle processing unit 9. The posture of the robot hand 8 is changed according to the position and posture of the article O detected by the processing unit 10, and the article O moving by the conveyor 2 is grasped by the robot hand 8 and picked up from the conveyor 2 is performed. It comes to let you.

  More specifically, as shown in FIG. 3, for example, the high cycle processing unit 9 acquires three images in the same field of view at different times t1, t2, and t3 with a time interval Δt of the first cycle. When this is done, simple image processing with a low processing load (for example, center of gravity position detection processing or blob detection processing) is performed to detect the position of the article O, and the coordinate positions d1, d2, and the center of gravity of the detected article O are detected. d3 is calculated. In the figure, the speed V is the conveying speed of the article O.

  Then, the high cycle processing unit 9 recognizes the article O having the center of gravity arranged at the same coordinate in the direction orthogonal to the transport direction in the image acquired adjacent to the time axis direction as the same article O. The conveyance speed is calculated by dividing the difference between the coordinate values in the conveyance direction of the center of gravity of each article O by the photographing time interval Δt. When the conveyance speed is calculated a plurality of times for the same article O, an average value or a value fitted by the least square method or the like is output as the conveyance speed.

  In this case, the high-cycle processing unit 9 demarcates a partial region that includes any one of the articles O in the center-of-gravity position detection process of the articles O that is repeatedly performed in the first period, and is sent from the two-dimensional camera 4. The center-of-gravity position is sequentially detected by performing image processing on only a part of the image limited by the partial area, not the entire image to come. Of course, the entire image can be used. In this case, the processing load becomes larger than when a part of the image is used.

  On the other hand, the low-cycle processing unit 10 performs image processing with large load processing on the image sent from the two-dimensional camera 4 (for example, processing for detecting the position and orientation of the article O by pattern matching from the contour of the article O). ) To output the detected position and posture of the article O to the control unit 6.

The operation of the robot system 1 according to this embodiment configured as described above will be described below.
According to the robot system 1 according to the present embodiment, when the article O is conveyed by the conveyor 2, the article O is photographed by the two-dimensional camera 4 in the first period. Images acquired by shooting are sequentially sent to the high cycle processing unit 9 of the image processing unit 5 and also triggered by the low cycle processing unit 10 in response to a trigger output from the low cycle processing unit 10 in the second cycle. The image immediately before (or immediately after) is received is sent.

  In the high-cycle processing unit 9, the article O is recognized by performing image processing with a relatively small processing load on each image sent from the two-dimensional camera 4, and a partial area around the article O is recognized. The coordinate positions d1, d2, d3 of the center of gravity of the article O are detected from the correct image. Then, as shown in FIG. 3, based on the coordinate positions d1, d2, d3 of the center of gravity of the same article O detected from images acquired at different times t1, t2, t3 with a predetermined time interval Δt, In the first period, the conveyance speed of the article O by the conveyor 2 is calculated and input to the control unit 6. Since the two-dimensional camera 4 and the robot 3 are arranged apart from each other by a predetermined distance, the article O moves within the operation range of the robot 3 after a time obtained by dividing the distance by the conveyance speed.

  Further, in the low cycle processing unit 10, for each image sent from the two-dimensional camera 4 in response to the trigger, image processing with large load processing such as pattern matching is performed in the second cycle longer than the first cycle. As a result, the position and posture of the article O are detected from the entire image with high accuracy and input to the control unit 6.

The control unit 6 sets the tracking coordinate system TF by recognizing the position and orientation of the article O at any time when the article O exists in the image, and the conveyor 2 in units of a first period (for example, 1 ms). The amount of movement of the conveyor 2 is obtained from the conveying speed of the article O and is integrated to calculate the amount of movement of the center of gravity position of the article O from the time when the article O is recognized to the current time. Then, the current tracking coordinate system TF ′ is calculated by multiplying the tracking coordinate system TF by a coordinate transformation matrix having the movement amount as a component.
TF '= T · TF
The control unit 6 moves the robot hand 8 following the article O being conveyed by the conveyor 2 with the calculated tracking control system TF ′ as a reference, and matches the recognized position and posture of the article O. The position and posture of the robot hand 8 can be set and the article O can be gripped and picked up from the conveyor 2.

  In this case, when the robot 3 is driven so that the robot hand 8 follows the article O on the conveyor 2, the two-dimensional camera 4 captures the subsequent article O, and a new conveyance is performed by the high cycle processing unit 9. Since the speed is calculated, the control unit 6 controls the robot 3 using the newly calculated transport speed. Thereby, even if the conveyance speed by the conveyor 2 fluctuates, the article O can be picked up correctly.

  Thus, according to the robot system 1 according to the present embodiment, the moving speed of the article O that can be detected by processing with a relatively small processing load is detected with a resolution equivalent to that of the encoder by the first cycle of the high cycle. It is possible to perform the tracking operation of the robot 3 with respect to the article O being conveyed with high accuracy. In addition, the detection of the position and orientation of the article O that requires processing with a relatively large processing load can be detected with high accuracy by performing it in a second period longer than the first period. There is an advantage that the posture can be adjusted more accurately to match the posture of the article O and can be gripped more reliably.

In this manner, since the moving speed, position, and posture of the article O are detected using the single two-dimensional camera 4, there is an advantage that the cost can be reduced.
Further, by using a part of the image sent from the two-dimensional camera 4 in order to detect the moving speed of the article O in the high cycle processing unit 9, the processing load is further reduced, and high cycle processing is performed. Can also be made easier.

  As a result, as shown in parentheses in FIG. 2, it is possible to detect the rough position and posture of the article O in the high cycle processing unit 9. For example, the longitudinal direction of the article O or the rough shape of the blob. Then, the rough position and posture of the article O detected in this way are also input to the control unit 6, so that the control unit 6 sends the high level of the article O sent from the low cycle processing unit 10 in the second cycle. Without waiting for accurate position and orientation information, it is possible to start the operation of the robot 3 based on the rough position and orientation information of the article O sent from the high-cycle processing unit 9 in the first period, There is an advantage that tracking can be made easier.

As a part of the image sent from the two-dimensional camera 4, a partial region including the article O is used. Instead, information is obtained by thinning out pixels from the image sent from the two-dimensional camera 4. The amount may be reduced.
Moreover, in this embodiment, although the two-dimensional camera 4 was illustrated as a visual sensor which acquires visual information, it is not limited to this, You may use a three-dimensional camera and another visual sensor.

In this embodiment, the conveyance speed of the conveyor 2 is calculated based on the change in the position of the article O that has been conveyed by the conveyor 2. Instead, the surface of the conveyor 2 is marked at an appropriate interval. And the conveyance speed may be calculated by changing the position of the mark by recognizing the mark included in the image acquired by the two-dimensional camera 4.
In this case, since the mark is more reliably supplied into the field of view of the two-dimensional camera 4 than the article O, it is useful for calculating the moving speed.

In the present embodiment, the case where a single article O is detected in the image has been described. Instead, as shown in FIG. 4, a plurality of articles a1, a2, and a3 are simultaneously 2 The present invention can also be applied to the case where the dimensional camera 4 is disposed within the field of view.
That is, when a plurality of articles a1, a2, and a3 are recognized in the image, each article a1, a2, and a3 has the same identity as the articles a1, a2, and a3 in the image acquired at different times. The conveyance speed may be calculated by identifying and averaging the speeds V1, V2, and V3 calculated based on the movement distances between the identified same articles a1, a2, and a3.

  In this case, the moving distance between the same articles a1, a2, a3 is the center of gravity of the same articles a1, a2, a3 calculated from images acquired at different times t1, t2, t3 with a predetermined time interval Δt. It is obtained from the difference between the coordinate positions d11, d12, d13, d22, d23, d24, d33, d34.

  Further, in the present embodiment, the case where the image processing unit 5 that processes the image output from the two-dimensional camera 4 is provided as a separate body is exemplified, but instead of this, as shown in FIG. The processing unit 9 may be arranged in the two-dimensional camera 4, and the low cycle processing unit 10 may be arranged in the control unit 6.

  By incorporating the high cycle processing unit 9 in the two-dimensional camera 4, it is possible to prevent the occurrence of communication delay and more reliably detect the conveyance speed of the article O by the conveyor 2 at a high cycle. Since the low-cycle processing unit 10 does not require high-speed processing, it is preferably incorporated in the control unit 6. When the control cycle of the control unit 6 is longer than the first cycle by the high cycle processing unit 9, the center-of-gravity position information of the article O can be transmitted from the high cycle processing unit 9 in a unit collected by the control cycle of the control unit 6. preferable. For example, when the first cycle is 1 ms and the control cycle of the control unit 6 is 8 ms, 8 pieces of gravity center position information for 8 ms of the high cycle processing unit 9 may be sent to the control unit 6 at a time with a cycle of 8 ms.

Moreover, in this embodiment, although the case where the single robot 3 was controlled was illustrated, instead of this, as shown in FIG. 6, a plurality of robots 3 are arranged along the conveying direction of the conveyor 2. These control units 6 may be connected to the host cell control device 11.
When working with articles O conveyed by the same conveyor 2 by a plurality of robots 3, the conveyance speed of the conveyor 2 calculated based on an image taken by one two-dimensional camera 4 is set at one place. Can be managed.

When the transport speed of the conveyor 2 is managed by the control unit 6 of each robot 3, it is necessary to synchronize between the control units 6, and an error may occur due to the influence of communication delay or the like. By doing so, such inconvenience can be prevented.
Further, in order to further suppress the influence of communication delay, an example in which the high cycle processing unit 9 is arranged in the two-dimensional camera 4 and the low cycle processing unit 10 is arranged in the cell control device 11 is shown. It is not limited to. For example, the high cycle processing unit 9 may also be arranged in the cell control device 11.

Further, in the present embodiment, the case where the article O being conveyed by the conveyor 2 is grasped and picked up has been described as an example, but instead, any other processing is performed on the article O being conveyed. You may apply when applying.
Further, as shown in FIG. 7, a trigger is output from the control unit 6 to the two-dimensional camera 4 as necessary, and the two-dimensional camera 4 outputs an image to the control unit 6 in response to the trigger. The image may be confirmed by displaying it on the display unit 12.

  As described above, the control unit 6 sequentially calculates the position of the article O on the conveyor 2 based on the position of the article O detected by the high cycle processing unit 9 at a high cycle, and the article O obtained by the calculation is calculated. A drive signal corresponding to the position may be output to the robot 3. Further, the tracking that follows the article O on the conveyor 2 may be performed based on the position of the article O thus obtained by calculation. Even in this case, the same effect as described above can be achieved.

  Further, the two-dimensional camera 4 may be attached to the tip of the robot 3. In this case, the control unit 6 associates a reference coordinate system, which is a coordinate system used for controlling the robot 3, with the position and orientation (sensor coordinate system) of the two-dimensional camera 4. Thereby, even if the position of the front end portion of the robot 3 changes, the control unit 6 can know the position and orientation of the two-dimensional camera 4, and thereby the article O based on the image obtained by the two-dimensional camera 4. Can be accurately converted into the position and orientation of the object O viewed from the reference coordinate system.

  When the two-dimensional camera 4 is attached to the tip of the robot 3 in this way, the article O that the robot 3 follows in the field of view of the two-dimensional camera 4, the article O in the vicinity of the article O, and the like. Existing. Even in this case, since the position of the article O can be detected at a high cycle by the high cycle processing unit 9, the same effect as described above can be achieved.

  Further, the two-dimensional camera 4 is fixed in the vicinity of the robot 3 by a frame, a stand or the like, and the article O that the robot 3 follows in the field of view of the two-dimensional camera 4, the article O in the vicinity of the article O, etc. You may do it. In this case, the control unit 6 associates the reference coordinate system of the robot 3 with the position and orientation (sensor coordinate system) of the two-dimensional camera 4, and the control unit 6 displays the image obtained by the two-dimensional camera 4. The detection result of the position and orientation of the article O can be converted into the position and orientation of the object O viewed from the reference coordinate system with high accuracy. Even in this case, since the position of the article O can be detected at a high cycle by the high cycle processing unit 9, the same effect as described above can be achieved.

  The embodiment has the single two-dimensional camera 4 for detecting the article O. However, even if another two-dimensional camera for performing inspection, arrival detection, etc. of the article O is provided. The high-cycle processing and the low-cycle processing are possible even when there are a plurality of two-dimensional cameras 4 for detecting the article O.

  Further, instead of the conveyor 2, the article O can be transported by a transport device that moves the article O not only in the X-axis direction but also in the Y-axis direction. The X axis and the Y axis extend in the horizontal direction, and the X axis is orthogonal to the Y axis. Even in this case, the high cycle processing unit 9 can detect the position of the article O at a high cycle, and the high cycle processing unit 9 can transport the article O in the X axis direction and the Y axis direction at a high cycle. Since the speed can be calculated, the same effect as described above can be achieved. The same applies when the conveying device moves the article O in the Z-axis direction.

  Further, the article O may be transported using another robot instead of the conveyor 2. Furthermore, when the article O to be transported is an automobile body or the like, the article O may be transported by an engine, wheels, or the like. Further, the article O may be transported using a shooter in which the article O slides, rolls down, or falls due to gravity. In these cases, other robots, engines, wheels, shooters and the like function as the transfer device.

1 Robot system 2 Conveyor
3 Robot 4 Two-dimensional camera (visual sensor)
6 Control unit 9 High cycle processing unit 10 Low cycle processing unit O, a1, a2, a3 Article (target)

Claims (6)

A transport device for transporting the object;
A robot that performs processing on the object being transported by the transport device;
A visual sensor for obtaining visual information of the object conveyed by the conveying device;
A high-cycle processing unit that processes the visual information acquired by the visual sensor in a first cycle and detects at least one of a conveyance speed and a position of the object being conveyed by the conveyance device;
A low-cycle processing unit that detects the position of the object by processing the visual information acquired by the visual sensor in a second cycle longer than the first cycle;
A control unit that controls the robot based on at least one of the conveyance speed and the position of the target detected by the high cycle processing unit, and the position of the target detected by the low cycle processing unit. Robot system.   The robot system according to claim 1, wherein the high cycle processing unit processes a part of the visual information acquired by the visual sensor in the first cycle.   The robot system according to claim 1, wherein the visual sensor acquires an image of the object on the transport device as the visual information.   The robot system according to claim 1, wherein the transfer device includes a mark that is moved at the same speed as the target.   The robot system according to any one of claims 1 to 4, wherein the robot works while chasing the target being transported by the transport device. The robot system according to any one of claims 1 to 5, wherein the visual sensor outputs the visual information immediately before or immediately after the trigger is received by an external trigger.

Images