JP2016013591A - Robot system - Google Patents

Abstract

A robot system capable of efficiently controlling a robot is provided. A robot system moves a robot using a robot that moves a workpiece, an imaging device that images the workpiece that the robot moves, a control device that operates the robot, and a captured image captured by the imaging device. The image processing apparatus calculates a positional relationship between the first position of the robot and the second position of the work moved by the robot using the captured image. [Selection] Figure 3

Description

  The present invention relates to a robot system.

  Conventionally, a robot system that aligns work objects picked up by a robot at a predetermined position has been researched and developed. In such a robot system, it is necessary to calculate a deviation (offset) of the position and posture of the picked-up work target (hereinafter referred to as a position / posture) with respect to the robot reference posture in order to correctly align the work target at a predetermined position. There is.

  In relation to this, an image processing unit is installed to detect the position and orientation of the work target gripped by the robot, and detects (calculates) the position and orientation of the work target from the captured image captured by the image capturing unit. Is known separately from a control device for controlling a robot (see Patent Document 1).

JP 2012-230041 A

  However, in such a conventional robot system, the control device receives an image processing result from the image processing device, and performs image processing such as the above-described offset, and calculation of the position and orientation of the manipulator and the position and orientation of the gripper in consideration of the offset. The control device was performing the calculation regarding.

  For this reason, it is necessary to embed a program for performing these calculations in the program that defines the robot operation, which increases the load on the user for creating the program. In addition, since the program to be created is different for each control device, the load on the user for creating the program has increased. Due to these loads, it is difficult for the user to improve the work efficiency when using the conventional robot system. In addition, calculations related to image processing are performed not by the image processing apparatus but by the control apparatus. As described above, when the calculation related to the image processing is shared between the control device and the image processing apparatus, and the control device performs the calculation related to the image processing, it is more efficient than the case where the control device only controls the robot. In some cases, the robot could not be controlled.

  Therefore, the present invention has been made in view of the above-described problems of the prior art, and provides a robot system capable of efficiently controlling a robot.

According to one embodiment of the present invention, a robot that moves a workpiece, an imaging device that captures the workpiece that the robot moves, a control device that operates the robot, and a captured image captured by the imaging device are used. An image processing device that detects the workpiece that the robot moves, and the image processing device uses the captured image to determine a first position of the robot and a second position of the workpiece that the robot moves. This is a robot system for calculating the positional relationship.
With this configuration, in the robot system, the image processing apparatus calculates the positional relationship between the first position of the robot and the second position of the workpiece moved by the robot using the captured image. Thereby, the robot system can control the robot efficiently.

According to another aspect of the present invention, in the robot system, the robot includes a hand, moves the workpiece gripped by the hand, and the image processing apparatus determines a predetermined position of the hand from the captured image. A configuration for detecting the first position may be used.
With this configuration, the robot system detects a predetermined position of the hand from the captured image as the first position. Thereby, the robot system can calculate the positional relationship between the predetermined position of the hand and the second position of the workpiece.

According to another aspect of the present invention, in the robot system, the image processing device detects the positional relationship between the first position of the robot and the second position of the workpiece from the captured image by pattern matching. .
With this configuration, the robot system detects the positional relationship between the first position of the robot and the second position of the workpiece from the captured image by pattern matching. Thereby, the robot system can calculate the positional relationship between the first position of the robot and the second position of the workpiece from the captured image by pattern matching.

  As described above, in the robot system, the image processing apparatus calculates the positional relationship between the first position of the robot and the second position of the workpiece moved by the robot using the captured image. As a result, the robot system has a high degree of versatility related to the replacement of the device provided, and can reduce the labor involved in the replacement.

It is a figure which shows the structure of the robot system 1 which concerns on this embodiment. 2 is a diagram illustrating an example of a hardware configuration of an image processing apparatus 40. FIG. 3 is a diagram illustrating an example of a functional configuration of an image processing device 40. FIG. It is a flowchart which shows an example of the flow of a process in which the image processing apparatus 40 calculates an offset. 3 is a diagram illustrating an example of a state in which an imaging unit 10 captures an imaging range and an example of a captured image captured by the imaging unit 10. FIG.

<Embodiment>
Hereinafter, embodiments of the present invention will be described with reference to the drawings. FIG. 1 is a diagram illustrating a configuration of a robot system 1 according to the present embodiment. The robot system 1 includes an imaging unit 10, a single-arm robot 20 including a gripping unit HND (end effector) and a manipulator MNP, a control device 30, an image processing device 40, and an information processing terminal 50. In the present embodiment, a single arm robot refers to a robot having one arm composed of a gripping unit HND and a manipulator MNP.

  The robot system 1 images a range including the work object O gripped by the robot 20 by the imaging unit 10. The work object O is an object that can be gripped by the robot 20, for example, industrial parts such as screws, bolts, gears, and jigs, but is not limited thereto, and may be any other object that can be gripped by the robot 20. It may be an object. The work target O is an example of a work. The robot system 1 has a relative position and a relative posture between the control point TCP and the representative point OP set in advance on the work object O gripped by the robot 20 based on a captured image in which the range including the work object O is captured ( Hereinafter, the offset is calculated.

  The control point TCP is a point representing the position and orientation of the gripper HND when the control device 30 moves the gripper HND (and the manipulator MNP) of the robot 20, and in this example, the control point TCP of the manipulator MNP provided with the gripper HND. This is the center point (tool center point) of the flange provided at the end. The control point TCP is an example of the first position. The control point TCP is an example of a predetermined position of the hand. The representative point OP is a point representing the position and orientation of the work target O set in advance in the control device 30, and is a characteristic point of the work target O (in this example, the angle OP shown in FIG. 1). The representative point OP is an example of a second position. Based on the calculated offset, the robot system 1 matches the position and orientation of the representative point OP (hereinafter referred to as the work target position and orientation) with the position and orientation of the target point TP (hereinafter referred to as the target point position and orientation). As described above, the control point TCP is moved by the manipulator MNP and the gripper HND of the robot 20. Thereby, the robot system 1 arranges the work object O in the target area TA with the correct position and orientation by the robot 20. Hereinafter, for convenience of explanation, the position and orientation of the control point TCP will be described as the control point position and orientation.

  Here, the target point TP is a point in the target area TA where the work target O is placed by the robot 20, and the representative point OP coincides when the work target O is placed in the target area TA in the correct position and orientation. It is. The target area TA is an area in which the work target O is arranged by the robot 20 and is provided on the table TB in FIG. The table TB is, for example, a table on which the robot 20 places the work target O.

  The imaging unit 10 is a camera including, for example, a CCD (Charge Coupled Device), a CMOS (Complementary Metal Oxide Semiconductor), or the like, which is an imaging element that converts collected light into an electrical signal. Moreover, although the imaging part 10 is a monocular camera, it may be comprised by 2 or more cameras like a stereo camera, for example.

  The imaging unit 10 is communicably connected to the image processing device 40 via a cable. Wired communication via a cable is performed according to standards such as Ethernet (registered trademark) and USB (Universal Serial Bus), for example. The imaging unit 10 and the image processing apparatus 40 may be configured to be connected by wireless communication performed according to a communication standard such as Wi-Fi (registered trademark).

  When the robot 20 moves the work object O to a predetermined image pickup position by the manipulator MNP and the gripping part HND, the imaging unit 10 is configured to include the control point TCP and the representative point OP of the work target O gripped by the gripping part HND. A range including the following (hereinafter referred to as an imaging range) is installed at a position where imaging is possible. Moreover, in FIG. 1, the imaging part 10 is installed in the position which can image the above-mentioned imaging range from the vertically downward direction to the vertically upward direction.

  Note that the imaging unit 10 is configured to be installed at a position where the imaging range can be imaged in the horizontal direction, instead of being configured at a position where the imaging range can be imaged from vertically downward to vertically upward. It may be configured to be installed at a position where the imaging range can be imaged from vertically upward to vertically downward, or may be configured to be able to image the imaging range from other directions. . Further, although the imaging unit 10 is configured to capture a still image as a captured image, it may be configured to capture a moving image as a captured image instead. The imaging unit 10 is an example of an imaging device.

  The robot 20 is, for example, a single-arm robot including a gripping unit HND, a manipulator MNP, and a plurality of actuators (not shown). The robot 20 may be a scalar robot (horizontal articulated robot), a double-arm robot, or the like instead of the single-arm robot. A scalar robot is a robot in which the manipulator moves only in the horizontal direction and only the slide shaft at the tip of the manipulator moves up and down. Further, the double-armed robot refers to a robot having two arms each constituted by a gripper HND and a manipulator MNP.

  The arm of the robot 20 is a 6-axis vertical articulated type, and can operate with 6 degrees of freedom by an operation in which the support base, the manipulator MNP, and the gripper HND are linked by an actuator. The arm of the robot 20 may operate with 5 degrees of freedom (5 axes) or less, or may operate with 7 degrees of freedom (7 axes) or more. Below, the operation | movement of the robot 20 performed with the arm provided with the holding part HND and the manipulator MNP is demonstrated. Note that the grip portion HND of the robot 20 includes a claw portion that can grip an object. The gripping part HND is an example of a hand.

  The robot 20 is communicably connected to the control device 30 by a cable, for example. Wired communication via a cable is performed according to standards such as Ethernet (registered trademark) and USB, for example. The robot 20 and the control device 30 may be connected by wireless communication performed according to a communication standard such as Wi-Fi (registered trademark). Further, in the robot system 1, the robot 20 is configured to be connected to the control device 30 installed outside the robot 20 as shown in FIG. The structure built in 20 may be sufficient.

  The robot 20 acquires a control signal from the control device 30, and moves the work target O gripped by the gripping unit HND of the robot 20 from the current position to a predetermined imaging position by the manipulator MNP based on the acquired control signal. Further, the robot 20 acquires a control signal from the control device 30, and moves the work target O from the imaging position by the manipulator MNP based on the acquired control signal, and arranges it in the target area TA.

  The control device 30 controls the robot 20 by outputting a control signal to the robot 20. The control device 30 controls the robot 20 to move the work target O to the imaging position after the robot 20 grips the work target O. Then, the control device 30 causes the image processing device 40 to calculate the above-described offset. When the control device 30 acquires the information indicating the offset from the image processing device 40, the control device 30 is based on the acquired information indicating the offset and the target point position and orientation registered in advance in the control device 30 via the information processing terminal 50. Then, a control point position / posture (hereinafter referred to as a control point target position / posture) when the representative point position / posture related to the work target O coincides with the target point position / posture is calculated. After calculating the control point target position and orientation, the control device 30 moves the control point TCP to the robot 20 so that the control point position and orientation coincide with the control point target position and orientation.

  The robot system 1 may be configured such that the image processing device 40 calculates the control point target position and orientation instead of the configuration in which the control device 30 calculates the control point target position and orientation. In this case, the control device 30 outputs information indicating the target point position and orientation registered in advance in the control device 30 to the image processing device 40. Then, the image processing device 40 calculates the control point target position and orientation based on the target point position and orientation acquired from the control device 30 and the offset calculated by the own device, and the calculated control point target position and orientation is calculated. The information shown is output to the control device 30.

  In response to a request from the control device 30, the image processing device 40 causes the imaging unit 10 to capture the above-described imaging range. Further, when the captured image is acquired from the imaging unit 10, the image processing device 40 calculates the above-described offset based on the acquired captured image. The image processing device 40 outputs information indicating the calculated offset to the control device 30.

  The information processing terminal 50 registers (inputs) various information in the control device 30. The information processing terminal 50 is a notebook PC (Personal Computer), but instead is a desktop PC, a tablet PC, a mobile phone terminal, a multi-function mobile phone terminal (smartphone), a PDA (Personal Digital Assistant), or the like. Also good. The information processing terminal 50 outputs and registers (stores) the information indicating the target point position and orientation described above in the image processing apparatus 40. The information processing terminal 50 may be configured as an apparatus integrated with the image processing apparatus 40.

  Next, the hardware configuration of the image processing apparatus 40 will be described with reference to FIG. FIG. 2 is a diagram illustrating an example of a hardware configuration of the image processing apparatus 40. The image processing device 40 includes, for example, a CPU (Central Processing Unit) 41, a storage unit 42, an input reception unit 43, and a communication unit 44, and communicates with the control device 30 via the communication unit 44. These components are connected to each other via a bus Bus so that they can communicate with each other. The CPU 41 executes various programs stored in the storage unit 42.

  The storage unit 42 includes, for example, an HDD (Hard Disk Drive), an SSD (Solid State Drive), an EEPROM (Electrically Erasable Programmable Read-Only Memory), a ROM (Read-Only Memory), a RAM (Random Access Memory), and the like. Various information, images, and programs processed by the image processing apparatus 40 are stored. The storage unit 42 may be an external storage device connected by a digital input / output port such as a USB instead of the one built in the image processing device 40.

The input receiving unit 43 is, for example, a keyboard, mouse, touch pad, or other input device. In addition, the input reception part 43 may be comprised as a touch panel by functioning as a display part. Further, the input receiving unit 43 may be configured not to be provided in the image processing apparatus 40. In this case, input to the image processing device 40 is accepted from either one or both of the control device 30 and the information processing terminal 50.
The communication unit 44 includes, for example, a digital input / output port such as USB, an Ethernet port, and the like.

  Next, the functional configuration of the image processing apparatus 40 will be described with reference to FIG. FIG. 3 is a diagram illustrating an example of a functional configuration of the image processing apparatus 40. The image processing device 40 includes a storage unit 42, a communication unit 44, an image acquisition unit 45, and a control unit 46. Part or all of the functional units included in the control unit 46 is realized by, for example, the CPU 41 executing various programs stored in the storage unit 42. Some or all of these functional units may be hardware functional units such as LSI (Large Scale Integration) and ASIC (Application Specific Integrated Circuit).

The image acquisition unit 45 acquires a captured image from the imaging unit 10. The image acquisition unit 45 outputs the acquired captured image to the control unit 46.
The control unit 46 includes an imaging control unit 47, an offset calculation unit 48, and a communication control unit 49.
The imaging control unit 47 controls the imaging unit 10 so as to capture a captured image.
The offset calculation unit 48 calculates an offset between the control point position and orientation and the work target position and orientation based on the captured image acquired by the image acquisition unit 45.
The communication control unit 49 controls the communication unit 44 so as to output information indicating the offset calculated by the offset calculation unit 48 to the control device 30.

  Hereinafter, with reference to FIG. 4, processing in which the image processing apparatus 40 calculates an offset will be described. FIG. 4 is a flowchart illustrating an example of a process flow in which the image processing apparatus 40 calculates an offset. Hereinafter, a process after the work object O is moved to the imaging position by the robot 20 will be described. First, the imaging control unit 47 controls the imaging unit 10 so as to capture the above-described imaging range (step S100). Next, the image acquisition unit 45 acquires the captured image captured in step S100 from the imaging unit 10, and outputs the acquired captured image to the control unit 46 (step S110).

  Next, the offset calculation unit 48 calculates an offset between the control point position and orientation and the work target position and orientation based on the captured image acquired from the image acquisition unit 45 (step S120). Here, the calculation of the offset between the control point position and orientation based on the captured image and the work target position and orientation will be described with reference to FIG. FIG. 5 is a diagram illustrating an example of how the imaging unit 10 captures an imaging range and an example of a captured image captured by the imaging unit 10.

  FIG. 5A shows an example of how the imaging unit 10 captures an imaging range. In FIG. 5A, description will be made assuming that the work object O gripped by the gripper HND is in a state of being moved to the imaging position by the robot 20. After the work target O is moved to the imaging position, the imaging unit 10 images the imaging range in the direction C illustrated in FIG. The direction C is a direction orthogonal to the surface M of the work target O held by the holding unit HND. The surface M indicates a surface that does not face the flange F provided with the gripping portion HND when the work object O is gripped by the gripping portion HND. Therefore, the imaging unit 10 images the imaging range in a state where the surface M is parallel to the plane of the imaging element.

  The reason why the surface M is in a state parallel to the plane of the image sensor is an offset when the work object O is arranged in the correct position and orientation in the target area TA, and the control point TCP and the representative point OP are the target area TA. This is to detect an offset between the control point position and orientation on the target area surface (that is, on the plane of the table TB) when projected onto the target area surface. By treating the offset as an offset on the target area surface in this way, the calculation cost of the control device 30 and the image processing device 40 can be suppressed. Note that such offset handling is merely an example, and other offset handling methods may be used.

  FIG. 5B illustrates an example of a captured image captured by the imaging unit 10. As shown in FIG. 5B, the surface M is captured from directly below the captured image. The captured image includes a representative point OP related to the work object O. The image processing apparatus 40 causes the imaging unit 10 to capture a captured image as illustrated in FIG. 5B, and sets an offset OS between the control point position and orientation and the work target position and orientation based on the captured image. calculate.

  In FIG. 5B, the offset OS indicates a relative position and a relative posture on the XY plane stretched by the X axis and the Y axis on the captured image. Note that the image processing apparatus 40 detects the position of the control point TCP (that is, the center position of the flange F) on the captured image by detecting the shape of the gripper HND from the captured image by pattern matching or the like, for example. For example, the image processing apparatus 40 detects the attitude of the control point TCP on the captured image based on the direction in which the manipulator MNP (not shown) extends and the manipulator MNP included in the captured image extends. The image processing apparatus 40 may be configured to detect the attitude of the control point TCP by another method when detecting the attitude of the control point TCP.

  Further, the image processing device 40 detects the position of the representative point OP related to the work target O on the captured image by pattern matching, for example. Further, the image processing device 40 detects, for example, the direction of the side of the surface M on the captured image by pattern matching, and detects the orientation of the representative point OP on the captured image based on the detected direction. Note that the image processing device 40 may be configured to detect the posture of the representative point OP by another method when detecting the posture of the representative point OP. The image processing device 40 calculates an offset OS in the robot coordinate system based on the detected control point position and orientation on the captured image and the work target position and orientation on the captured image. Note that the coordinate (pixel coordinate) system on the captured image and the robot coordinate system are associated in advance by calibration or the like.

Next, the communication control unit 49 causes the communication unit 44 to output information indicating the offset between the control point position and orientation calculated by the offset calculation unit 48 and the work target position and orientation in step S120 to the control device 30. Control (step S130).
As described above, the robot system 1 causes the image processing device 40 separate from the control device 30 to perform image processing for calculating the offset between the control point position and orientation and the work target position and orientation. Thereby, the robot system 1 can suppress loads such as calculations related to processing by the control device 30. Further, the robot system 1 can suppress the number of communications performed by the control device 30.

  In addition, the robot system 1 can control the robot 20 even when the control device 30 that controls the robot 20 includes a new control device X that is different from the control device 30 due to version upgrade or the like. There is no need to create a new program relating to image processing for calculating an offset, which is an executable program. In this case, the robot system 1 simply calculates an offset simply by writing a code related to input / output of information with the image processing device 40 into a program for controlling the robot 20 of the control device X, and based on the calculated offset. Thus, the robot 20 can be controlled.

  As described above, in the robot system 1 according to the present embodiment, the image processing apparatus 40 uses the captured image captured by the imaging unit 10 to use the offset OS between the control point position / posture and the work target position / posture. Is calculated. Thereby, the robot system 1 can control the robot 20 efficiently.

The robot system 1 detects the control point position and orientation from the captured image. Thereby, the robot system can calculate the offset OS between the control point position and orientation and the work target position and orientation.
Further, the robot system 1 detects an offset OS between the control point position / posture and the work target position / posture from the captured image by pattern matching. Thereby, the robot system 1 can detect the offset OS between the control point position / posture and the work target position / posture from the captured image by pattern matching.

  The embodiment of the present invention has been described in detail with reference to the drawings. However, the specific configuration is not limited to this embodiment, and changes, substitutions, deletions, and the like are possible without departing from the gist of the present invention. May be.

  Further, a program for realizing the function of an arbitrary component in the apparatus described above (for example, the image processing apparatus 40 of the robot system 1) is recorded on a computer-readable recording medium, and the program is stored in the computer system. You may make it read and execute. Here, the “computer system” includes hardware such as an OS (Operating System) and peripheral devices. “Computer-readable recording medium” means a portable disk such as a flexible disk, a magneto-optical disk, a ROM (Read Only Memory), a CD (Compact Disk) -ROM, or a hard disk built in a computer system. Refers to the device. Further, the “computer-readable recording medium” means a volatile memory (RAM: Random Access) inside a computer system that becomes a server or a client when a program is transmitted via a network such as the Internet or a communication line such as a telephone line. Memory that holds a program for a certain period of time, such as Memory).

In addition, the above program may be transmitted from a computer system storing the program in a storage device or the like to another computer system via a transmission medium or by a transmission wave in the transmission medium. Here, the “transmission medium” for transmitting the program refers to a medium having a function of transmitting information, such as a network (communication network) such as the Internet or a communication line (communication line) such as a telephone line.
Further, the above program may be for realizing a part of the functions described above. Further, the program may be a so-called difference file (difference program) that can realize the above-described functions in combination with a program already recorded in the computer system.

DESCRIPTION OF SYMBOLS 1 Robot system, 10 Imaging part, 20 Robot, 30 Control apparatus, 40 Image processing apparatus, 41 CPU, 42 Storage part, 43 Input reception part, 44 Communication part, 45 Image acquisition part, 46 Control part, 47 Imaging control part, 48 offset calculation unit, 49 communication control unit, 50 information processing terminal

Claims (3)

A robot that moves the workpiece,
An imaging device for imaging the workpiece moved by the robot;
A control device for operating the robot;
An image processing device that detects the workpiece moved by the robot using a captured image captured by the imaging device;
Including
The image processing apparatus includes:
Calculating a positional relationship between a first position of the robot and a second position of the workpiece moved by the robot, using the captured image;
Robot system. The robot system according to claim 1,
The robot includes a hand, moves the workpiece gripped by the hand,
The image processing device detects a predetermined position of the hand as the first position from the captured image;
Robot system. The robot system according to claim 1 or 2,
The image processing device detects the positional relationship between the first position of the robot and the second position of the workpiece from the captured image by pattern matching;
Robot system.

Images