CN116157837A - Calibration method and device for robot - Google Patents

Abstract

A calibration method for a robot, comprising performing a camera calibration process, wherein performing the camera calibration process comprises the steps of: capturing an image of an environment and forming a three-dimensional virtual object of the environment (301); placing a calibration object (403) in a target area (302) in the environment based on the captured image of the environment and the three-dimensional virtual object of the environment; capturing an image (303) of a calibration object (403) using a first camera (102) by relative movement between the first camera (102) and the calibration object (403) associated with the robot (101), wherein the first camera (102) is a 2D camera; based on the image of the calibration object (403) captured by the first camera (102), parameters (304) of the first camera (102) for calibration are determined. The calibration method can guide the user (107) to calibrate the camera through the AR technology, so that the operation is visual and convenient, the operation complexity is reduced, and the accuracy is improved. And also relates to a calibration device for the robot.

Description

Calibration method and device for robot technical field

The present disclosure relates to the field of machine vision, and more particularly, to a calibration method, device, computing device, computer-readable storage medium and program product for a robot.

Background technique

With the development of robot technology, more and more robot workstations are used in industrial applications, such as automatic loading and unloading, welding, stamping, spraying, and other various processes. Robots can be flexibly combined with different equipment to meet tough production process requirements. It can easily realize multi-machine linkage automatic production line and digital factory layout, save manpower to the greatest extent and improve enterprise efficiency.

Despite the many advantages of robotics in industrial applications, the highly technical and specialized requirements for integration, operation and maintenance in robotic applications limit its use. In particular, calibration including vision system calibration (camera calibration), TCP (tool center point) calibration, and hand-eye calibration has become a major challenge for users using robots.

Contents of the invention

The calibration of the robot is one of the key technologies in the process of robot operation. However, for ordinary users, the existing calibration methods require the user to have considerable professional knowledge and spend a lot of time and energy on the calibration of the robot to determine the operation of the robot. Whether it is ideal or not, this increases the user's operating threshold. For example, the existing calibration methods require a lot of manual operations, the calibration efficiency is low and the operation process is complicated, and highly dependent on the operator's subjective judgment and experience.

The first embodiment of the present disclosure proposes a calibration method for a robot, the calibration method includes performing a camera calibration process, wherein performing the camera calibration process includes the following steps: capturing an image of an environment and forming the environment the three-dimensional virtual object of the environment; based on the captured image of the environment and the three-dimensional virtual object of the environment, place the calibration object in the target area in the environment; through the first camera associated with the robot and the calibration relative movement between objects to capture an image of the calibration object using the first camera, wherein the first camera is a 2D camera; and based on the image of the calibration object captured by the first camera, determine The parameters of the first camera used for calibration.

In this embodiment, the augmented reality (AR) technology can be used to guide the user to perform camera calibration, which makes the operation intuitive and convenient, advantageously reduces the operation complexity and improves the accuracy, thereby effectively reducing the user's operation threshold, making Camera calibration can be easily implemented by users without professional knowledge.

A second embodiment of the present disclosure provides a calibration device for a robot, the calibration device includes a camera calibration unit, the camera calibration unit includes: an environment capture module configured to capture an image of the environment and form the environment The three-dimensional virtual object of the environment; the placement module is configured to place the calibration object in the target area in the environment based on the captured image of the environment and the three-dimensional virtual object of the environment; the first camera capture module is configured to Configured to use the first camera associated with the robot to capture an image of the calibration object through relative movement between the first camera and the calibration object, wherein the first camera is a 2D camera; a parameter determination module , configured to determine parameters of the first camera used for calibration based on the image of the calibration object captured by the first camera.

A third embodiment of the present disclosure provides a computing device including: a processor; and a memory for storing computer-executable instructions that, when executed, cause the processor to The method described in the first embodiment is carried out.

A fourth embodiment of the present disclosure proposes a computer-readable storage medium having computer-executable instructions stored thereon for executing the computer-executable instructions described in the first embodiment. described method.

A fifth embodiment of the present disclosure proposes a computer program product tangibly stored on a computer-readable storage medium and comprising computer-executable instructions that, when executed, use At least one processor executes the method described in the first embodiment.

Description of drawings

The features, advantages and other aspects of the various embodiments of the present disclosure will become more apparent with reference to the following detailed description in conjunction with the accompanying drawings, which show several embodiments of the present disclosure by way of illustration and not limitation. In the attached picture:

Fig. 1 shows an exemplary scenario in which embodiments of the present disclosure may be applied.

Fig. 2 illustrates another exemplary scenario in which embodiments of the present disclosure may be applied.

Fig. 3 shows a flowchart of a calibration method for a robot according to an embodiment of the present disclosure.

Fig. 4 shows an exemplary arrangement of a camera calibration process usable for a robot according to an embodiment of the present disclosure.

Fig. 5 shows another flowchart of a calibration method for a robot according to an embodiment of the present disclosure.

FIG. 6 illustrates exemplary calibration artifacts for TCP calibration according to an embodiment of the disclosure.

Fig. 7 shows an exemplary arrangement of a TCP calibration process usable for a robot according to an embodiment of the present disclosure.

Fig. 8 shows another flowchart of a calibration method for a robot according to an embodiment of the present disclosure.

Fig. 9 shows an exemplary arrangement of a hand-eye calibration process that may be used for a robot according to an embodiment of the present disclosure.

Fig. 10 shows a block diagram of an exemplary calibration device for a robot according to an embodiment of the present disclosure.

Figure 11 shows a block diagram of an exemplary computing device for robot calibration according to an embodiment of the disclosure.

Detailed ways

Various exemplary embodiments of the present disclosure are described in detail below with reference to the accompanying drawings. While the example methods, apparatus described below include software and/or firmware executing on hardware, among other components, it should be noted that these examples are illustrative only and should not be viewed as limiting. For example, it is contemplated that any or all hardware, software, and firmware components may be implemented exclusively in hardware, exclusively in software, or in any combination of hardware and software. Therefore, although exemplary methods and apparatuses have been described below, those skilled in the art will readily understand that the examples provided are not intended to limit the manner in which these methods and apparatuses are implemented.

Furthermore, the flowchart and block diagrams in the Figures illustrate the architecture, functionality, and operation of possible implementations of methods and systems according to various embodiments of the present disclosure. It should be noted that the functions noted in the blocks may occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently, or they may sometimes be executed in the reverse order, depending upon the functionality involved. It should also be noted that each block in the flowchart and/or block diagrams, and combinations of blocks in the flowchart and/or block diagrams, can be implemented using a dedicated hardware-based system that performs the specified functions or operations , or can be implemented using a combination of dedicated hardware and computer instructions. In the different drawings, the same reference numerals indicate the same or similar elements.

The terms "including", "comprising" and similar terms used herein are open-ended terms, that is, "including/including but not limited to", which means that other contents may also be included. The term "based on" is "based at least in part on". The term "one embodiment" means "at least one embodiment"; the term "another embodiment" means "at least one further embodiment" and so on.

This article involves various coordinate systems of the robot system, such as the base coordinate system of the robot, the tool coordinate system, and the camera coordinate system. The base coordinate system of the robot is the coordinate system used to describe the motion of the robot body based on the robot installation base. The tool coordinate system TCS is a coordinate system established with the tool center point (TCP) as the origin. Before the tool is not assembled, the default TCP is the center point of the robot end flange. After the tool is assembled, the robot TCP moves to the end of the tool. The camera coordinate system is the coordinate system used to describe the motion of the object based on the camera.

FIG. 1 illustrates an exemplary scenario 100 in which embodiments of the present disclosure may be applied. The scene 100 includes a robot 101 and an associated first camera 102 fixed to the robot 101 . For example, the robot 101 may be a multi-joint manipulator or a multi-degree-of-freedom machine device for the industrial field. The robot 101 includes a movable end 103 , the first camera 102 can be fixed on the end 103 and move together with the movement of the end 103 of the robot 101 . That is to say, in this scene 100, since the end of the robot 101 and the first camera 102 are fixed together, it is called eye in hand (eye in hand). The end 103 of the robot 101 may also have a tool portion 104 for assembling (eg, absorbing, inserting, etc.) tools or components (eg, welding guns, nozzles, bolts, etc.) for processing target workpieces. The scene 100 is also arranged with target objects 105 . The target object 105 may be an actual object to be processed (eg, a target workpiece) or a calibration object (eg, a calibration plate, etc.). The first camera 102 is a 2D camera, and a second camera 106 is also arranged in the scene 100 . The second camera 106 is a 3D camera, which can be a binocular camera, a structured light camera, or any camera capable of returning depth information. In the scene 100, a second camera 106 may be disposed relatively above the robot 101 to capture and track movements of the robot 101 (eg, its extremity, etc.) and the like. In scene 100, user 107 may be carrying (e.g., handheld, worn on head, etc.) AR device 108, which may include, but is not limited to, a smartphone, tablet, teach pendant, or headset. equipment. The AR device 108 can take pictures of the real environment or objects in the environment from different positions and different angles within its field of view 109 through multiple cameras arranged on or outside the AR device 108 to form a three-dimensional virtual object. The AR device 108 may have a display to display in the virtual environment corresponding three-dimensional virtual objects of the real environment or objects in the environment.

FIG. 2 illustrates an exemplary scenario 200 in which embodiments of the present disclosure may be applied. The scene 200 is similar to the scene 100 except that the associated first camera 102 is fixed outside the robot 101 . That is to say, in this scene 200, the first camera 102 is located outside the robot 101 and does not move together with the movement of the end 103 of the robot 101, which is called eye to hand.

FIG. 3 shows a flowchart of a calibration method 300 for a robot according to an embodiment of the present disclosure, and FIG. 4 shows an exemplary arrangement 400 of a camera calibration process for a robot according to an embodiment of the present disclosure. The method 300 can be applied to the exemplary scene 100 (eye in hand) shown in FIG. 1 and the exemplary scene 200 (eye out of hand) shown in FIG. 2 . The method 300 is described below in conjunction with FIG. 3 and FIG. 4 .

Referring to FIG. 3, method 300 includes steps 301-304 for performing a camera calibration process. In addition, the method 300 may also include steps for performing a TCP calibration process (see FIG. 5 ) and for performing a hand-eye calibration process (see FIG. 8 ).

In the process of image measurement and machine vision applications, in order to determine the relationship between the three-dimensional geometric position of a point on the surface of a space object and its corresponding point in the image, it is necessary to establish a geometric model of camera imaging, and these geometric model parameters are camera parameters (such as , internal and external parameters of the camera, distortion parameters). These parameters are usually obtained through experiments and calculations, and the process of solving the parameters is called camera calibration. By determining camera parameters through camera calibration, lens distortion can be corrected, corrected images can be generated, and 3D scenes can be reconstructed from the obtained images.

Method 300 begins at step 301 by capturing an image of an environment and forming a three-dimensional virtual object of the environment.

In some embodiments, step 301 may include: taking pictures of the environment from different positions and different angles, so as to obtain at least two images for each scene; The depth information of each pixel and the three-dimensional virtual object of the environment are formed based on the depth information and the two-dimensional information contained in the image. For example, the AR device held by the user 107 may be equipped with or connected to multiple cameras that take pictures of the real environment from different positions and different angles. Taking two cameras as an example, when the two cameras take photos of the same scene from different positions, due to the different shooting angles of the two cameras, it can be based on the different positions of the pixels in the two images of the same scene, Triangulates the depth of each pixel in the 2D image. Therefore, although the image captured by a single camera is two-dimensional, the three-dimensional information required to form a three-dimensional virtual object in a real environment can be obtained by supplementing the depth information obtained through triangulation on this basis.

Next, the method 300 proceeds to step 302 . In step 302, based on the captured image of the environment and the three-dimensional virtual object of the environment, a calibration object is placed in a target area in the environment. For example, the target area can be a specific area in the workbench.

In some embodiments, step 302 may include: detecting visual markers disposed in the environment from the captured image of the environment; determining a target area based on the detected visual markers; capturing an image of the calibration object and forming the calibration object superimposing the 3D virtual object of the calibration object on the 3D virtual object of the environment to display as an augmented reality image; and moving the calibration object so that the 3D virtual object of the calibration object is located in the 3D virtual object of the target area. For example, a target area in an environment may be defined by a plurality of identifiable visual markers, and the target area can be quickly determined by detecting the visual markers. A plurality of visual markers may be arranged to overlay within the field of view of the first camera for capture by the first camera. Similarly, the AR device 108 held by the user 107 can capture images of the calibration object through multiple cameras and form a three-dimensional virtual object of the calibration object. When the 3D virtual object of the calibration object is superimposed on the 3D virtual object of the environment to be displayed as an augmented reality image (for example, real-time tracking display), the user 107 or the robot 101 can be guided to move and calibrate through voice, text, and images, for example. object. For example, when the 3D virtual object of the marked object is at least partially located outside the 3D virtual object of the target area, the AR device 108 may prompt the user 107 or guide the robot 101 to continue moving the marked object until the 3D virtual object of the marked object is located outside the 3D virtual object of the target area. in the dummy object.

Referring to FIG. 4 , an exemplary arrangement 400 includes a plurality of visual markers 402 (eg, four shown) defining a target area 401 . Visual markers 402 may have a characteristic color (eg, a different color from the rest of the environment) and/or a characteristic pattern (eg, a specific graphic code (barcode, QR code, etc.), etc.). For example, the AR device 108 may capture an image of the environment through a camera, and recognize the visual marker 402 from the image, so as to determine the target area 401 defined by the visual marker 402 . The exemplary arrangement 400 also includes a calibration object 403 (e.g. a calibration plate) that has been placed in the target area 401. The calibration object 403 may have a reference pattern attached to the surface for camera calibration, the reference pattern includes but not limited to a checkerboard pattern, a circular array pattern, a non-circular array pattern, and the like.

Next, the method 300 proceeds to step 303 . In step 303, the first camera associated with the robot is used to capture an image of the calibration object through relative movement between the first camera and the calibration object, wherein the first camera is a 2D camera.

In some embodiments, when the first camera is fixed outside the robot, step 303 may include: obtaining multiple actual positions for placing the calibration object in the target area, the multiple actual positions corresponding to the three-dimensional virtual objects in the environment multiple virtual positions; moving the calibration object so that the three-dimensional virtual objects of the calibration object are respectively located at the multiple virtual positions; and using the first camera to capture images of the calibration object at the multiple actual positions respectively. For example, when the first camera 102 is fixed outside the robot 101 (that is, the eyes are outside the hand), it is necessary to place the calibration object 403 at a suitable position in the target area 401 so that the corner points on the calibration object 403 are covered by the first camera. 102, and the correlation between camera parameters and different poses of the calibration object 403 will affect the determination of camera-specific parameters. In an example, based on factors such as whether the corner point of the calibration object 403 is covered in the field of view of the first camera 102, the correlation between camera parameters, different poses, etc., the decision to place the calibration object 403 in the target area 401 can be calculated. Multiple actual positions, and the user 107 or the robot 101 is designated by the AR device 108 to place the calibration object 403 at the calculated multiple actual positions. Similar to step 302, the calibration object 403 can be placed at multiple actual positions by determining whether the 3D virtual object of the calibration object 403 is placed at multiple virtual positions corresponding to the multiple actual positions, and using the first camera 102 to respectively Images of the calibration object 403 are captured at multiple actual locations.

In some embodiments, when the first camera is fixed on the robot, in step 303, the first camera can be used to capture images of the calibration object at multiple actual locations by moving the first camera around the calibration object to multiple actual locations. .

Next, the method 300 proceeds to step 304 . In step 304, parameters of the first camera used for calibration are determined based on the image of the calibration object captured by the first camera. As mentioned above, the parameters of the first camera may include at least one of internal parameters, external parameters, and distortion parameters. In one example, the classic Zhang Zhengyou calibration method (see "A Flexible New Technique for Camera Calibration", IEEE TRANSACTIONS ANALYSIS AND MACHINE INTELLIGENCE, Volume 22, Issue 11, 2000) can be used to determine the parameters of the first camera, the used The number of checkerboard grid points is p x q. Every time the calibration board is placed, the corresponding camera parameters will change, and the feature points (for example, the corner points of the checkerboard pattern) in the captured calibration board images are detected to form p x q equations. After placing the calibration board n times, n x p x q equations are formed to estimate the internal parameters and all external parameters in the ideal case of no distortion, and then apply the least square method to estimate the distortion coefficient under the actual radial distortion, and finally use the maximum likelihood method , Optimize the estimation to improve the estimation accuracy. In other examples, the parameters of the first camera may also be determined according to any applicable existing camera calibration method, which will not be described in detail.

Compared with the traditional camera calibration process, the camera calibration process according to the method 300 can provide guidance to the user based on AR technology during camera calibration, making the operation intuitive and convenient, advantageously reducing the operation complexity and improving the accuracy, thus effectively The user's operation threshold is lowered, so that users without professional knowledge can easily achieve camera calibration.

FIG. 5 shows a flowchart of a calibration method 500 for a robot according to an embodiment of the present disclosure, FIG. 6 shows an exemplary calibration workpiece 600 for TCP calibration according to an embodiment of the present disclosure, and FIG. 7 shows An exemplary arrangement 700 of a TCP calibration process usable for a robot according to an embodiment of the present disclosure is shown. The method 500 can be applied to the exemplary scene 100 (eye in hand) shown in FIG. 1 and the exemplary scene 200 (eye out of hand) shown in FIG. 2 . The method 500 is described below in conjunction with FIG. 5 , FIG. 6 and FIG. 7 .

Referring to FIG. 5, method 500 includes steps 501-504 for performing a TCP calibration procedure. Method 500 may be included in method 300 or performed independently of method 300 .

As previously described, the tool portion 104 on the tip 103 of the robot 101 may be equipped with tools to process a target workpiece. In order to describe the pose of the tool in space, a coordinate system is bound (defined) on the tool, that is, the tool coordinate system TCS, and the origin of the tool coordinate system TCS is TCP. Before the tool is assembled, the default TCP is located on the end 103 of the robot 101 , such as the center point of the flange of the tool part 104 . For example, the coordinates of the center point of the robot end flange can be obtained from the robot controller. Alternatively, the coordinates of the center point of the robot end flange can be obtained from the teaching pendant, wherein the teaching pendant is a hand-held device for manual manipulation, program writing, parameter configuration and monitoring of the robot. After the tool is assembled, the robot TCP (eg, on the tool tip) needs to be calibrated and the origin of the tool coordinate system TCS is set from the default TCP to the robot TCP.

The method 500 starts at step 501, using a second camera to respectively capture a plurality of images of a first workpiece placed in a plurality of poses, wherein the plurality of poses have different spatial inclination angles, and the first workpiece is mounted on a robot On the end, the second camera is a 3D camera and is fixed outside the robot. For example, the first workpiece may be, for example, a calibration workpiece 600 as shown in FIGS. 6 and 7 for simulating an actual tool used. A side view 601 and a top view 602 , respectively, of a calibration workpiece 600 are shown in FIG. 6 , the calibration workpiece having an apex 611 and an end 612 . The calibration workpiece 600 may be mounted to the tool portion 104 of the tip 103 of the robot 101 via the tip 612 . Referring to FIG. 7, an exemplary arrangement 700 includes a second camera 106 and a calibration artifact 600 placed in a plurality of different poses (e.g., four poses 701, 702, 703, 704 in FIG. 7), each pose having Different spatial inclination angles.

Next, the method 500 proceeds to step 502 . In step 502, a plurality of workpiece end coordinates of the end of the first workpiece in a 3D camera coordinate system of the second camera in a plurality of poses are determined based on the captured images. As mentioned before, before assembling the tool, the default TCP is the center point of the robot end flange, and after assembling the tool/workpiece, the robot TCP (for example, located on the end of the tool/workpiece) needs to be calibrated. In this step, the position of the workpiece end in the 3D camera coordinate system is determined using the 3D camera as a reference. In some embodiments, the number of poses may be at least four.

In some embodiments, step 502 may include: detecting the marker points arranged on the first workpiece from the multiple captured images to obtain a plurality of marker point coordinates of the marker points in the 3D camera coordinate system; The coordinates of the marking points are used to determine the end coordinates of the plurality of workpieces in the 3D camera coordinate system of the end of the first workpiece. As shown in FIG. 6, the calibration workpiece 600 may have one or more marking points 613 (for example, 4 in FIG. 613 and end 612) are overlaid in the field of view of the second camera 106. After identifying the marker point 613 in the captured image, multiple marker point coordinates of the marker point in the 3D camera coordinate system under different poses can be obtained. By calibrating the rigid relationship between the marking point 613 on the workpiece 600 and the end 612 (for example, the marking point is rigidly connected to the robot TCP, so the distance between each other is determined), the 3D camera coordinate system of the end 612 can be obtained The corresponding multiple workpiece end coordinates in .

In some embodiments, the marker points may have a characteristic color and/or a characteristic pattern. For example, marking points 613 may have a characteristic color (eg, a different color from the rest of the calibration workpiece 600) and/or a characteristic pattern (eg, a specific graphic code (barcode, QR code, etc.), etc.) for identification/detection.

Next, the method 500 proceeds to step 503 . In step 503, a plurality of end coordinates of the robot in the base coordinate system of the robot in the plurality of poses are obtained. For example, a plurality of robot end coordinates in different poses of the robot's end in the robot's base coordinate system can be obtained from the robot controller or a teach pendant.

Next, the method 500 proceeds to step 504 . In step 504, based on a plurality of workpiece end coordinates and a plurality of robot end coordinates, determine the coordinates of the robot TCP to be calibrated under a posture of the robot in the base coordinate system of the robot, and determine whether the robot TCP is located on the end of the robot The relative relationship between the default TCP. An exemplary TCP calibration process is described below in conjunction with FIG. 7 .

Referring to FIG. 7 , the robot 101 poses in four different poses, the second camera 106 recognizes the marking point 613 on the calibration workpiece 600 , obtains the coordinates of the marking point 613 in the 3D camera coordinate system, and deduces the end of the calibration workpiece 600 612 the coordinates in the 3D camera coordinate system, while obtaining the coordinates of the center point of the end flange of the robot 101 from, for example, the robot controller. The following coordinates can be obtained:

In the first pose 701 : coordinates (x1, y1, z1) of the robot end in the robot base coordinate system, and calibrate the coordinates (mx1, my1, mz1) of the workpiece end in the 3D camera coordinate system.

In the second pose 702 : coordinates (x2, y2, z2) of the robot end in the robot base coordinate system, and calibrate the coordinates (mx2, my2, mz2) of the workpiece end in the 3D camera coordinate system.

In the third pose 703 : coordinates (x3, y3, z3) of the robot end in the robot base coordinate system, and calibrate the coordinates (mx3, my3, mz3) of the workpiece end in the 3D camera coordinate system.

In the fourth pose 704 : coordinates (x4, y4, z4) of the end of the robot in the robot base coordinate system, and coordinates (mx3, my3, mz3) of the end of the workpiece in the 3D camera coordinate system.

Mathematically translate the above coordinates so that the coordinates of the end of the calibration workpiece in the 3D camera coordinate system are the same. The following coordinates can be obtained:

In the first pose 701 : coordinates (x1, y1, z1) of the robot end in the robot base coordinate system, and calibrate the coordinates (mx1, my1, mz1) of the workpiece end in the 3D camera coordinate system.

In the second pose 702: the coordinates of the end of the robot in the robot base coordinate system (x2-mx2+mx1, y2-my2+my1, z2-mz2+mz1), and the coordinates of the end of the workpiece in the 3D camera coordinate system (mx1, my1, mz1).

In the third pose 703: the coordinates of the end of the robot in the robot base coordinate system (x3-mx3+mx1, y3-my3+my1, z3-mz3+mz1), and the coordinates of the end of the workpiece in the 3D camera coordinate system (mx1, my1, mz1).

In the fourth pose 704: coordinates of the end of the robot in the robot base coordinate system (x4-mx4+mx1, y4-my4+my1, z4-mz4+mz1), calibrate the coordinates of the end of the workpiece in the 3D camera coordinate system (mx1, my1, mz1).

After the translation transformation, the coordinates (x0, y0, z0) of the robot TCP in the robot base coordinate system are at the center of the sphere, and the default TCP point (the center coordinate point of the end flange is obtained from the teach pendant) is located on the sphere, which can be calculated according to the following equation (1)-(4) to solve the coordinates (x0, y0, z0) of the robot TCP:

(x1-x0) ² +(y1-y0) ² +(z1-z0) ² ＝R ² (1)

(x2-mx2+mx1-x0) ² +(y2-my2+my1-y0) ² (2)

+(z2-mz2+mz1-z0) ² ＝R2

(x3-mx3+mx1-x0) ² +(y3-my3+my1,-y0) ² (3)

+(z3-mz3+mz1-z0) ² ＝R ²

(x4-mx4+mx1-x0) ² +(y4-my4+my1-y0) ² (4)

+(z4-mz4+mz1-z0) ² ＝R ²

The TCP value (x _tcp , y _tcp , z _tcp , rx _tcp , ry _tcp , rz _tcp ), that is, the relative relationship between the robot TCP and the default TCP located on the end of the robot can be determined.

The traditional TCP calibration process usually uses 3, 4 or 5 points method, the user needs to use different poses to move the TCP to a reference point (for example, a fixed point placed within the robot's workspace) 3, 4 or 5 times to make the TCP coincident with the reference point. However, such a traditional TCP calibration method requires manual participation of the user, requires the user to be familiar with the operation and professional knowledge, and has the disadvantages of slow calibration speed and insufficient calibration accuracy. Compared with the traditional TCP calibration process, according to the TCP calibration process of method 500, TCP calibration can be performed automatically without manual participation, which effectively reduces the operation complexity, can quickly realize TCP calibration, and avoids errors caused by manual participation , which improves the calibration accuracy.

FIG. 8 shows another flowchart of a calibration method 800 for a robot according to an embodiment of the present disclosure, and FIG. 9 shows an exemplary arrangement 900 of a hand-eye calibration process for a robot according to an embodiment of the present disclosure. The method 800 can be applied to the exemplary scene 100 (eye in hand) shown in FIG. 1 and the exemplary scene 200 (eye out of hand) shown in FIG. 2 . The method 800 is described below in conjunction with FIG. 8 and FIG. 9 .

Referring to FIG. 8, method 800 includes steps 801-805 for performing a hand-eye calibration process. Method 800 may be included in method 300 or performed independently of method 300 .

The purpose of hand-eye calibration is to obtain the relationship between the robot coordinate system and the camera coordinate system, and finally transfer the results of visual recognition to the robot coordinate system. As mentioned earlier, depending on where the camera is fixed, hand-eye calibration can be divided into two forms. If the camera and the end of the robot are fixed together, it is called eye-in-hand; if the camera is fixed on the outer base of the robot, it is called The eye is out of the hand.

Method 800 begins at step 801 by using a first camera to capture a first target image of a second workpiece to be processed by a robot and using a second camera to capture a second target image of the second workpiece. The second workpiece may be, for example, a target workpiece 901 to be processed by the robot 101 as shown in FIG. 9 . A first target image of target workpiece 901 may be captured by first camera 102 and a second target image of target workpiece 901 may be captured by second camera 106 .

Method 800 then proceeds to step 802 . In step 802, based on the first target image and the second target image, a first transformation relationship between the 2D camera coordinate system of the first camera and the 3D camera coordinate system of the second camera is determined. In this step, the transformation relationship between different camera coordinate systems can be determined, that is, the mapping relationship between the coordinates of points in the first target image and the coordinates of corresponding points in the second target image can be determined.

In some embodiments, step 802 may include: detecting a plurality of feature points arranged in the second workpiece from the first target image and the second target image, so as to obtain the first position of the plurality of feature points in the 2D camera coordinate system A plurality of feature coordinates and a second plurality of feature coordinates in the 3D camera coordinate system. In some embodiments, the number of feature points may be at least four. For example, the coordinates of the 2D camera coordinate system of the first camera 102 may be based on camera parameters obtained after camera calibration of the first camera 102 (the camera calibration may be, for example, the camera calibration process described herein or any other applicable camera calibration process ), calculated using the two-dimensional information in the target image.

Referring to FIG. 9, a plurality of feature points 902 (for example, 4 in the figure) can be arranged on the target workpiece 901, and the feature points 902 can have, for example, specific shapes, sizes, marks, etc., so as to be recognized/detected from captured images Feature points. For example, the target workpiece 901 may be a PCB board, and the feature points 902 may be CAD-marked screw holes arranged on the PCB board. The first camera 102 and the second camera 106 can respectively identify the feature points 902 from the captured target image, and the multiple feature points 902 in the 2D camera coordinate system of the first camera 102 and the 3D camera coordinate system of the second camera 106 can be obtained as follows Coordinates in:

The first feature point 902, 3D camera coordinate system coordinates (x1, y1, z1), 2D camera coordinate system coordinates (X1, Y1, Z1).

The second feature point 902, 3D camera coordinate system coordinates (x2, y2, z2), 2D camera coordinate system coordinates (X2, Y2, Z2).

The third feature point 903, the coordinates of the 3D camera coordinate system (x3, y3, z3), and the coordinates of the 2D camera coordinate system (X3, Y3, Z3).

The fourth feature point 902, the coordinates of the 3D camera coordinate system (x4, y4, z4), and the coordinates of the 2D camera coordinate system (X4, Y4, Z4).

The coordinate transformation relationship in the 2D camera and 3D camera coordinate system satisfies the following equation (5):

According to P _3D = H1·P _2D , where P _3D and P _2D are the coordinates of the same point in different camera coordinate systems, the homography matrix H1 between the two camera coordinate systems can be obtained, that is, the first A first transformation relationship between the 2D camera coordinate system of the camera 102 and the 3D camera coordinate system of the second camera 106 .

Then the method 800 proceeds to step 803 . In step 803, the tip of the robot is moved to a plurality of positions, a first plurality of tool coordinates of the robot TCP in the robot's base coordinate system is determined, and a second plurality of tool coordinates of the robot TCP in the 3D camera coordinate system is determined. In this step, a first plurality of tool coordinates of the robot TCP in the robot's base coordinate system can be determined, for example, by capturing an image of the tip of the robot with a second camera (e.g., using the TCP calibration procedure described herein or any other applicable TCP calibration procedure) and the second plurality of tool coordinates in the 3D camera coordinate system. In some embodiments, the number of locations may be at least four.

For example, by moving the end 103 of the robot (for example, translation, or posing different poses) to multiple positions (for example, four), respectively determine the base coordinate system of the robot TCP in the robot 101 and the camera of the second camera 106 Coordinates in the coordinate system.

In some embodiments, step 803 may include: for at least one of the plurality of positions, translating to the at least one position in the one posture of the robot, based on the robot TCP and the default The relative relationship between TCPs determines at least one tool coordinate of the robot TCP in the robot's base coordinate system. For example, in order to facilitate calculation, based on the relative relationship between the robot TCP and the default TCP obtained under a certain posture of the robot, the end of the robot can be translated to at least one position under the posture of the robot, according to the slave robot control Determine the coordinates of the robot TCP in the robot base coordinate system by using the default TCP coordinates obtained by the robot in the robot base coordinate system.

Method 800 then proceeds to step 804 . In step 804, a second transformation relationship between the base coordinate system of the robot and the 3D camera coordinate system is determined using the first plurality of tool coordinates and the second plurality of tool coordinates.

The coordinate transformation relationship between the base coordinate system of the robot 101 and the 3D camera coordinate system satisfies the following equation (6):

P _3D ＝H2·P _robot (6)

P _3D and P _robot are the coordinates of the same point in the 3D camera coordinate system and the robot base coordinate system respectively, thus the homography matrix H2 between the 3D camera coordinate system and the robot base coordinate system can be obtained, that is, the robot’s The second transformation relationship between the base coordinate system and the 3D camera coordinate system.

The method 800 then proceeds to step 805 . In step 805, based on the first transformation relationship and the second transformation relationship, a third transformation relationship between the robot's base coordinate system and the 2D camera coordinate system is determined.

Based on equations (5) and (6), the coordinate transformation relationship between the robot base coordinate system and the 3D camera coordinate system satisfies the following equation (7):

P _robot ＝ ^H2-1 ·H1·P _2D (7)

P _robot and P _2D are the coordinates of the same point in the robot base coordinate system and the 2D camera coordinate system respectively, so the homography matrix H2 ^-1 H1 between the robot base coordinate system and the 2D camera coordinate system can be obtained, namely , determine the third transformation relationship (hand-eye relationship) between the base coordinate system of the robot and the 2D camera coordinate system.

In some embodiments, the at least one location is at or near the at least one feature point. For example, in order to reduce hand-eye calibration errors, the at least one position may be at or near at least one identifiable feature point. In some embodiments, the AR technology as previously described herein can be used to intuitively and conveniently guide the end of the robot to move to or near at least one recognizable feature point, for example, insert the end 612 of the calibration workpiece 600 into the feature point 902 to further reduce the hand-eye calibration error.

Compared with the traditional hand-eye calibration process, according to the hand-eye calibration process of method 800, the hand-eye calibration can be performed automatically without manual participation, effectively reducing the complexity of the operation, and the calculation is simple, which can quickly realize the hand-eye calibration and avoid the need for manual calibration. Participate in the error caused by it, and improve the calibration accuracy.

FIG. 10 shows a block diagram of an exemplary calibration device 1000 for a robot according to an embodiment of the present disclosure. The device 1000 includes a camera calibration unit 1010, a TCP calibration unit 1020 and a hand-eye calibration unit 1030. The device 1000 also includes a communication unit (not shown) to communicate with other external devices (eg, receive/transmit instructions and data from and/or to the external device).

The camera unit 1010 includes an environment capture module 1011 , a placement module 1012 , a first camera capture module 1013 and a parameter determination module 1014 .

The environment capture module 1011 is configured to: capture an image of the environment and form a three-dimensional virtual object of the environment. In some embodiments, the environment capture module 1011 can be further configured to: take pictures of the environment from different positions and different angles, so as to obtain at least two images for each scene; Depth information of each pixel in the image; and forming a three-dimensional virtual object of the environment based on the depth information and the two-dimensional information contained in the image.

The placing module 1012 is configured to: place the calibration object in the target area in the environment based on the captured image of the environment and the three-dimensional virtual object of the environment. In some embodiments, the placement module 1012 may be further configured to: detect visual markers placed in the environment from the captured image of the environment; determine the target area based on the detected visual markers; and forming a three-dimensional virtual object of the calibration object; superimposing the three-dimensional virtual object of the calibration object on the three-dimensional virtual object of the environment to be displayed as an augmented reality image; moving the calibration object so that the three-dimensional virtual object of the calibration object is located at the in the 3D virtual object of the target area. In some embodiments, visual markers have a characteristic color and/or pattern.

The first camera capture module 1013 is configured to: use the first camera to capture an image of the calibration object through relative movement between the first camera associated with the robot and the calibration object, wherein the first camera is a 2D camera. In some embodiments, when the first camera is fixed outside the robot, the first camera capture module 1013 can be further configured to: obtain multiple actual positions for placing the calibration object in the target area, the multiple actual positions correspond to multiple virtual locations in the 3D virtual object of the environment; moving the calibration object so that the 3D virtual object of the calibration object is respectively located at multiple virtual locations; and using the first camera to capture images of the calibration object at multiple actual locations respectively.

The parameter determination module 1014 is configured to: determine the parameters of the first camera used for calibration based on the image of the calibration object captured by the first camera.

The TCP calibration unit 1020 includes a first workpiece capture module 1021 , a workpiece coordinate determination module 1022 , a robot coordinate acquisition module 1023 and a TCP coordinate determination module 1024 .

The first workpiece capture module 1021 is configured to: use a second camera to respectively capture multiple images of a first workpiece placed in multiple poses, wherein the first workpiece is mounted on the end of the robot, and the second camera is The 3D camera is fixed on the outside of the robot.

The workpiece coordinate determination module 1022 is configured to: based on the captured images, determine a plurality of workpiece end coordinates of the end of the first workpiece in the 3D camera coordinate system of the second camera in the plurality of poses. In some embodiments, the workpiece coordinate determination module 1022 may be configured to: detect the marker points arranged on the first workpiece from the multiple captured images to obtain a plurality of marker point coordinates of the marker points in the 3D camera coordinate system ; and based on the plurality of marker point coordinates, determining a plurality of workpiece end coordinates of the end of the first workpiece in the 3D camera coordinate system. In some embodiments, the marker points have a characteristic color and/or a characteristic pattern.

The robot coordinate acquiring module 1023 is configured to: acquire a plurality of coordinates of the robot end in the base coordinate system of the robot in the plurality of poses.

The TCP coordinate determining module 1024 is configured to: based on a plurality of workpiece end coordinates and a plurality of robot end coordinates, determine the coordinates of the robot TCP to be calibrated in the robot's base coordinate system in one posture of the robot, and determine the relationship between the robot TCP and the robot located at The relative relationship between the default TCPs on the end of the robot.

The hand-eye marking unit 1030 includes a second workpiece capture module 1031 , a first transformation determination module 1032 , a tool coordinate determination module 1033 , a second transformation determination module 1034 and a third transformation determination module 1035 .

The second workpiece capturing module 1031 is configured to capture a first target image of a second workpiece to be processed by the robot using the first camera, and capture a second target image of the second workpiece using the second camera.

The first transformation determination module 1032 is configured to determine a first transformation relationship between the 2D camera coordinate system of the first camera and the 3D camera coordinate system of the second camera based on the first target image and the second target image. In some embodiments, the first transformation determination module 1032 may be further configured to: detect a plurality of feature points arranged in the second workpiece from the first target image and the second target image, so as to obtain a plurality of feature points in 2D a first plurality of feature coordinates in a camera coordinate system and a second plurality of feature coordinates in a 3D camera coordinate system; and using the first plurality of feature coordinates and the second plurality of feature coordinates to determine the first A transformation relationship.

The tool coordinate determination module 1033 is configured to: move the tip of the robot to a plurality of positions, determine a first plurality of tool coordinates of the robot TCP in the robot's base coordinate system, and determine a second plurality of tool coordinates of the robot TCP in the 3D camera coordinate system tool coordinates. In some embodiments, the tool coordinate determination module 1033 can be further configured to: for at least one position among the plurality of positions, translate to the at least one position under a pose of the robot, based on the relative relationship between the robot TCP and the default TCP relationship, determining at least one tool coordinate of the robot TCP in the base coordinate system of the robot. In some embodiments, the at least one location is at or near at least one feature point.

The second transformation determination module 1034 is configured to: use the first plurality of tool coordinates and the second plurality of tool coordinates to determine a second transformation relationship between the base coordinate system of the robot and the 3D camera coordinate system.

The third transformation determination module 1035 is configured to: determine a third transformation relationship between the base coordinate system of the robot and the 2D camera coordinate system based on the first transformation relationship and the second transformation relationship.

Although the camera calibration unit 1010, the TCP calibration unit 1020, and the hand-eye calibration unit 1030 are shown as being integrated in the device 1000 in the example of FIG. Calibration process.

FIG. 11 shows a block diagram of an exemplary computing device 1100 for enabling industrial camera focusing, according to an embodiment of the disclosure. The computing device 1100 includes a processor 1101 and a memory 1102 coupled with the processor 1101 . The memory 1102 is used for storing computer-executable instructions, and when the computer-executable instructions are executed, the processor 1101 executes the methods in the above embodiments (for example, any one or more steps in the aforementioned methods 300, 500 or 800).

In addition, alternatively, the above method can be implemented by a computer-readable storage medium. A computer-readable storage medium carries computer-readable program instructions for implementing various embodiments of the present disclosure. A computer readable storage medium may be a tangible device that can retain and store instructions for use by an instruction execution device. A computer readable storage medium may be, for example, but is not limited to, an electrical storage device, a magnetic storage device, an optical storage device, an electromagnetic storage device, a semiconductor storage device, or any suitable combination of the foregoing. More specific examples (a non-exhaustive list) of computer-readable storage media include: portable computer diskettes, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM), or flash memory), static random access memory (SRAM), compact disc read only memory (CD-ROM), digital versatile disc (DVD), memory stick, floppy disk, mechanically encoded device, such as a printer with instructions stored thereon A hole card or a raised structure in a groove, and any suitable combination of the above. As used herein, computer-readable storage media are not to be construed as transient signals per se, such as radio waves or other freely propagating electromagnetic waves, electromagnetic waves propagating through waveguides or other transmission media (e.g., pulses of light through fiber optic cables), or transmitted electrical signals.

Accordingly, in another embodiment, the present disclosure provides a computer-readable storage medium having computer-executable instructions stored thereon for performing various implementations of the present disclosure. method in the example.

In another embodiment, the present disclosure provides a computer program product tangibly stored on a computer-readable storage medium and comprising computer-executable instructions that, when executed, cause At least one processor executes the methods in various embodiments of the present disclosure.

In general, the various example embodiments of the present disclosure may be implemented in hardware or special purpose circuits, software, firmware, logic, or any combination thereof. Certain aspects may be implemented in hardware, while other aspects may be implemented in firmware or software, which may be executed by a controller, microprocessor or other computing device. When aspects of the embodiments of the present disclosure are illustrated or described as block diagrams, flowcharts, or using some other graphical representation, it is to be understood that the blocks, devices, systems, techniques, or methods described herein may serve as non-limiting Examples are implemented in hardware, software, firmware, special purpose circuits or logic, general purpose hardware or controllers or other computing devices, or some combination thereof.

The computer-readable program instructions or computer program products used to execute various embodiments of the present disclosure can also be stored in the cloud, and when called, the user can access the program stored on the cloud for execution through the mobile Internet, fixed network or other networks. The computer-readable program instructions of an embodiment of the present disclosure implement the technical solutions disclosed in accordance with various embodiments of the present disclosure.

While embodiments of the present disclosure have been described with reference to several specific embodiments, it is to be understood that the embodiments of the present disclosure are not limited to the specific embodiments disclosed. Embodiments of the present disclosure are intended to cover various modifications and equivalent arrangements included within the spirit and scope of the appended claims. The scope of the claims is accorded the broadest interpretation to encompass all such modifications and equivalent structures and functions.

Claims (27)

A calibration method for a robot, the calibration method includes performing a camera calibration process, wherein performing the camera calibration process includes the following steps: A, capturing an image of an environment and forming a three-dimensional virtual object of said environment, B, placing a calibration object in a target area in the environment based on the captured image of the environment and the three-dimensional virtual object of the environment, C, using the first camera associated with the robot to capture an image of the calibration object through relative movement between the first camera and the calibration object, wherein the first camera is a 2D camera, and D. Determine parameters of the first camera used for calibration based on the image of the calibration object captured by the first camera. The calibration method according to claim 1, wherein said step A comprises: A1, taking pictures of the environment from different positions and different angles, so as to obtain at least two images for each scene, A2, measure the depth information of each pixel in the image based on the triangulation method of the stereo image, and A3, forming a three-dimensional virtual object of the environment based on the depth information and the two-dimensional information contained in the image. The calibration method according to claim 1, wherein said step B comprises: B1, detecting from a captured image of the environment visual markers arranged in said environment, B2, determining the target area based on the detected visual marker, B3, capturing the image of the calibration object and forming a three-dimensional virtual object of the calibration object, B4, superimposing the 3D virtual object of the calibration object on the 3D virtual object of the environment to display as an augmented reality image, and B5. Moving the calibration object so that the 3D virtual object of the calibration object is located in the 3D virtual object of the target area. The marking method according to claim 3, wherein the visual marker has a characteristic color and/or a characteristic pattern. The calibration method according to claim 3, wherein, when the first camera is fixed outside the robot, the step C comprises: C1, obtaining multiple actual positions for placing the calibration object in the target area, the multiple actual positions corresponding to multiple virtual positions in the three-dimensional virtual object of the environment, C2, moving the calibration object so that the three-dimensional virtual objects of the calibration object are respectively located at the plurality of virtual positions, and C3. Using the first camera to respectively capture images of the calibration object at the multiple actual positions. The calibration method according to claim 1, wherein the calibration method further comprises executing a TCP calibration process, wherein performing the TCP calibration process comprises the following steps: E, using a second camera to respectively capture a plurality of images of the first workpiece placed in a plurality of poses, wherein the plurality of poses have different spatial inclination angles, and the first workpiece is mounted on the on the end of the robot, the second camera is a 3D camera and is fixed outside the robot, F, based on the multiple captured images, determine a plurality of workpiece end coordinates of the end of the first workpiece in the 3D camera coordinate system of the second camera in multiple poses, G, obtaining a plurality of robot end coordinates of the end of the robot in the base coordinate system of the robot under the plurality of poses, and H, based on the plurality of workpiece end coordinates and the plurality of robot end coordinates, determine the coordinates of the robot TCP to be calibrated in the robot's base coordinate system in a posture of the robot, and determine the robot The relative relationship between the TCP and the default TCP located on the end of the robot. The calibration method according to claim 6, wherein said step F comprises: F1, detecting the marking points arranged on the first workpiece from the multiple captured images to obtain a plurality of marking point coordinates of the marking points in the 3D camera coordinate system, and F2, based on the coordinates of the plurality of marker points, determine a plurality of workpiece end coordinates of the end of the first workpiece in the 3D camera coordinate system. The calibration method according to claim 7, wherein the marking points have characteristic colors and/or characteristic patterns. According to the calibration method according to claim 6, the calibration method also includes performing a hand-eye calibration process, wherein performing the hand-eye calibration process includes the following steps: I, using the first camera to capture a first target image of a second workpiece to be processed by the robot, and using the second camera to capture a second target image of the second workpiece, J. Based on the first target image and the second target image, determine a first transformation relationship between the 2D camera coordinate system of the first camera and the 3D camera coordinate system of the second camera, K, move the tip of the robot to a plurality of positions, determine the first plurality of tool coordinates of the robot TCP in the robot's base coordinate system, and determine the first plurality of tool coordinates of the robot TCP in the 3D camera coordinate system the second plurality of tool coordinates, L, using the first plurality of tool coordinates and the second plurality of tool coordinates to determine a second transformation relationship between a base coordinate system of the robot and the 3D camera coordinate system, and M. Determine a third transformation relationship between a base coordinate system of the robot and the 2D camera coordinate system based on the first transformation relationship and the second transformation relationship. The calibration method according to claim 9, wherein said step J comprises: J1, detecting a plurality of feature points arranged in the second workpiece from the first target image and the second target image, so as to obtain the first position of the plurality of feature points in the 2D camera coordinate system a plurality of feature coordinates and a second plurality of feature coordinates in said 3D camera coordinate system, and J2. Using the first plurality of feature coordinates and the second plurality of feature coordinates to determine the first transformation relationship. The calibration method according to claim 10, wherein said step K comprises: For at least one of the plurality of positions, the robot is translated to the at least one position under the one attitude of the robot, and based on the relative relationship between the robot TCP and the default TCP, the robot is determined to be At least one tool coordinate of the TCP in the robot's base coordinate system. The calibration method according to claim 10, wherein the at least one position is at or near the at least one feature point. A calibration device for a robot, the calibration device includes a camera calibration unit, and the camera calibration unit includes: an environment capture module configured to capture images of the environment and form a three-dimensional virtual object of the environment, a placement module configured to place a calibration object in a target area in the environment based on the captured image of the environment and the three-dimensional virtual object of the environment, The first camera capture module is configured to use the first camera associated with the robot to capture an image of the calibration object through relative movement between the first camera and the calibration object, wherein the first camera for the 2D camera, and The parameter determination module is configured to determine parameters of the first camera used for calibration based on the image of the calibration object captured by the first camera. The calibration device according to claim 13, wherein the environment capture module is further configured to: taking pictures of the environment from different positions and different angles, thereby obtaining at least two images for each scene, Depth information of each pixel in the image is measured based on a triangulation method of the stereo image, and A three-dimensional virtual object of the environment is formed based on the depth information and the two-dimensional information included in the image. The calibration device according to claim 13, wherein the placement module is further configured to: detecting visual markers arranged in the environment from the captured image of the environment, determining said region of interest based on said detected visual marker, capturing an image of the calibration object and forming a three-dimensional virtual object of the calibration object, superimposing the 3D virtual object of the calibration object on the 3D virtual object of the environment for display as an augmented reality image, and The marking object is moved so that the 3D virtual object of the marking object is located in the 3D virtual object of the target area. The marking device according to claim 13, wherein the visual marker has a characteristic color and/or a characteristic pattern. The calibration device according to claim 13, wherein when the first camera is fixed outside the robot, the first camera capture module is further configured to: obtaining a plurality of actual positions for placing the calibration object in the target area, the plurality of actual positions corresponding to a plurality of virtual positions in a three-dimensional virtual object of the environment, moving the calibration object so that the three-dimensional virtual objects of the calibration object are respectively located at the plurality of virtual positions, and Images of the calibration object are respectively captured at the plurality of actual locations using the first camera. The calibration device according to claim 13, the calibration device further comprising a TCP calibration unit, the TCP calibration unit comprising: The first workpiece capture module is configured to use a second camera to respectively capture a plurality of images of the first workpiece placed in a plurality of poses, wherein the first workpiece is mounted on an end of the robot, and The second camera is a 3D camera and is fixed outside the robot, a workpiece coordinate determination module configured to determine a plurality of workpiece end coordinates of the end of the first workpiece in the 3D camera coordinate system of the second camera in a plurality of poses based on the captured images, a robot coordinate acquiring module configured to acquire a plurality of robot end coordinates in which the end of the robot is in the base coordinate system of the robot under the plurality of poses, and The TCP coordinate determination module is configured to determine the coordinates of the robot TCP to be calibrated in the robot's base coordinate system in a posture of the robot based on the plurality of workpiece end coordinates and the plurality of robot end coordinates , and determine the relative relationship between the robot TCP and the default TCP located on the end of the robot. The calibration device according to claim 18, wherein the workpiece coordinate determination module is further configured to: detecting a marker point arranged on the first workpiece from a plurality of captured images to obtain a plurality of marker point coordinates of the marker point in a 3D camera coordinate system, and Based on the plurality of marker point coordinates, determine a plurality of workpiece end coordinates of the end of the first workpiece in a 3D camera coordinate system. The marking device according to claim 19, wherein the marking points have characteristic colors and/or characteristic patterns. The calibration device according to claim 18, said calibration device further comprising a hand-eye calibration unit, said hand-eye calibration unit comprising: A second workpiece capture module configured to use the first camera to capture a first target image of a second workpiece to be processed by the robot, and use the second camera to capture a second image of the second workpiece. target image, A first transformation determination module configured to determine a second transformation between the 2D camera coordinate system of the first camera and the 3D camera coordinate system of the second camera based on the first target image and the second target image a transformation relationship, a tool coordinate determination module configured to move the tip of the robot to a plurality of positions, determine a first plurality of tool coordinates of the robot TCP in a base coordinate system of the robot, and determine a position of the robot TCP in the the second plurality of tool coordinates in the 3D camera coordinate system, A second transformation determination module configured to use the first plurality of tool coordinates and the second plurality of tool coordinates to determine a second transformation relationship between a base coordinate system of the robot and the 3D camera coordinate system ,as well as The third transformation determination module is configured to determine a third transformation relationship between the base coordinate system of the robot and the 2D camera coordinate system based on the first transformation relationship and the second transformation relationship. The calibration device according to claim 21, wherein the first transformation determination module is further configured to: Detecting a plurality of feature points arranged in the second workpiece from the first target image and the second target image, to obtain a first plurality of feature points in the 2D camera coordinate system feature coordinates and a second plurality of feature coordinates in said 3D camera coordinate system, and The first transformation relationship is determined using the first plurality of feature coordinates and the second plurality of feature coordinates. The calibration device according to claim 22, wherein the tool coordinate determination module is further configured to: For at least one of the plurality of positions, the robot is translated to the at least one position under the one attitude of the robot, and based on the relative relationship between the robot TCP and the default TCP, the robot is determined to be At least one tool coordinate of the TCP in the robot's base coordinate system. The indexing device according to claim 23, wherein said at least one location is at or near said at least one feature point. Computing equipment, said computing equipment comprising: processor; and A memory for storing computer-executable instructions which, when executed, cause the processor to perform the method according to any one of claims 1-12. A computer-readable storage medium having computer-executable instructions stored thereon for performing the method according to any one of claims 1-12. A computer program product tangibly stored on a computer-readable storage medium and comprising computer-executable instructions which, when executed, cause at least one processor to perform the any one of the methods described.

Images