CN112809668A - Method, system and terminal for automatic hand-eye calibration of mechanical arm - Google Patents
Method, system and terminal for automatic hand-eye calibration of mechanical arm Download PDFInfo
- Publication number
- CN112809668A CN112809668A CN202011602148.4A CN202011602148A CN112809668A CN 112809668 A CN112809668 A CN 112809668A CN 202011602148 A CN202011602148 A CN 202011602148A CN 112809668 A CN112809668 A CN 112809668A
- Authority
- CN
- China
- Prior art keywords
- calibration
- camera
- mechanical arm
- data
- sampling
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 238000000034 method Methods 0.000 title claims abstract 11
- 238000005070 sampling Methods 0.000 claims abstract 29
- 230000003287 optical effect Effects 0.000 claims abstract 4
- 230000000007 visual effect Effects 0.000 claims abstract 4
- 230000036544 posture Effects 0.000 claims 8
- 239000011159 matrix material Substances 0.000 claims 6
- 238000004364 calculation method Methods 0.000 claims 4
- 238000004590 computer program Methods 0.000 claims 1
- 238000013500 data storage Methods 0.000 claims 1
Images
Classifications
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B25—HAND TOOLS; PORTABLE POWER-DRIVEN TOOLS; MANIPULATORS
- B25J—MANIPULATORS; CHAMBERS PROVIDED WITH MANIPULATION DEVICES
- B25J9/00—Programme-controlled manipulators
- B25J9/16—Programme controls
- B25J9/1628—Programme controls characterised by the control loop
- B25J9/1653—Programme controls characterised by the control loop parameters identification, estimation, stiffness, accuracy, error analysis
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B25—HAND TOOLS; PORTABLE POWER-DRIVEN TOOLS; MANIPULATORS
- B25J—MANIPULATORS; CHAMBERS PROVIDED WITH MANIPULATION DEVICES
- B25J13/00—Controls for manipulators
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B25—HAND TOOLS; PORTABLE POWER-DRIVEN TOOLS; MANIPULATORS
- B25J—MANIPULATORS; CHAMBERS PROVIDED WITH MANIPULATION DEVICES
- B25J9/00—Programme-controlled manipulators
- B25J9/16—Programme controls
- B25J9/1602—Programme controls characterised by the control system, structure, architecture
- B25J9/161—Hardware, e.g. neural networks, fuzzy logic, interfaces, processor
Landscapes
- Engineering & Computer Science (AREA)
- Robotics (AREA)
- Mechanical Engineering (AREA)
- Automation & Control Theory (AREA)
- Physics & Mathematics (AREA)
- Artificial Intelligence (AREA)
- Evolutionary Computation (AREA)
- Fuzzy Systems (AREA)
- Mathematical Physics (AREA)
- Software Systems (AREA)
- Manipulator (AREA)
Abstract
The invention provides a method, a system and a terminal for automatic hand-eye calibration of a mechanical arm, wherein the method comprises the following steps: s1, setting gesture boundary points required by data acquisition, and setting a distance range from the center of the camera to the calibration board and an angle range from the optical axis of the camera to the normal of the calibration board; s2, fine-tuning the posture to acquire a small number of data points for estimating the initial position of the calibration plate under the condition of ensuring the calibration plate to be in the visual field according to the initial position of the mechanical arm; s3, uniformly sampling a predetermined number of sampling points in the range set in the S1, wherein the sampling points are the attitude positions of the camera under the coordinate system of the calibration plate; and S4, controlling the mechanical arm to pass through the S3 sampling points one by one, and controlling the camera to take pictures and calculate at each sampling point, so that automatic hand-eye calibration of the mechanical arm is realized. According to the invention, only the boundary of the gesture needs to be preset manually when the calibration gesture is selected, and each calibration point position does not need to be defined manually, so that the labor cost is greatly reduced.
Description
Technical Field
The invention relates to the field of computer vision camera calibration and mechanical arm control, in particular to a method, a system and a terminal for automatic hand-eye calibration of a mechanical arm.
Background
Camera calibration is a fundamental problem in the field of computer vision. The camera calibration is divided into camera internal reference calibration and external reference calibration. The internal reference calibration aims at obtaining the imaging attribute of the camera, and the external reference calibration aims at obtaining the posture and the position of the camera. Hand-eye calibration generally refers to a process in which one or more robotic arms, one or more cameras, and a pose relationship between the cameras and the robotic arms are solved in a system.
The vision sensor system can be divided into the following parts by the position relation of the vision sensor and the mechanical arm of the industrial robot: Eye-in-Hand and Eye-to-Hand. The two forms are all that the posture relation between the hand and the eye is calculated through selecting a series of sampling points, wherein the posture relation between the calibration plate and the camera in each sampling point is different. The selection of the current sampling point is mainly realized through manual control or preset point position.
Through the search of the prior art, a general hand-eye calibration process with eyes outside the hands is provided in the Chinese invention patent 'a method and a system for calibrating a camera at the tail end of a mechanical arm' (CN109658460A), the relation between the camera and the mechanical arm is solved by a least square method, the selection of the posture of the mechanical arm during calibration is not involved, and manual operation or pre-definition is required.
The selection of the calibration posture can influence the calibration effect to a great extent, the selection of the posture depends on the position of the calibration plate, and the preset posture cannot obtain the expected precision when the position of the calibration plate changes; secondly, the selection of the pose of the calibration sampling point is not mentioned, no matter manual operation or position pre-definition, point selection and other operations need to be carried out again when the new scene is deployed in a new environment, and because the difference of the point selection can cause large fluctuation of the calibration result, the labor cost is inevitably introduced.
Disclosure of Invention
Aiming at the defects in the prior art, the invention aims to provide a method, a system and a terminal for automatic hand-eye calibration of a mechanical arm, which have low cost and high precision.
In one aspect of the present invention, a method for automatic hand-eye calibration of a robot arm is provided, including:
s1, setting gesture boundary points required by data acquisition, and setting a distance range from the center of the camera to the calibration board and an angle range from the optical axis of the camera to the normal of the calibration board;
s2, fine-tuning the posture to acquire a small number of data points for estimating the initial position of the calibration plate under the condition of ensuring the calibration plate to be in the visual field according to the initial position of the mechanical arm;
s3, taking the initial position of the calibration plate in S2 as a reference point generated by sampling points, and uniformly sampling a predetermined number of sampling points in the range set in S1, wherein the sampling points are attitude positions of the camera under the coordinate system of the calibration plate;
and S4, controlling the mechanical arm to pass through the S3 sampling points one by one, and controlling the camera to take pictures and calculate at each sampling point, so that automatic hand-eye calibration of the mechanical arm is realized.
Optionally, in S2, the fine-tuning the pose while ensuring that the calibration board is in the field of view according to the initial position of the robot arm and collecting a small number of data points for estimating the initial position of the calibration board includes:
s2.1: randomly generating a distance within the distance range defined by S1, and randomly generating a group of angles within the angle range;
s2.2: controlling the mechanical arm to move from the initial position to the posture position generated in 2.1, acquiring a picture when the mechanical arm moves a set distance, and stopping if the calibration plate cannot be identified in the picture or the posture generated in 2.1 is reached; when stopping, the collected data is taken as one of the data points of the initial calibration, otherwise, the data is repeated for 2.1 until the data is collected;
s2.3: repeating 2.1 and 2.2 until a sufficient amount of initial data has been collected;
s2.4: and (4) calculating the calibration of the eye on the hand according to the data points collected in the step (2.3), and solving an external parameter matrix from the calibration plate to the base and from the camera to the tip of the mechanical arm.
Alternatively, in S3, the uniformly sampling the predetermined number of sample points within the range set in S1 includes:
s3.1: in the distance range set in S1, the equal probability is used for sampling n distances, wherein n is the number of preset sampling points;
s3.2: sampling n groups of angles at equal probability in the angle range set by S1, wherein each group of angles comprises four parameters, the first parameter describes the azimuth angle from the camera to the coordinate center of the calibration plate, and the angles are extracted from 0 degree to 360 degrees at equal intervals; the second parameter describes the pitch angle from the camera to the coordinate center of the calibration plate, and is extracted at medium intervals from the range set by S1; the third parameter describes the roll angle of the camera to the coordinate center of the calibration plate, and is extracted at medium intervals from the range set by S1;
s3.3: determining the position and the posture of a unique camera relative to a coordinate system of a calibration board based on the distance and the angle sampled in S3.1 and S3.2, wherein the optical axis of the camera faces to the center of the coordinate system of the calibration board, and randomly extracting an offset angle in the directions of the x axis and the y axis of the camera under the condition of ensuring that the calibration board is in a visual field so as to ensure that the calibration board can appear at any position in a picture;
s3.4: and combining the distances and the angles generated in S3.1, S3.2 and S3.3 to obtain n calibration sampling points.
Optionally, in S4, the controlling the robot arm to pass through the S3 sampling points one by one, and simultaneously controlling the camera to take a picture and calculate at each sampling point, so as to implement automatic hand-eye calibration of the robot arm, including:
s4.1: for all the sampling points generated in the S3, converting the poses of the sampling points into the joint space of the mechanical arm, wherein each pose is described by the angle of all the joints of the mechanical arm;
s4.2: calculating the shortest path for traversing all sampling points by taking the maximum value of the angle difference of the corresponding joint between the two postures as an index for measuring the distance between the two postures;
s4.3: and in the process of traversing all sampling points according to the shortest path sequence obtained in the S4.2, recalculating the hand-eye calibration every time one point is collected, solving an external parameter matrix from the calibration plate to the base and from the camera to the tip of the mechanical arm, and collecting the next sampling point according to the updated matrix.
In a second aspect of the present invention, there is provided a system for unifying hand-eye calibration of two kinds of mechanical arms, comprising:
the path planning module plans a path of the mechanical arm moving in the space according to gesture boundary points required by data acquisition, and sets a distance range from the center of the camera to the calibration plate and an angle range from the optical axis of the camera to the normal of the calibration plate;
the data acquisition module controls the mechanical arm to move according to the path planned by the path planning module, uniformly samples a predetermined number of sampling points in a set range, and controls the camera to take a picture at each sampling point; according to the initial position of the mechanical arm, fine-tuning the posture under the condition of ensuring that the calibration plate is in the visual field, and collecting a small number of data points for estimating the initial position of the calibration plate; the sampling point is the attitude position of the camera under a calibration plate coordinate system;
and the calibration calculation module is used for calculating at each sampling point according to the data acquired by the data acquisition module so as to realize automatic hand-eye calibration of the mechanical arm.
Optionally, the data acquisition module includes:
the mechanical arm control module is interacted with the mechanical arm, and is used for sending an action instruction to the mechanical arm and reading the running state of the mechanical arm;
the camera control module is interacted with the camera, controls the work of the camera, sets camera parameters, sends a shooting instruction and reads a shooting result of the camera;
and the data storage module is used for sorting and storing the mechanical arm data and the camera data acquired by the mechanical arm control module and the camera control module, wherein the sorting is to serialize the posture information of the mechanical arm according to the sequence of sampling points, and name and gather the camera information according to timestamps.
Optionally, the calibration calculation module includes:
the data reading module is used for reading collected original data for calibration, wherein the original data refer to pictures sampled by a camera in each sampling point and the gesture of the mechanical arm;
the PNP module calculates the pose between the calibration plate and the camera according to the data read by the data reading module;
and the hand-eye calibration calculation module is used for calculating the hand-eye calibration by using various optional algorithms according to the pose obtained by the PNP module, and calculating to obtain the attitude relationship between the camera and the mechanical arm and between the camera and the calibration plate.
In a third aspect of the invention, a terminal is provided, comprising a memory, a processor and a computer program stored on the memory and executable on the processor, the processor being operable to execute the method for robotic hand-eye calibration of a robotic arm when executing the program.
Compared with the prior art, the embodiment of the invention has at least one of the following beneficial effects:
1) according to the invention, the position of the sampling point is automatically selected based on the position of the calibration plate, and the selected sampling points are similar under different scene settings, so that a stable and uniform calibration effect can be obtained, and the precision is improved.
2) According to the invention, the position of the sampling point is automatically selected based on the position of the calibration plate, only the distance and the angle boundary need to be manually selected, excessive manual operation is not needed, the point re-selection every time is avoided, the efficiency is improved, the labor cost is reduced, and the deployment under different environments is convenient.
Drawings
Other features, objects and advantages of the invention will become more apparent upon reading of the detailed description of non-limiting embodiments with reference to the following drawings:
FIG. 1 is a flowchart of a method for robotic arm hand-eye calibration according to an embodiment of the present invention;
FIG. 2 is a block diagram of a system for robotic hand-eye calibration according to an embodiment of the present invention;
FIG. 3 is a flowchart of a method for robotic arm automated hand-eye calibration in accordance with a preferred embodiment of the present invention;
FIG. 4 is a schematic diagram of a coordinate relationship between a calibration plate and a robot arm according to an embodiment of the present invention.
Detailed Description
The present invention will be described in detail with reference to specific examples. The following examples will assist those skilled in the art in further understanding the invention, but are not intended to limit the invention in any way. It should be noted that variations and modifications can be made by persons skilled in the art without departing from the spirit of the invention. All falling within the scope of the present invention.
FIG. 1 is a flowchart of a method for robotic arm hand-eye calibration according to an embodiment of the present invention; as shown in fig. 1, the method in this embodiment may be implemented as the following steps:
s1, setting gesture boundary points required by data acquisition, and setting a distance range from the center of the camera to the calibration board and an angle range from the optical axis of the camera to the normal of the calibration board;
s2, fine-tuning the posture to acquire a small number of data points for estimating the initial position of the calibration plate under the condition of ensuring the calibration plate to be in the visual field according to the initial position of the mechanical arm;
s3, uniformly sampling a predetermined number of sampling points in the range set in the S1, wherein the sampling points are attitude positions of the camera under a calibration plate coordinate system;
and S4, controlling the mechanical arm to pass through the S3 sampling points one by one, and controlling the camera to take pictures and calculate at each sampling point, so as to realize automatic hand-eye calibration of the mechanical arm.
According to the embodiment of the invention, the position of the sampling point is automatically selected based on the position of the calibration plate, and the selected sampling points are similar under different scene settings, so that a stable and uniform calibration effect can be obtained. Of course, in other pre-selected embodiments, after the above-mentioned S4, the result may be further calibrated for self-checking.
As a preferred embodiment, in the above S2, the fine-tuning attitude acquiring a small number of data points for estimating the initial position of the calibration board while ensuring that the calibration board is in the field of view according to the initial position of the robot arm includes:
s2.1: randomly generating a distance within the distance range defined by S1, and randomly generating a group of angles within the angle range;
s2.2: controlling the mechanical arm to move from the initial position to the posture position generated in 2.1, acquiring a picture when the mechanical arm moves a set distance, and stopping if the calibration plate cannot be identified in the picture or the posture generated in 2.1 is reached; when stopping, the collected data is taken as one of the data points of the initial calibration, otherwise, the data is repeated for 2.1 until the data is collected;
s2.3: repeating 2.1 and 2.2 until a sufficient amount of initial data has been collected;
s2.4: and (4) calculating the calibration of the eye on the hand according to the data points collected in the step (2.3), and solving an external parameter matrix from the calibration plate to the base and from the camera to the tip of the mechanical arm.
As a preferred embodiment, in the above S3, the uniformly sampling the predetermined number of sample points within the range set in S1 includes:
s3.1: in the distance range set in S1, the equal probability is used for sampling n distances, wherein n is the number of preset sampling points;
s3.2: sampling n groups of angles at equal probability in the angle range set by S1, wherein each group of angles comprises four parameters, the first parameter describes the azimuth angle from the camera to the coordinate center of the calibration plate, and the angles are extracted from 0 degree to 360 degrees at equal intervals; the second parameter describes the pitch angle from the camera to the coordinate center of the calibration plate, and is extracted at medium intervals from the range set by S1; the third parameter describes the roll angle of the camera to the coordinate center of the calibration plate, and is extracted at medium intervals from the range set by S1;
s3.3: determining the position and the posture of a unique camera relative to a coordinate system of a calibration board based on the distance and the angle sampled in S3.1 and S3.2, wherein the optical axis of the camera faces to the center of the coordinate system of the calibration board, and randomly extracting an offset angle in the directions of the x axis and the y axis of the camera under the condition of ensuring that the calibration board is in a visual field so as to ensure that the calibration board can appear at any position in a picture;
s3.4: and combining the distances and the angles generated in S3.1, S3.2 and S3.3 to obtain n calibration sampling points.
According to the embodiment of the invention, the position of the sampling point is automatically selected based on the position of the calibration plate, the distance and the angle boundary are only manually selected, excessive manual operation is not needed, the point re-selection every time is omitted, the efficiency is improved, and the deployment under different environments is convenient.
As a preferred embodiment, in the above S4, the controlling the robot arm to pass through the S3 sampling points one by one, and simultaneously controlling the camera to take a picture and calculate at each sampling point, so as to achieve automatic hand-eye calibration of the robot arm, includes:
s4.1: for all the sampling points generated in the S3, converting the poses of the sampling points into the joint space of the mechanical arm, wherein each pose is described by the angle of all the joints of the mechanical arm;
s4.2: calculating the shortest path for traversing all sampling points by taking the maximum value of the angle difference of the corresponding joint between the two postures as an index for measuring the distance between the two postures;
s4.3: and in the process of traversing all sampling points according to the shortest path sequence obtained in the S4.2, recalculating the hand-eye calibration every time one point is collected, solving an external parameter matrix from the calibration plate to the base and from the camera to the tip of the mechanical arm, and collecting the next sampling point according to the updated matrix.
Fig. 2 is a block diagram of a system for robot hand-eye calibration according to an embodiment of the present invention.
Referring to fig. 2, in this embodiment, a system for robot hand-eye calibration of a robot arm is provided, including:
the path planning module plans a path of the mechanical arm moving in the space according to gesture boundary points required by data acquisition, and sets a distance range from the center of the camera to the calibration plate and an angle range from the optical axis of the camera to the normal of the calibration plate;
the data acquisition module controls the mechanical arm to move according to the path planned by the path planning module, uniformly samples a predetermined number of sampling points in a set range, and controls the camera to take a picture at each sampling point; according to the initial position of the mechanical arm, fine-tuning the posture under the condition of ensuring that the calibration plate is in the visual field, and collecting a small number of data points for estimating the initial position of the calibration plate; the sampling point is the attitude position of the camera under the coordinate system of the calibration plate;
and the calibration calculation module is used for calculating at each sampling point according to the data acquired by the data acquisition module so as to realize automatic hand-eye calibration of the mechanical arm.
In a preferred embodiment, the data acquisition module comprises:
the mechanical arm control module is interacted with the mechanical arm, and is used for sending an action instruction to the mechanical arm and reading the running state of the mechanical arm;
the camera control module is interacted with the camera, controls the work of the camera, sets camera parameters, sends a shooting instruction and reads a shooting result of the camera;
and the data storage module is used for sorting and storing the mechanical arm data and the camera data acquired by the mechanical arm control module and the camera control module, wherein the sorting means that the attitude information of the mechanical arm is serialized according to the sequence of sampling points, and the camera information is named and summarized according to the time stamps.
In a preferred embodiment, the calibration calculation module includes:
the data reading module is used for reading collected original data for calibration, wherein the original data refer to pictures sampled by a camera in each sampling point and the gesture of the mechanical arm;
the PNP module calculates the pose between the calibration plate and the camera according to the data read by the data reading module;
and the hand-eye calibration calculation module is used for calculating the hand-eye calibration by using various optional algorithms according to the pose obtained by the PNP module, and calculating to obtain the attitude relationship between the camera and the mechanical arm and between the camera and the calibration plate.
On the basis of the embodiment shown in fig. 2, in order to further ensure the accuracy of the result, as a preferred embodiment, the system for robot arm automatic hand-eye calibration comprises the following modules:
a path planning module: the system is used for planning the motion path of the mechanical arm in the space;
a data acquisition module: the system is used for acquiring and storing data required by calibration calculation;
a calibration calculation module: the system comprises a calibration board, a camera, a mechanical arm, a calibration board and a control system, wherein the calibration board is used for calibrating the camera and the mechanical arm;
a self-checking module: the method is used for carrying out preliminary self-check on the calibration result.
In the system in the above embodiment of the present invention, the specific technology implemented by each module may correspond to the specific technology in the above method embodiment, and is not described herein again.
In another embodiment of the present invention, there is also provided a terminal, including a memory, a processor, and a computer program stored in the memory and executable on the processor, where the processor, when executing the computer program, may be configured to perform the method for robot arm automatic hand-eye calibration in any of the above embodiments.
In order to better explain the above technology of the present invention, the following specific application examples are combined, but it should be understood that the following specific examples are not intended to limit the present invention.
Referring to fig. 3, in conjunction with the above method and system, eye-to-eye calibration of a six-axis robotic arm and a camera mounted at the tip of the robotic arm is performed; the embodiment provides a hand-eye calibration method for a six-axis mechanical arm, which realizes the hand-eye calibration of the eyes of the mechanical arm on the hand and the hands of the eyes outside the hand through the steps of path planning, data acquisition, calibration calculation and the like. The method specifically comprises the following steps:
step 1: installing equipment to be subjected to hand-eye calibration, and placing a calibration plate in the middle area of the camera view;
step 2: setting gesture boundary points required by data acquisition, wherein the distance range from the center of a camera to a calibration plate and the angle range from the optical axis of the camera to the normal of the calibration plate need to be specified; specifically, including distance, azimuth, pitch and roll;
and step 3: through coordination of the path planning module and the calibration calculation module, according to the initial position of the mechanical arm, fine-tuning the posture under the condition of ensuring the calibration plate to be in the visual field, and collecting a small number of data points (including but not limited to 5 data points) for estimating the initial position of the calibration plate;
and 4, step 4: uniformly sampling a predetermined number of sampling points in the range defined in the step 2, wherein the sampled points are the attitude positions of the camera under a calibration plate coordinate system;
and 5: controlling the mechanical arm to pass through the sampling points one by one, and simultaneously controlling the camera to take pictures, store and calculate at each sampling point; updating the position of the calibration plate under the coordinate system of the mechanical arm and the position of the camera relative to the tip of the mechanical arm after calculating each new sampling point, and updating the position of the next sampling point by using the updated result;
step 6: after the calculation is finished, a self-checking module is used for carrying out preliminary self-checking on the calibration result;
in this embodiment, step 3 includes the following steps:
step 3.1: randomly generating a distance within the distance range defined in the step 2, and randomly generating a group of angles within the angle range;
step 3.2: controlling the mechanical arm to move from the initial position to the posture position generated in the step 3.1, acquiring a picture every 5% of the movement distance, and stopping if the calibration plate cannot be identified in the picture or the posture generated in the step 3.1 is reached; if the collected data exist during stopping, taking the data collected for the last time as one of the data points of the initial calibration, otherwise repeating the step 3.1 until the data are collected;
step 3.3: repeat 3.1 and 3.2 until a sufficient number of initial data (including but not limited to 5 data points) have been collected;
step 3.4: and (3) calculating the calibration of the eyes on the hands by using a calibration calculation module according to the data points collected in the step (3.3), and solving an external parameter matrix from a calibration plate to a base and from a camera to the tip of the mechanical arm (including but not limited to using an algorithm based on Kronecker product, quaternion and the like).
In the preferred embodiment, the effect of automatically starting calibration in the new environment is realized through the step 3. no longer like the prior art, a series of postures for calibration need to be manually set, so that manual operation is further reduced, and the deployment efficiency of the new environment is also improved.
In this embodiment, step 4 includes the following steps:
step 4.1: the equal probability is used to sample n distances (where n is the number of predetermined sampling points) within the distance range defined in step 2;
step 4.2: sampling n groups of angles at equal probability within the angle range defined in the step 2, wherein each group of angles comprises four parameters, the first parameter describes the azimuth angle from the camera to the coordinate center of the calibration plate, and the angles are extracted at equal intervals from 0 degree to 360 degrees; the second parameter describes the pitch angle from the camera to the coordinate center of the calibration plate, and is extracted at medium intervals from the preset range in the step 2; the third parameter describes the roll angle from the camera to the coordinate center of the calibration plate, and is extracted at medium intervals from the preset range in the step 2;
step 4.3: the distance and the angle sampled in the step 4.1 and the step 4.2 can determine the position and the posture of a unique camera relative to a coordinate system of the calibration plate, at the moment, the optical axis of the camera faces to the center of the coordinate system of the calibration plate, and at the moment, under the condition that the calibration plate is still in a visual field, a deviation angle is randomly extracted in the directions of the x axis and the y axis of the camera, so that the calibration plate can be ensured to appear at any position in a picture;
step 4.4: and (4) combining the distances and the angles generated in the steps 4.1, 4.2 and 4.3 to obtain n calibration sampling points.
In the above preferred embodiment, step 4 generates a series of points fixed with respect to the position of the calibration board for sampling based on the preliminary result obtained in step 3, and compared with the prior art, the same sampling point taking mode can be used in different deployment scenes, so that the calibration result has stronger stability, and the calibration effect is not affected by the position of the selected point due to point re-selection in the new deployment scene as in the prior art.
In this embodiment, step 5 includes the following steps:
step 5.1: converting the poses of all the sampling points generated in the step 4 into a mechanical arm joint space, wherein each pose is described by using the angles of all joints of the mechanical arm;
step 5.2: calculating the shortest path for traversing all sampling points by taking the maximum value of the angle difference of the corresponding joint between the two postures as an index for measuring the distance between the two postures;
step 5.3: and 5.3, in the process of traversing all sampling points according to the shortest path sequence obtained in the step 5.3, recalculating the hand-eye calibration every time one point is collected, solving an external parameter matrix from the calibration plate to the base and from the camera to the tip of the mechanical arm, and collecting the next sampling point according to the updated matrix.
In the above preferred embodiment, step 5, by using the joint coordinates to calculate the shortest path, the movement of the mechanical arm in the whole calibration process is as small as possible, so as to achieve the effect of calibrating as fast as possible; meanwhile, a step of recalculating and updating the calibration result is provided in the calibration process, so that the influence caused by the initial calibration deviation in the step 3 can be reduced.
In this embodiment, step 6 includes the following steps:
step 6.1: for each data point collected we can calculate AxB, whether the eye is on hand or off hand-1To obtain y' calculated based on x (coordinate relation is shown in figure 4);
step 6.2: and (4) carrying out error analysis (including but not limited to statistics after conversion into Euler angles or quaternions) on the multiple groups of y' calculated in the step 6.1 and the y finally calculated in the step 5.
In the above preferred embodiment, step 6 is mainly to implement error analysis, and there may be a preliminary evaluation on the calibration result, which may be compared with the error index in the standard condition.
According to the embodiments of the invention, when the calibration posture is selected, only the boundary of the posture needs to be preset manually, and each calibration point position does not need to be defined manually, so that the labor cost is greatly reduced.
It should be noted that, the steps in the method provided by the present invention may be implemented by using corresponding modules, devices, units, and the like in the system, and those skilled in the art may refer to the technical solution of the system to implement the step flow of the method, that is, the embodiment in the system may be understood as a preferred example for implementing the method, and details are not described herein.
Those skilled in the art will appreciate that, in addition to implementing the system and its various devices provided by the present invention in purely computer readable program code means, the method steps can be fully programmed to implement the same functions by implementing the system and its various devices in the form of logic gates, switches, application specific integrated circuits, programmable logic controllers, embedded microcontrollers and the like. Therefore, the system and various devices thereof provided by the present invention can be regarded as a hardware component, and the devices included in the system and various devices thereof for realizing various functions can also be regarded as structures in the hardware component; means for performing the functions may also be regarded as structures within both software modules and hardware components for performing the methods.
The foregoing description of specific embodiments of the present invention has been presented. It is to be understood that the present invention is not limited to the specific embodiments described above, and that various changes and modifications may be made by one skilled in the art within the scope of the appended claims without departing from the spirit of the invention. The above-described preferred features may be used in any combination without conflict with each other.
Claims (10)
1. A method for automatic hand-eye calibration of a mechanical arm is characterized by comprising the following steps:
s1, setting gesture boundary points required by data acquisition, and setting a distance range from the center of the camera to the calibration board and an angle range from the optical axis of the camera to the normal of the calibration board;
s2, fine-tuning the posture to acquire a small number of data points for estimating the initial position of the calibration plate under the condition of ensuring the calibration plate to be in the visual field according to the initial position of the mechanical arm;
s3, taking the initial position of the calibration plate in S2 as a reference point generated by sampling points, and uniformly sampling a predetermined number of sampling points in the range set in S1, wherein the sampling points are attitude positions of the camera under the coordinate system of the calibration plate;
and S4, controlling the mechanical arm to pass through the S3 sampling points one by one, and controlling the camera to take pictures and calculate at each sampling point, so that automatic hand-eye calibration of the mechanical arm is realized.
2. The method for robotic arm-robot hand-eye calibration of claim 1, wherein in S2, the collecting a small number of data points for estimating the initial position of the calibration plate based on the initial position of the robotic arm with the fine-tuning pose ensuring the calibration plate is within the field of view comprises:
s2.1: randomly generating a distance within the distance range defined by S1, and randomly generating a group of angles within the angle range;
s2.2: controlling the mechanical arm to move from the initial position to the posture position generated in 2.1, acquiring a picture when the mechanical arm moves a set distance, and stopping if the calibration plate cannot be identified in the picture or the posture generated in 2.1 is reached; when stopping, the collected data is taken as one of the data points of the initial calibration, otherwise, the data is repeated for 2.1 until the data is collected;
s2.3: repeating 2.1 and 2.2 until a sufficient amount of initial data has been collected;
s2.4: and (4) calculating the calibration of the eye on the hand according to the data points collected in the step (2.3), and solving an external parameter matrix from the calibration plate to the base and from the camera to the tip of the mechanical arm.
3. The method for robot arm automatic hand-eye calibration recited in claim 1, wherein said evenly sampling a predetermined number of sample points within the range set in S1 in S3 comprises:
s3.1: in the distance range set in S1, the equal probability is used for sampling n distances, wherein n is the number of preset sampling points;
s3.2: sampling n groups of angles at equal probability in the angle range set by S1, wherein each group of angles comprises four parameters, the first parameter describes the azimuth angle from the camera to the coordinate center of the calibration plate, and the angles are extracted from 0 degree to 360 degrees at equal intervals; the second parameter describes the pitch angle from the camera to the coordinate center of the calibration plate, and is extracted at medium intervals from the range set by S1; the third parameter describes the roll angle of the camera to the coordinate center of the calibration plate, and is extracted at medium intervals from the range set by S1;
s3.3: determining the position and the posture of a unique camera relative to a coordinate system of a calibration board based on the distance and the angle sampled in S3.1 and S3.2, wherein the optical axis of the camera faces to the center of the coordinate system of the calibration board, and randomly extracting an offset angle in the directions of the x axis and the y axis of the camera under the condition of ensuring that the calibration board is in a visual field so as to ensure that the calibration board can appear at any position in a picture;
s3.4: and combining the distances and the angles generated in S3.1, S3.2 and S3.3 to obtain n calibration sampling points.
4. The method for robot arm automatic hand-eye calibration according to claim 1, wherein in step S4, the robot arm is controlled to pass through the sampling points S3 one by one, and the camera is controlled to take a picture and calculate at each sampling point, so as to realize robot arm automatic hand-eye calibration, comprising:
s4.1: for all the sampling points generated in the S3, converting the poses of the sampling points into the joint space of the mechanical arm, wherein each pose is described by the angle of all the joints of the mechanical arm;
s4.2: calculating the shortest path for traversing all sampling points by taking the maximum value of the angle difference of the corresponding joint between the two postures as an index for measuring the distance between the two postures;
s4.3: and in the process of traversing all sampling points according to the shortest path sequence obtained in the S4.2, recalculating the hand-eye calibration every time one point is collected, solving an external parameter matrix from the calibration plate to the base and from the camera to the tip of the mechanical arm, and collecting the next sampling point according to the updated matrix.
5. The method for robotic arm-robot hand-eye calibration of claim 1, further comprising: and carrying out self-check on the calibration result.
6. The method for robotic arm-robot hand-eye calibration of claim 1, further comprising: after the calculation of S4 is finished, a self-check is made on the calibration result;
the self-checking of the calibration result comprises:
whether the eye is on the hand or out of the hand,by calculating AxB for each acquired data point-1Obtaining y' calculated based on x, wherein A is an external parameter matrix from a calibration plate to a camera, x is an external parameter matrix from the camera to a mechanical arm base, and B is an external parameter matrix from the mechanical arm base to the mechanical arm tip;
and carrying out error analysis on the multiple groups of calculated y' and the finally calculated y.
7. A system for automatic hand-eye calibration of a mechanical arm is characterized by comprising:
the path planning module plans a path of the mechanical arm moving in the space according to gesture boundary points required by data acquisition, and sets a distance range from the center of the camera to the calibration plate and an angle range from the optical axis of the camera to the normal of the calibration plate;
the data acquisition module controls the mechanical arm to move according to the path planned by the path planning module, uniformly samples a predetermined number of sampling points in a set range, and controls the camera to take a picture at each sampling point; according to the initial position of the mechanical arm, fine-tuning the posture under the condition of ensuring that the calibration plate is in the visual field, and collecting a small number of data points for estimating the initial position of the calibration plate; the sampling point is the attitude position of the camera under a calibration plate coordinate system;
and the calibration calculation module is used for calculating at each sampling point according to the data acquired by the data acquisition module so as to realize automatic hand-eye calibration of the mechanical arm.
8. The system for robotic hand-eye calibration of a robotic arm of claim 7, wherein the data acquisition module comprises:
the mechanical arm control module is interacted with the mechanical arm, and is used for sending an action instruction to the mechanical arm and reading the running state of the mechanical arm;
the camera control module is interacted with the camera, controls the work of the camera, sets camera parameters, sends a shooting instruction and reads a shooting result of the camera;
and the data storage module is used for sorting and storing the mechanical arm data and the camera data acquired by the mechanical arm control module and the camera control module, wherein the sorting is to serialize the posture information of the mechanical arm according to the sequence of sampling points, and name and gather the camera information according to timestamps.
9. The system for robotic arm-eye hand calibration according to claim 7, wherein the calibration calculation module comprises:
the data reading module is used for reading collected original data for calibration, wherein the original data refer to pictures sampled by a camera in each sampling point and the gesture of the mechanical arm;
the PNP module calculates the pose between the calibration plate and the camera according to the data read by the data reading module;
and the hand-eye calibration calculation module is used for calculating the hand-eye calibration by using various optional algorithms according to the pose obtained by the PNP module, and calculating to obtain the attitude relationship between the camera and the mechanical arm and between the camera and the calibration plate.
10. A terminal comprising a memory, a processor and a computer program stored on the memory and executable on the processor, wherein the processor when executing the program is operable to perform a method for robotic arm hands-eye calibration as claimed in any of claims 1-6.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011602148.4A CN112809668B (en) | 2020-12-30 | 2020-12-30 | Method, system and terminal for automatic hand-eye calibration of mechanical arm |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011602148.4A CN112809668B (en) | 2020-12-30 | 2020-12-30 | Method, system and terminal for automatic hand-eye calibration of mechanical arm |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN112809668A true CN112809668A (en) | 2021-05-18 |
| CN112809668B CN112809668B (en) | 2022-08-30 |
Family
ID=75855361
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202011602148.4A Active CN112809668B (en) | 2020-12-30 | 2020-12-30 | Method, system and terminal for automatic hand-eye calibration of mechanical arm |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN112809668B (en) |
Cited By (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113664835A (en) * | 2021-09-15 | 2021-11-19 | 上海景吾智能科技有限公司 | Automatic hand-eye calibration method and system for robot |
| CN113744341A (en) * | 2021-07-21 | 2021-12-03 | 北京旷视科技有限公司 | Camera pose calibration method and device for robot system and electronic equipment |
| CN115139305A (en) * | 2022-08-02 | 2022-10-04 | 北京理工大学 | Six-dimensional force sensor offset compensation method and system |
| CN115414072A (en) * | 2022-08-31 | 2022-12-02 | 美的集团(上海)有限公司 | Pharyngeal swab sampling method and device, sampling equipment and computer program product |
| CN117838311A (en) * | 2024-03-07 | 2024-04-09 | 杭州海沛仪器有限公司 | Target spot ablation respiration gating method and system based on optical positioning |
| CN119526477A (en) * | 2023-08-30 | 2025-02-28 | 武汉联影智融医疗科技有限公司 | Robotic arm registration method and robot arm registration device |
Citations (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103115615A (en) * | 2013-01-28 | 2013-05-22 | 山东科技大学 | Fully-automatic calibration method for hand-eye robot based on exponential product model |
| US20160114486A1 (en) * | 2014-10-27 | 2016-04-28 | Quanta Storage Inc. | Method for automatically recalibrating a robotic arm |
| CN108436909A (en) * | 2018-03-13 | 2018-08-24 | 南京理工大学 | A kind of hand and eye calibrating method of camera and robot based on ROS |
| CN109658460A (en) * | 2018-12-11 | 2019-04-19 | 北京无线电测量研究所 | A kind of mechanical arm tail end camera hand and eye calibrating method and system |
| CN110238845A (en) * | 2019-05-22 | 2019-09-17 | 湖南视比特机器人有限公司 | Optimal Calibration point chooses and the automatic hand and eye calibrating method and device of error measurement |
| CN110834333A (en) * | 2019-11-14 | 2020-02-25 | 中科新松有限公司 | A robot hand-eye calibration method and storage medium |
| CN111203881A (en) * | 2020-01-16 | 2020-05-29 | 河北工业大学 | An online feedback hand-eye calibration method based on weighted least squares |
| CN111390901A (en) * | 2019-01-02 | 2020-07-10 | 中达电子零组件(吴江)有限公司 | Automatic calibration method and calibration device for mechanical arm |
-
2020
- 2020-12-30 CN CN202011602148.4A patent/CN112809668B/en active Active
Patent Citations (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103115615A (en) * | 2013-01-28 | 2013-05-22 | 山东科技大学 | Fully-automatic calibration method for hand-eye robot based on exponential product model |
| US20160114486A1 (en) * | 2014-10-27 | 2016-04-28 | Quanta Storage Inc. | Method for automatically recalibrating a robotic arm |
| CN108436909A (en) * | 2018-03-13 | 2018-08-24 | 南京理工大学 | A kind of hand and eye calibrating method of camera and robot based on ROS |
| CN109658460A (en) * | 2018-12-11 | 2019-04-19 | 北京无线电测量研究所 | A kind of mechanical arm tail end camera hand and eye calibrating method and system |
| CN111390901A (en) * | 2019-01-02 | 2020-07-10 | 中达电子零组件(吴江)有限公司 | Automatic calibration method and calibration device for mechanical arm |
| CN110238845A (en) * | 2019-05-22 | 2019-09-17 | 湖南视比特机器人有限公司 | Optimal Calibration point chooses and the automatic hand and eye calibrating method and device of error measurement |
| CN110834333A (en) * | 2019-11-14 | 2020-02-25 | 中科新松有限公司 | A robot hand-eye calibration method and storage medium |
| CN111203881A (en) * | 2020-01-16 | 2020-05-29 | 河北工业大学 | An online feedback hand-eye calibration method based on weighted least squares |
Cited By (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113744341A (en) * | 2021-07-21 | 2021-12-03 | 北京旷视科技有限公司 | Camera pose calibration method and device for robot system and electronic equipment |
| CN113744341B (en) * | 2021-07-21 | 2025-01-17 | 北京旷视科技有限公司 | Camera pose calibration method and device of robot system and electronic equipment |
| CN113664835A (en) * | 2021-09-15 | 2021-11-19 | 上海景吾智能科技有限公司 | Automatic hand-eye calibration method and system for robot |
| CN115139305A (en) * | 2022-08-02 | 2022-10-04 | 北京理工大学 | Six-dimensional force sensor offset compensation method and system |
| CN115414072A (en) * | 2022-08-31 | 2022-12-02 | 美的集团(上海)有限公司 | Pharyngeal swab sampling method and device, sampling equipment and computer program product |
| CN119526477A (en) * | 2023-08-30 | 2025-02-28 | 武汉联影智融医疗科技有限公司 | Robotic arm registration method and robot arm registration device |
| CN117838311A (en) * | 2024-03-07 | 2024-04-09 | 杭州海沛仪器有限公司 | Target spot ablation respiration gating method and system based on optical positioning |
| CN117838311B (en) * | 2024-03-07 | 2024-05-31 | 杭州海沛仪器有限公司 | Target spot ablation respiratory gating system based on optical positioning |
Also Published As
| Publication number | Publication date |
|---|---|
| CN112809668B (en) | 2022-08-30 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN112809668B (en) | Method, system and terminal for automatic hand-eye calibration of mechanical arm | |
| CN110599541B (en) | Method and device for calibrating multiple sensors and storage medium | |
| CN113601503B (en) | Hand-eye calibration method, device, computer equipment and storage medium | |
| CN110136208B (en) | Joint automatic calibration method and device for robot vision servo system | |
| CN107598977B (en) | Method and system for realizing automatic robot teaching by using vision and laser range finder | |
| CN111515950B (en) | Method, device, device and storage medium for determining transformation relationship of robot coordinate system | |
| KR20180120647A (en) | System and method for tying together machine vision coordinate spaces in a guided assembly environment | |
| CN111571082B (en) | Automatic welding method and device, mobile terminal and readable storage medium | |
| CN111452048B (en) | Calibration method and device for relative spatial position relation of multiple robots | |
| CN112743548B (en) | Method, system and terminal for unifying hand-eye calibration of two mechanical arms | |
| JP2005201824A (en) | Measuring device | |
| KR101972432B1 (en) | A laser-vision sensor and calibration method thereof | |
| CN104802186A (en) | Robot programming apparatus for creating robot program for capturing image of workpiece | |
| US11577400B2 (en) | Method and apparatus for managing robot system | |
| JP2019119027A (en) | Method of controlling robot system, and robot system | |
| CN113664835A (en) | Automatic hand-eye calibration method and system for robot | |
| CN109448062A (en) | A kind of camera calibration method, apparatus, terminal device and storage medium | |
| CN115519547A (en) | Hand-eye calibration method and system for steel bar binding | |
| CN114998446A (en) | Calibration method between camera and manipulator, controller, device and storage medium | |
| JP2019098409A (en) | Robot system and calibration method | |
| CN112985359B (en) | Image acquisition method and image acquisition equipment | |
| JP2022152845A (en) | Calibration device for robot control | |
| WO2025194747A1 (en) | Object pose determination method and system, device, medium, and program product | |
| WO2022172471A1 (en) | Assistance system, image processing device, assistance method and program | |
| CN114407012B (en) | Robot motion calibration method, apparatus, computer device and storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |