CN117169828A

CN117169828A - External parameter calibration methods, equipment and systems

Info

Publication number: CN117169828A
Application number: CN202210590869.0A
Authority: CN
Inventors: 王剑伟; 马继彬; 郭剑艇
Original assignee: Huawei Technologies Co Ltd
Current assignee: Shenzhen Yinwang Intelligent Technology Co Ltd
Priority date: 2022-05-27
Filing date: 2022-05-27
Publication date: 2023-12-05

Abstract

The embodiment of the application provides an external parameter calibration method, equipment and a system. The method comprises the following steps: and sending a control instruction to the calibration control mechanism, wherein the control instruction is used for controlling the calibration plate to sequentially move to a plurality of target positions, receiving N point pair information of a first image point on the calibration plate in the process of controlling the calibration plate to sequentially move to the plurality of target positions, wherein the N point pair information corresponds to the N target positions in the plurality of target positions one by one, the point pair information comprises first coordinate information and second coordinate information, the first coordinate information comprises the coordinates of the first image point in a coordinate system of the first equipment when the calibration plate moves to the corresponding target position, the second coordinate information comprises the coordinates of the first image point in a coordinate system of the second equipment when the calibration plate moves to the corresponding target position, and determining external parameters between the first equipment and the second equipment according to the N point pair information. The automatic movement control of the calibration plate is realized, and the calibration efficiency and the calibration precision are improved.

Description

External parameter calibration method, device and system

Technical Field

The application relates to the technical field of intelligent vehicles, in particular to an external parameter calibration method, device and system.

Background

At present, a plurality of sensors deployed in a vehicle are used for detecting target objects around the vehicle, and data fusion is carried out on detection data of different sensors through calibrated external parameters, so that rich detection data are obtained, and the visual perception capability of the vehicle is enhanced. In the process of external parameter calibration, the calibration plate needs to be manually moved to realize detection of the calibration plate at different positions. However, manual movement of the calibration plate results in lower calibration efficiency, and there is a problem that the calibration accuracy is low due to insufficient accuracy of the movement position, and the calculation result of the external parameters enters a local optimal solution due to insufficient movement area.

Disclosure of Invention

The external parameter calibration method, the external parameter calibration equipment and the external parameter calibration system provided by the embodiment of the application are used for improving the calibration efficiency and the calibration precision and avoiding the problem that the technical result enters a local optimal solution.

In a first aspect, an embodiment of the present application provides an external parameter calibration method, including: sending a control instruction to the calibration control mechanism, wherein the control instruction is used for controlling the calibration plate to sequentially move to a plurality of target positions; in the process of controlling the calibration plate to sequentially move to a plurality of target positions, receiving N point pair information of a first image point on the calibration plate, wherein N is an integer greater than or equal to 4, the N point pair information corresponds to N target positions in the plurality of target positions one by one, the point pair information comprises first coordinate information and second coordinate information, the first coordinate information comprises coordinates of the first image point in a coordinate system of a first device when the calibration plate moves to the corresponding target position, and the second coordinate information comprises coordinates of the first image point in a coordinate system of a second device when the calibration plate moves to the corresponding target position; and determining external parameters between the first equipment and the second equipment according to the N point pair information.

According to the external parameter calibration method provided by the first aspect, the upper computer controls the calibration plate to move to a plurality of target positions through the calibration control mechanism, N point pair information of a first image point on the calibration plate is obtained in the process that the calibration plate moves to the plurality of target positions, and then external parameters between the first equipment and the second equipment are determined according to the N point pair information, so that automatic movement control of the calibration plate is realized, and the calibration efficiency and the calibration precision are improved.

In one possible implementation, the control instructions include a plurality of first control instructions, where the first control instructions are used to control the calibration plate to move from a current target position to a next target position, and send control instructions to a calibration plate control mechanism, where the first control instructions include: transmitting the first control instruction to the control mechanism in response to a trigger event; wherein the triggering event includes at least one of: receiving first point pair information corresponding to a current target position of the calibration plate; receiving information indicating that the calibration of the target position where the calibration plate is currently located fails; a periodic trigger occasion is reached.

According to the external parameter calibration method provided by the embodiment, the upper computer controls the calibration plate to move to the next target position based on the response to the trigger event, so that the real-time determination of the position of the calibration plate and the automatic control of the calibration plate are realized, and the calibration efficiency is improved.

In one possible embodiment, the sending the first control instruction to the control mechanism in response to a trigger event includes: when the number of the received point pair information is smaller than a first preset value M, the calibration plate is controlled to move from the current target position to the next target position in response to the trigger event, and M is an integer greater than or equal to N.

Optionally, the method further comprises: and ending the process of controlling the calibration plate to move when the number of the received point pair information is greater than or equal to a first preset value M, wherein M is an integer greater than or equal to N.

According to the external parameter calibration method provided by the embodiment, before the calibration board is controlled to move to the next target position, whether enough point pair information is received is determined, and the problem that excessive point pair information is acquired to cause larger consumption of calculation resources is avoided.

In one possible implementation, the control instruction carries coordinates of the plurality of target positions.

According to the external parameter calibration method provided by the embodiment, the upper computer indicates the coordinates of a plurality of target positions to the calibration control mechanism, so that the calibration control mechanism can accurately move the calibration plate to the plurality of target positions.

In a possible implementation, the control instruction carries a movement rule, where the movement rule includes a step size and/or a movement direction, and the movement rule is used to control the calibration plate to sequentially move to the plurality of target positions.

According to the external parameter calibration method provided by the embodiment, the upper computer indicates the movement rule to the calibration control mechanism, so that the calibration control mechanism can move according to the indicated movement rule, and the signaling overhead can be reduced under the condition that the required moving target positions are more.

In one possible implementation manner, the control instruction includes a plurality of first control instructions, where the first control instructions are used for controlling the calibration board to move from a current target position to a next target position, and coordinates of the next target position carried by the first control instructions are determined based on first point pair information corresponding to the current target position of the calibration board.

According to the external parameter calibration method, the next target position of the calibration plate is determined based on the first point pair information corresponding to the current target position, so that the target position moved by the calibration plate is more flexible, the external parameter calibration requirement is met, and the accuracy of the external parameter calibration is improved.

In one possible implementation, the first device and the second device are both deployed on a first vehicle, and the plurality of target locations are both located in a detection area of the first vehicle; the plurality of target positions are distributed on L detection planes, L is a positive integer, the L detection planes are perpendicular to the ground plane, and the L detection planes are parallel to each other; for each of the L detection planes, a plurality of target positions in the detection plane have different heights.

According to the external parameter calibration method provided by the embodiment, the external parameters are distributed at a plurality of target positions of L detection planes, so that the calibration areas of external parameter calibration are enriched, and the calibrated external parameters are prevented from entering a local optimal solution.

In one possible embodiment, when L is an integer greater than 1, the L detection planes are sequentially spaced apart by a second preset value in a direction perpendicular to the L detection planes.

According to the external parameter calibration method provided by the embodiment, L detection planes are separated by a certain distance, so that a plurality of target positions are richer, and the accuracy of external parameter calibration is improved.

In one possible implementation, determining the external parameters between the first device and the second device according to the N point pair information includes: according to the N point pair information, Q calibration groups are determined, wherein the Q calibration groups comprise N point pair information in the N point pair information, N is the number of the point pair information required for determining the external parameter, and the Q calibration groups are different from each other; for each calibration group in the Q calibration groups, determining an alternative external parameter corresponding to the calibration group; if q=1, using the alternative external parameter as an external parameter between the first device and the second device; or if Q is greater than 1, verifying the calibration accuracy of the alternative external parameters corresponding to the Q calibration groups according to the Q verification groups, determining the alternative external parameter with the highest calibration accuracy of the alternative external parameters corresponding to the Q calibration groups as the external parameter, wherein the Q verification groups are in one-to-one correspondence with the Q calibration groups, and N-N point pair information in one verification group and N point pair information in the corresponding calibration group form the N point pair information.

According to the external parameter calibration method provided by the embodiment, the alternative external parameters obtained based on calculation of each calibration group are verified based on the verification group, so that the alternative external parameter with the highest accuracy is obtained, and the alternative external parameter is used as the external parameter between the first equipment and the second equipment, so that the accuracy of external parameter calibration is improved.

In a possible implementation manner, the verifying the calibration accuracy of the alternative external parameters corresponding to the Q calibration groups according to the Q verification groups includes: and aiming at each verification group in the Q verification groups, obtaining N-N initial calibration accuracies according to N-N point pair information in the verification group and the alternative external parameters, and determining the calibration accuracy of the alternative external parameters according to the N-N initial calibration accuracies.

According to the external parameter calibration method provided by the embodiment, when the alternative external parameters are verified based on the plurality of point pair information in one verification group, the upper computer can determine initial calibration accuracy based on the point pair information and determine final calibration accuracy of the alternative external parameters, so that the calibration accuracy of the alternative external parameters is more reliable.

In one possible implementation, the obtaining N-N initial calibration accuracies according to the N-N point pair information in the verification group and the alternative external parameters includes: determining third coordinate information according to the first coordinate information and the alternative external parameters in the point pair information aiming at each point pair information in the N-N point pair information; and determining the initial calibration accuracy according to the coordinate deviation between the third coordinate information and the second coordinate information in the point pair information.

By the external reference calibration method provided by the embodiment, the identification of the initial calibration accuracy is realized.

Optionally, the receiving N point pair information of the first image point on the calibration board includes: the N point pair information transmitted by the control device of the first vehicle is received, and the control device of the first vehicle is connected with the first equipment and the second equipment.

In one possible embodiment, the calibration plate is circular in shape, and the first image point is the center of the calibration plate.

By the external parameter calibration method provided by the embodiment, when the shape of the calibration plate is circular, the first image point, namely the point pair information of the circle center, is more easily fitted.

In a second aspect, an embodiment of the present application provides an external parameter calibration method, including: and responding to the periodical triggering time or the control instruction, and controlling the calibration plate to sequentially move to a plurality of target positions.

In one possible embodiment, the method further comprises: and receiving the control instruction sent by the upper computer.

In one possible embodiment, the control calibration plate moves to a plurality of target positions in sequence, including: according to the movement rule, controlling the calibration plate to sequentially move to the target positions; wherein the movement rules comprise a step size and/or a movement direction.

In one possible implementation, the control instruction carries the movement rule.

In one possible embodiment, the control calibration plate moves to a plurality of target positions in sequence, including: and controlling the calibration plate to sequentially move to the target positions according to the coordinates of the target positions.

In one possible implementation manner, the control instruction includes a plurality of first control instructions, the coordinates of the next target position carried by the first control instructions, and the control calibration board moves to a plurality of target positions in sequence, including: and controlling the calibration plate to move from the current target position to the next target position according to one of the first control instructions.

In one possible embodiment, the control calibration plate moves to a plurality of target positions in sequence, including: and controlling the calibration control mechanism to move the calibration plate to the target positions in the horizontal direction and/or the vertical direction.

In one possible implementation, the first device and the second device are both deployed on a first vehicle, and the plurality of target locations are both located in a detection area of the first vehicle; the plurality of target positions are distributed on L detection planes, L is a positive integer, the L detection planes are parallel to each other, and the L detection planes are perpendicular to the ground plane; for each of the L detection planes, a plurality of target positions in the detection plane have different heights.

In one possible embodiment, when L is an integer greater than 1, the L detection planes are spaced apart by a second preset value in a direction perpendicular to the L detection planes.

The advantages of the method for calibrating external parameters provided by the second aspect and the possible embodiments of the second aspect may be referred to the advantages of the first aspect and the possible embodiments of the first aspect, and are not described herein.

In a third aspect, an embodiment of the present application provides an external parameter calibration method, including: when the calibration plate moves to a target position, first detection data obtained by detecting the calibration plate by the first equipment and second detection data obtained by detecting the calibration plate by the second equipment are obtained; determining point pair information of a first image point on the calibration plate according to the first detection data and the second detection data, wherein the point pair information comprises first coordinate information and second coordinate information, the first coordinate information comprises coordinates of the first image point in a coordinate system of the first equipment when the calibration plate moves to the target position, the second coordinate information comprises coordinates of the first image point in the coordinate system of the second equipment when the calibration plate moves to the target position, and the point pair information is used for determining external parameters between the first equipment and the second equipment; the point pair information is transmitted.

In a possible embodiment, the determining the point pair information of the first image point on the calibration plate according to the first detection data and the second detection data includes: and if the pattern deviation between the calibration plate pattern reflected by the first detection data and the calibration plate pattern reflected by the second detection data is smaller than the preset deviation, determining the first coordinate information according to the first detection data, determining the second coordinate information according to the second detection data, and taking the first coordinate information and the second coordinate information as the point pair information.

The advantages of the method for calibrating external parameters provided by the third aspect and the possible embodiments of the third aspect may be referred to the advantages brought by the first aspect and the possible embodiments of the first aspect, and are not described herein.

In a fourth aspect, an embodiment of the present application provides an upper computer, including: a processor and a memory for storing a computer program for invoking and running the computer program stored in the memory for performing the method of the first aspect and embodiments described above.

The advantages of the upper computer provided by the fourth aspect and the possible embodiments of the fourth aspect may be referred to the advantages of the first aspect and the possible embodiments of the first aspect, and are not described herein.

In a fifth aspect, an embodiment of the present application provides a calibration control mechanism, including: a processor and a memory for storing a computer program for invoking and running the computer program stored in the memory for performing the method of the second aspect and embodiments described above.

The advantages of the calibration control mechanism provided by the fifth aspect and the possible embodiments of the fifth aspect may be referred to the advantages of the first aspect and the possible embodiments of the first aspect, and are not described herein.

In a sixth aspect, an embodiment of the present application provides a control apparatus, including: a processor and a memory for storing a computer program for invoking and running the computer program stored in the memory for performing the method of the third aspect and embodiments described above.

The advantages of the control device provided by the sixth aspect and the possible embodiments of the sixth aspect may be referred to the advantages of the first aspect and the possible embodiments of the first aspect, and are not described herein.

In a seventh aspect, an embodiment of the present application provides an external parameter calibration system, including: the host computer and the calibration control apparatus in the fourth aspect described above, the calibration control apparatus including the calibration control mechanism in the fifth aspect described above.

In one possible embodiment, the calibration control device further comprises a calibration plate; the calibration control mechanism comprises a movable device and a connecting device, one end of the connecting device is fixed on the movable device, and the other end of the connecting device is fixed with the calibration plate; the movable device is used for controlling the calibration plate to move in the horizontal direction.

In one possible embodiment, the movable device comprises a ground moving device or a flying device.

In one possible embodiment, the connection means is a liftable means; the liftable device is used for controlling the calibration plate to move in the vertical direction.

In an eighth aspect, an embodiment of the present application provides a calibration control apparatus, including: the calibration control mechanism and the calibration plate as in the fifth aspect described above, the calibration control mechanism comprising a movable device and a connecting device, one end of the connecting device being fixed to the movable device, the other end of the connecting device being fixed to the calibration plate; the movable device is used for controlling the calibration plate to move in the horizontal direction.

In a ninth aspect, an embodiment of the present application provides a chip, including: a processor for invoking and executing computer instructions from memory to cause a device on which the chip is mounted to perform a method as in the first aspect, the second aspect, the third aspect or in each of the possible implementations.

In a tenth aspect, embodiments of the present application provide a computer readable storage medium storing computer program instructions for causing a computer to perform a method as in the first aspect, the second aspect, the third aspect or each possible implementation.

In an eleventh aspect, embodiments of the application provide a computer program product comprising computer program instructions for causing a computer to perform the method as in the first aspect, the second aspect, the third aspect or in each of the possible implementations.

In a twelfth aspect, embodiments of the present application provide an apparatus comprising logic circuitry and an input-output interface, wherein the input-output interface is for receiving signals from or transmitting signals to or from other communication devices than the apparatus, the logic circuitry is for executing code instructions to implement a method as in the first, second, third or each possible implementation.

Drawings

Fig. 1 is a schematic structural diagram of a vehicle according to an embodiment of the present application;

FIG. 2 is a schematic structural diagram of an external parameter calibration system according to an embodiment of the present application;

FIG. 3 is a schematic diagram of a detection plane according to an embodiment of the present application;

FIG. 4 is a schematic diagram of a calibration control device according to an embodiment of the present application

FIG. 5 is a schematic diagram of another calibration control device according to an embodiment of the present application;

FIG. 6 is a schematic flow chart of an external parameter calibration method according to an embodiment of the present application;

FIG. 7 is a schematic flow chart of an external parameter calibration method according to an embodiment of the present application;

FIG. 8 is a schematic flow chart of an external parameter calibration method according to an embodiment of the present application;

fig. 9 is a schematic structural diagram of an electronic device according to an embodiment of the present application.

Detailed Description

The technical solutions in the embodiments of the present application will be described below with reference to the accompanying drawings in the embodiments of the present application.

Fig. 1 is a schematic structural diagram of a vehicle 100 according to an embodiment of the present application. As shown in fig. 1, a control device 110 is disposed in a vehicle 100, and the control device 110 may be connected to sensors 120 disposed in the vehicle to detect environments outside the vehicle through the sensors 120, respectively. The sensor 120 may include, for example, a camera sensor 121, a radar sensor 122 (e.g., a laser radar sensor, a millimeter wave radar sensor, etc.), a combination positioning sensor 123, an ultrasonic sensor 124, etc.

It should be noted that, when the control device 110 detects through the sensor 120, the detection data of a plurality of different sensors may be fused to obtain a more accurate detection result of the target object. For example, the RGB information of the target object is acquired by the camera sensor 121, which is advantageous for extracting various texture features to realize various high-difficulty image recognition and segmentation tasks, but it is difficult to acquire three-dimensional information of the target object; the radar sensor 122 can acquire two-dimensional (2D) or three-dimensional (3D) point cloud data of the detected target object, where the point cloud data can accurately reflect the depth and reflection intensity of the target object, and perform image acquisition on the target object around the vehicle, but sparsity and disorder of the point cloud make it difficult to implement image recognition and segmentation tasks. The detection data of the camera sensor 121 and the radar sensor 122 can be fused, so that the detection result can accurately reflect not only the two-dimensional RGB information of the target object, but also the depth information of the target object, and the detection capability of the vehicle to the external environment is improved.

Generally, the mounting positions of different sensors in a vehicle are different, and the pose of the target object is different when the target object is detected, so that the detection result of each sensor belongs to the coordinate system of each sensor, and external parameters of different sensors need to be calibrated, so that the conversion of detection data among different sensors is realized, and the fusion of the detection data is realized.

As shown in fig. 1, the upper computer 210 may obtain the detection data of the sensor to the calibration board from the control device 110, and calibrate based on the detection data of different sensors, to obtain external parameters between different sensors. Optionally, the upper computer 210 may send the obtained external parameters to the control device 110, so that the control device 110 fuses the detection data of each sensor.

By using the upper computer to calibrate the camera sensor 121 and the radar sensor 122, the upper computer 210 can calibrate the coordinate (u, v) of one image point in the 2D image data collected by the camera sensor 121 and the 3D point cloud number detected by the radar sensor 122According to the coordinates (X _L ，Y _L ，Z _L ) As a set of point pair information, the outlier matrix is solved by the following equation (1):

where s is a scaling factor, and the coordinate matrix formed by the image points is multiplied by the scaling factor to implement scaling on the 2D image, and optionally s may be 1; vector quantityWhere u, v are the 2D coordinates of the image points, respectively, and 1 is the added constant; k is an internal reference of the camera sensor 121, and is generally provided by the manufacturer of the camera sensor 121 and is constant for the camera sensor 121; [ R ] _3×3 t _3×1 ]Is an external parameter between the camera sensor 121 and the radar sensor 122, which is a matrix of 3 rows and 4 columns, wherein R _3×3 A rotation matrix of 3 rows and 3 columns, e.g.>t _3×1 A translation matrix of 3 rows and 1 columns, e.g. +.>[R _3×3 t _3×1 ]May also be referred to as a conversion matrix between the first device and the second device _3×3 t _3×1 ]Or can be expressed as

Inputting a plurality of point pair information of the image points into the formula (1) to obtain an external parameter R _3×3 t _3×1 ]. The plurality of point pair information may include point pair information collected for a same image point on the calibration plate when the calibration plate is in a plurality of positions, and/or point pair information collected for a plurality of image points on the calibration plate when the calibration plate is in a same position.

In the process of detecting the calibration plate by the control device 110 through the sensor to obtain the multiple point pair information of the image point, the calibration plate needs to be manually moved to detect the calibration plate at different positions, however, the manual movement of the calibration plate results in lower calibration efficiency, further, the movement position is inaccurate, the calibration precision is lower, and the movement area is not big enough, so that the calculation result of the external parameters enters the local optimal solution.

In order to solve the problems of low calibration efficiency and low calibration accuracy, the application provides an external parameter calibration scheme, which is characterized in that a calibration board control mechanism is instructed to control the movement of a calibration board based on a control instruction, so that the calibration board is detected through two different sensors at a plurality of positions in the movement process of the calibration board, a plurality of point pair information of the calibration board is obtained, and the external parameter between the two different sensors is determined according to the plurality of point pair information.

Fig. 2 is a schematic structural diagram of an external parameter calibration system 200 according to an embodiment of the present application. As shown in connection with FIG. 2, the external reference calibration system 200 may include a host computer 210 and a calibration control device 220. The control device 220 may include a calibration control mechanism 221. The calibration control device 220 may also include a calibration plate 222. The upper computer 210 and the calibration control mechanism 221 may be connected by a wired or wireless manner, and the calibration control mechanism 221 and the calibration plate 222 may be connected by a mechanical component. The calibration control mechanism 221 may control the calibration plate 222 to sequentially move to a plurality of target positions.

The plurality of target locations may be located in a detection zone around a first vehicle (e.g., vehicle 100 in fig. 1 or 2), which may be a zone where each sensor deployed in the first vehicle is capable of detection. The plurality of target positions may be distributed in L detection planes in the detection area, where L is a positive integer, the L detection planes are parallel to each other, and all of the L detection planes are perpendicular to the ground plane. It should be noted that the detection plane may not have a planar entity, in other words, a spatial plane including at least one target position may be used as a detection plane, which is generally opposite to the vehicle, e.g. the detection plane in front of the vehicle should be opposite to the head of the vehicle The lateral detection plane should be opposite to the vehicle body. In each detection plane, a plurality of target positions having different heights may be included in the detection plane. Referring to FIG. 3, a plurality of target locations may be distributed in the detection plane L ₁ Detection plane L ₂ And a detection plane L ₂ Each detection plane includes 9 different target positions in 3 columns by 3 rows. In the direction perpendicular to the detection planes, there should be a preset interval between two adjacent detection planes, and the interval between every two adjacent detection planes in the L detection planes may be the same or different, e.g. the interval between every two adjacent detection planes in the L detection planes may be a second preset value, e.g. the detection plane L in FIG. 3 ₁ And a detection plane L ₂ A second preset value is arranged between the two detection planes L ₂ And a detection plane L ₂ The interval is a second preset value. Alternatively, the second preset value may be 20 meters.

The calibration control mechanism 221 may control the movement of the calibration plate to a plurality of target positions in the horizontal direction and/or the vertical direction. The horizontal direction is the ground plane direction, and the vertical direction is the direction vertical to the ground plane. When the calibration control mechanism 221 moves the calibration plate in the horizontal direction, the calibration plate may be moved to the target position at an arbitrary angle. The calibration control mechanism 221 may move the calibration plate 222 between different columns of the same detection plane in the horizontal direction, or the calibration control mechanism 221 may move the calibration plate 222 between the same columns of different detection planes in the horizontal direction, or the calibration control mechanism 221 may move the calibration plate 222 between different columns of different detection planes in the horizontal direction; the calibration control mechanism 221 may move the calibration plate in the same column of the same detection plane in the vertical direction to target positions of different rows (heights), or the calibration control mechanism 221 may move from a lower target position of one detection plane to a higher target position of another detection plane in the detection space. Still taking the multiple target positions shown in fig. 3 as an example, the calibration control mechanism 221 may move the calibration plate 222 from the detection plane L in the horizontal direction ₁ Is moved to the detection plane L ₁ Or (d) target position 21, orThe calibration control mechanism 221 can horizontally move the calibration plate 222 from the detection plane L ₁ Is moved to the detection plane L ₂ Or the calibration control mechanism 221 may move the calibration plate 222 in the horizontal direction from the detection plane L ₁ Is moved to the detection plane L ₃ Target position 31 of (c), etc.; still referring to FIG. 3, the calibration control mechanism 221 may vertically move the calibration control plate 222 from the detection plane L ₁ Is moved to the detection plane L ₁ Is a target location 12 of (2); alternatively, the calibration control mechanism 221 may move the calibration plate 222 from the detection plane L in the detection space ₁ Is moved to the detection plane L ₂ Target position 33 of (c), etc.

By way of example, the calibration control mechanism 221 may comprise a movable device 221-1 and a connection device 221-2, one end of the connection device 221-2 being fixed to the movable device 221-1, and the other end of the connection device 221-2 being fixed to the calibration plate 222. The movable device 221-1 may be a ground moving device (see fig. 4) or a flying device (see fig. 5).

Continuing with the above example, when the movable device 221-1 is a floor mobile device (e.g., a remote control cart), referring to the top view shown in fig. 4 a and the front view of the vehicle shown in b, the movable device 221-1 may be moved in the horizontal direction within the detection area of the vehicle, and the connection device 221-2 may be a lever based on the fixed carrying calibration plate of the connection device 221-2. In some embodiments, to expand the movement range of the calibration plate, the connection device 221-2 may be a liftable device, such as an electric lifting rod, and the calibration control mechanism 221 may control the calibration plate to move in a vertical direction through the connection device 221-2.

Continuing the above example, when the movable apparatus 221-1 is a flying apparatus (e.g., a drone, an unmanned aerial vehicle, etc.), referring to the top view shown in fig. 5 a and the front view of the vehicle shown in b, the movable apparatus 221-1 may move in a horizontal direction and/or a vertical direction within a detection area of the vehicle.

The calibration control mechanism 221 may move between a plurality of target positions according to a preset period, for example, the calibration control mechanism 221 controls the calibration plate to move to one of the plurality of target positions at each periodic trigger timing; alternatively, the calibration control mechanism 221 may move the calibration plate to a plurality of target positions after receiving the control command transmitted from the host computer 210.

In a first example, the control instructions may instruct the calibration control mechanism 221 to begin controlling the calibration plate to move and move to a plurality of target positions. In this case, the control instruction may carry coordinates of a plurality of target positions, the calibration control mechanism 221 may sequentially move the calibration plate 222 to the plurality of target positions according to the coordinates of the plurality of target positions indicated in the control instruction, and optionally, the calibration plate control mechanism 221 may move the calibration plate 222 to the plurality of target positions indicated by the control instruction according to a preset period after receiving the control instruction; or the control command may carry a movement rule, where the movement rule includes a step size and/or a movement direction, after the calibration control mechanism 221 receives the control command, the calibration plate may be moved to multiple target positions according to the movement rule indicated by the control command, and still taking L detection planes shown in fig. 3 as an example, according to the step size and the movement direction indicated by the movement rule, the calibration control mechanism 221 may calibrate the calibration plate from the detection plane L ₁ The calibration plate is moved in the vertical direction from the target position 11 to the target position 12 and then from the target position 12 to the target position 13, and the calibration control mechanism 221 may move in the horizontal direction from the target position 13 to the target position 23 and continue to move the calibration plate in the order of the target positions 23-22-21-31-32-33.

In a second example, the control instructions may include a plurality of first control instructions, where each first control instruction is used to instruct the control calibration board to move from the current target position to the next target position, and the first control instruction carries coordinates of the next target position. The calibration control mechanism 221 may move the calibration plate 222 to the next target position according to the coordinates of the next target position indicated in the control instruction. In other words, the calibration control mechanism 221 controls the calibration plate to move the target position once every time the first control instruction is received.

Continuing with the second example, the coordinates of the next target position carried in the first control instruction may be determined by the upper computer 210 based on the first point pair information corresponding to the target position where the calibration board is currently located. Wherein the first point pair information is obtained after the control device 110 detects the calibration plate through two different sensor devices (a first device and a second device in the following) when the calibration plate is located at the current target position, the point pair information includes first coordinate information and second coordinate information, the first coordinate information includes coordinates of the first image point in a coordinate system of the first device when the calibration plate moves to the current target position, and the second coordinate information includes coordinates of the first image point in a coordinate system of the second device when the calibration plate moves to the corresponding target position. For example, the upper computer 210 may determine that the next target location is closer to the first device and/or the second device and further from the boundary of the detection area in the case where the reliability of the first point pair information is lower.

It should be understood that the embodiment of the present application does not limit the number of the first devices and the second devices, for example, when the number of the second devices is two, the upper computer 210 may determine the external parameters between the first device and each of the second devices, respectively.

When the vehicle of the present application is an autonomous vehicle, the control device 110 may be implemented as an autonomous controller.

In the first example described above, the upper computer 210 may be transmitted in response to a trigger event when transmitting a control instruction to the calibration control mechanism 221. For example, after receiving the first point pair information corresponding to the current target position of the calibration board, the upper computer 210 sends a first control instruction to the calibration control mechanism 221, so that the calibration control mechanism 221 controls the calibration board 222 to move to the next target position; or, after receiving the information indicating that the calibration of the calibration plate 222 at the current target position is failed, the upper computer 210 sends a control instruction to the calibration control mechanism 221, so that the calibration control mechanism 221 controls the calibration plate 222 to move to the next target position; or, the upper computer 210 sends a control instruction to the calibration control mechanism 221 when the periodic trigger timing is reached, so that the calibration control mechanism 221 controls the calibration plate 222 to move to the next target position. It should also be appreciated that the triggering event may include multiple or all of the events described in the examples above, in which case the host computer 210 may send a control instruction to the calibration control mechanism 221 to cause the calibration control mechanism 221 to control the calibration plate 222 to move to the next target position in the event of one of the multiple triggering events described in the examples above.

In some embodiments, the upper computer 210 may first determine whether the number of the received point pair information reaches a first preset value M, where M is an integer greater than or equal to N, in response to the above trigger event. If the number of the received point pair information is smaller than the first preset value M, the upper computer 210 sends a first control instruction to the calibration control mechanism 221 to control the calibration plate 222 to move from the current target position to the next target position; if the number of the received point pair information reaches (or is greater than or equal to) the first preset value M, the upper computer 210 ends the process of controlling the movement of the calibration plate. The problem of high consumption of computing resources caused by acquiring excessive point pair information is avoided.

Alternatively, the above description is given by taking the example that the control command is sent to the calibration control mechanism 221 by the host computer 210, but it should not be understood that any limitation of the present application is limited, for example, the control command may be sent to the calibration control mechanism 221 by other devices (for example, a control device of a vehicle), and for example, the control command may also be generated by the calibration control mechanism 221.

For example, the control command may be transmitted from the control device 110 of the vehicle to the calibration control mechanism 221. The control device 110 may send a control instruction to the calibration control mechanism 221, instruct the calibration control mechanism 221 to start controlling the calibration plate to move, and move to a plurality of target positions; alternatively, the control device 110 may send a first control instruction to the calibration control mechanism 221 in response to a trigger event, so that the calibration control mechanism 221 controls the calibration plate 222 to move to the next target position.

After determining the first point pair information, the control device 110 of the vehicle may send a first control instruction to the calibration control mechanism 221, so that the calibration control mechanism 221 controls the calibration plate 222 to move to the next target position; or, after determining that the calibration of the calibration plate 222 at the current target position fails, the control device 110 sends a control instruction to the calibration control mechanism 221, so that the calibration control mechanism 221 controls the calibration plate 222 to move to the next target position; alternatively, the control device 110 sends a control command to the calibration control mechanism 221 each time a periodic trigger timing is reached, so that the calibration control mechanism 221 controls the calibration plate 222 to move to the next target position. It should also be appreciated that the triggering event may include more than one or all of the events illustrated above, in which case the control device 110 may send a control command to the calibration control mechanism 221 to cause the calibration control mechanism 221 to control the calibration plate 222 to move to the next target position in the event of one of the various triggering events illustrated above. The calculation of the external parameters can be calculated by the upper computer 210, in which case the control device 110 sends the data to the upper computer 210, and the upper computer finishes the calibration of the external parameters; the calculation of the external parameters can also be calculated by the control device 110, and the calibration function of the upper computer can be understood to be integrated into the control device 110.

For another example, the control command is generated by the calibration control mechanism 221. The calibration control mechanism 221 generates a control instruction, and controls the calibration plate to move to a plurality of target positions based on the control instruction; alternatively, the calibration control mechanism 221 may generate a first control instruction to control the calibration plate 222 to move to the next target position in response to a trigger event.

The calibration control mechanism 221 may generate a first control instruction after receiving the first point pair information from the control device 110 of the vehicle, and control the calibration plate 222 to move to the next target position; alternatively, after receiving the information of failure in calibrating the current target position of the control device 110, the calibration control mechanism 221 generates a first control instruction to control the calibration plate 222 to move to the next target position; alternatively, the calibration control mechanism 221 generates a control command each time a periodic trigger timing is reached, and controls the calibration plate 222 to move to the next target position. It should also be appreciated that the triggering event may include more or all of the events illustrated above, in which case the calibration control mechanism 221 may generate a control command to control the calibration plate 222 to move to the next target position in the event of one of the various triggering events illustrated above.

Alternatively, when the calibration control mechanism 221 receives information from the control device 110, the control device 110 may send the information to the calibration control mechanism 221 based on wired or wireless communication, or may forward the information through the host computer 210.

In the above example, regardless of how the calibration control mechanism 221 controls the movement of the calibration plate 222, the external parameters between the first device and the second device may be determined by the host computer 210 or the control device 110 of the vehicle.

Referring to fig. 2, the upper computer 210 is further connected to the control device 110 of the vehicle to receive data transmitted from the control device 110. Alternatively, the upper computer 210 and the control device 110 may be connected by a wired or wireless manner. For example, it may be based on a controller area network (controller area network, CAN), or an ethernet connection. Alternatively, the communication between the host computer 210 and the control device 110 may satisfy a diagnostic protocol of the vehicle.

In the process that the upper computer 210 controls the calibration control mechanism to sequentially move to a plurality of target positions, each time the calibration plate 222 moves to one target position, the control device 110 of the vehicle can acquire first detection data obtained by detecting the calibration plate by the first device, and second detection data obtained by detecting the calibration plate by the second device, and according to the first detection data and the second detection data, point pair information of a first image point on the calibration plate is determined, wherein the point pair information comprises first coordinate information and second coordinate information, the first coordinate information comprises coordinates of the first image point in a coordinate system of the first device when the calibration plate moves to the target position, and the second coordinate information comprises coordinates of the first image point in a coordinate system of the second device when the calibration plate moves to the target position. Further, the control device 110 may transmit the point-to-point information to the host computer 210.

As mentioned above, the control device 110 may be connected to a plurality of sensors, such as a camera sensor 121, a radar sensor 122, a combination positioning sensor 123, an ultrasonic sensor 124, and the like. Alternatively, the control device 110 may be connected to the sensors via a bus. Alternatively, each of the above sensors may be a sensor group formed by a plurality of sensors to cooperate to perform detection, for example, the camera sensor 121 may be formed by a plurality of cameras and cooperate, the radar sensor 122 may be formed by a plurality of radars and cooperate, and so on.

It will be appreciated that the control device 110 may obtain the point pair information of one or more first image points when the calibration plate is at a target position, where the first image points are points on the calibration plate and the plurality of first image points are points on different positions on the calibration plate.

In some embodiments, the control device 110 may first detect the quality of the first detection data and the second detection data in determining the point pair information. For example, the control device 110 determines whether the pattern deviation between the calibration plate pattern and the calibration plate reflected by the first detection data is smaller than a preset deviation, determines whether the pattern deviation between the calibration plate pattern and the calibration plate reflected by the second detection data is smaller than a preset deviation, and when the pattern deviation between the calibration plate pattern and the calibration plate reflected by the first detection data is smaller than the preset deviation and the pattern deviation between the calibration plate pattern and the calibration plate reflected by the second detection data is smaller than the preset deviation, the control device 110 may determine the first coordinate information according to the first detection data, determine the second coordinate information according to the second detection data, and then send the first coordinate information and the second coordinate information as point pair information to the upper computer 210; when the pattern deviation between the calibration board pattern reflected by the first detection data and the calibration board is greater than the preset deviation, the control device 110 does not need to determine the point pair information based on the first detection data and the second detection data, and in this case, the control device 110 may send information indicating that the calibration of the target position where the calibration board is currently located fails to the upper computer 210.

In connection with the above embodiment, the shape of the calibration plate may be a circle, in which case the control device 110 determines that the pattern deviation between the calibration plate pattern reflected in the first detection data and the calibration plate may be a deviation between the radius of the calibration plate pattern reflected in the first detection data and the radius of the calibration plate. Assuming that the first device is the camera sensor 121, the calibration plate pattern in the first detection data may appear circular or elliptical according to the pose change of the camera sensor 121, and when the calibration plate pattern in the first detection data appears elliptical, the radius of the calibration plate pattern reflected in the first detection data may be the maximum radius of the ellipse. Similarly, the control device 110 determines that the pattern deviation between the calibration plate pattern reflected in the second detection data and the calibration plate may be a deviation between the radius of the calibration plate pattern reflected in the second detection data and the radius of the calibration plate, and if the second apparatus is the radar sensor 122, the radius of the calibration plate pattern reflected in the second detection data is a circle in the three-dimensional space, the radius of the calibration plate pattern reflected in the second detection data is a radius of the circle in the three-dimensional space.

Alternatively, the first image point may be any point on the calibration plate, or the first image point may be the geometric center of the calibration plate. For example, when the calibration plate is circular in shape, the first image point may be the center of the calibration plate.

The control device 110 may perform initial data processing on the first probe data and the second probe data so as to accurately acquire the point pair information.

Assuming the first device is a camera sensor 121, the control 110 may perform an initial image processing on the first detection data, including but not limited to color space conversion, color space filtering, calibration plate pattern fitting, etc. For example, the first detection data collected by the camera sensor 121 is image data of RGB color space, and the control device 110 may convert the image data from RGB color space to HSV color space; the control device 110 performs color space filtering on the image data in the HSV color space to obtain the edge of the calibration plate, and it should be noted that, in order to facilitate the control device 110 to filter to obtain the edge of the calibration plate, the outer edge of the calibration plate and the interior may be set with different colors, for example, as shown in fig. 4, the outer edge of the calibration plate is blue, and the content of the calibration plate is red, then the control device 110 may obtain the blue edge of the calibration plate through color space filtering; further, the control device 110 may fit a circle or an ellipse based on the edge of the calibration plate obtained by filtering, and determine the circle center and the radius of the calibration plate reflected by the first detection data based on the fitted circle or ellipse.

Assuming that the second device is the radar sensor 122, the control apparatus 110 may perform processing of the point cloud data on the second detection data. For example, the second detection data detected by the radar sensor 122 is original point cloud data, and the control device 110 may perform ground-removing clustering filtering on the second detection data, and remove the environmental data around the calibration plate to obtain the point cloud data of the calibration plate; the control device 110 performs filtering processing through an edge extraction technology (for example, a Ring structure-based parallel scheduling (Ring) algorithm) to obtain a point cloud of a circular boundary of the calibration plate; further, the control device 110 utilizes a three-dimensional space circle fitting algorithm to fit a circle in the three-dimensional space, and determines the circle center and the radius of the calibration plate reflected by the second detection data based on the fitted circle in the three-dimensional space.

In the above embodiment, only the first device is taken as the camera sensor 121 and the second device is taken as the radar sensor 122 for the sake of understanding. But is not to be construed as limiting the application in any way, for example the first device may be a camera sensor 121, the second device may be an ultrasonic sensor 124, or the first device may be a radar sensor 122, the second device may be a combination location sensor 123, etc.

The number N of the pair information transmitted to the upper computer 210 by the control device 110 should satisfy the number N of the pair information required for determining the external parameter between the first device and the second device, for example, 4 pair information are required when determining the external parameter based on the above formula (1), that is, N is equal to 4.

In order to improve accuracy of the parameter calibration, the number N of the point pair information sent by the control device 110 to the upper computer 210 may be greater than the number N of the point pair information required for determining the parameter between the first device and the second device, so as to determine a plurality of parameters, and determine an optimal parameter from the plurality of parameters.

Illustratively, the host computer 210 may be based onAnd determining Q calibration groups according to the received N point pair information, wherein the Q calibration groups comprise N point pair information in the N point pair information. It should be noted that, for any two calibration groups of the Q calibration groups, at least one point pair information is different in n point pair information included in the two calibration groups, for example, the calibration group a includes the point pair information a, B, c, d, and the calibration group B includes the point pair information a, B, c, e. For example, the upper computer 210 may traverse the N pieces of point pair information to obtain Q calibration groups according to the permutation and combination, where Q is equal to the number of permutation and combination Further, the upper computer 210 determines, for each of the Q calibration groups, an alternative external parameter corresponding to the calibration group.

Optionally, the upper computer 210 may further determine Q verification groups according to the received N point pair information, where the Q verification groups are in one-to-one correspondence with the Q calibration groups. For example, the upper computer 210 selects N pieces of point pair information from the N pieces of point pair information as one calibration group, and uses N-N pieces of point pair information other than the N pieces of point pair information from the N pieces of point pair information as a verification group corresponding to the calibration group.

Continuing with the above example, if q=1, the upper computer 210 may use the alternative external parameter corresponding to the calibration set as the external parameter between the first device and the second device. Optionally, the upper computer 210 may verify the alternative external parameter according to the verification group corresponding to the calibration group, if the verification is passed, the alternative external parameter is used as the external parameter between the first device and the second device, if the verification is not passed, the alternative external parameter is discarded, or the external parameter calibration process is re-executed.

Continuing with the above example, if Q > 1, the upper computer 210 may verify the calibration accuracy of the alternative external parameters corresponding to the Q calibration groups according to the Q verification groups, and then determine the alternative external parameter with the highest calibration accuracy of the alternative external parameters corresponding to the Q calibration groups as the external parameter between the first device and the second device.

In some embodiments, for each of the Q verification groups, the upper computer 210 obtains N-N initial calibration accuracies according to the N-N point pair information and the alternative external parameters in the verification group, and determines the calibration accuracy of the alternative external parameters according to the N-N initial calibration accuracies.

For example, in the verification process, for each of N-N point pair information in one verification group, the upper computer 210 may input the point pair information and the alternative external parameter into the above formula (1) to obtain third coordinate information, and determine the initial calibration accuracy corresponding to the point pair information according to the coordinate deviation between the third coordinate information and the second coordinate information in the point pair information. The coordinate deviation between the third coordinate information and the second coordinate information in the point pair information may refer to a distance between the coordinate indicated by the second coordinate information and the coordinate indicated by the third coordinate information. The larger the coordinate deviation is, the lower the initial calibration accuracy is, and the smaller the coordinate deviation is, the higher the initial calibration accuracy is.

Further, after verifying the N-N initial calibration accuracies obtained by the N-N point pair information on the candidate external parameters, the upper computer 210 may determine the accuracy of the candidate external parameters according to the N-N initial calibration accuracies, for example, one of the N-N initial calibration accuracies with the highest or lowest accuracy is used as the accuracy of the candidate external parameters, or the sum of the N-N initial calibration accuracies is used as the accuracy of the candidate external parameters, or the variance of the N-N initial calibration accuracies is used as the accuracy of the candidate external parameters.

In some embodiments, the upper computer 210 may perform dead point removal based on a random sample consensus algorithm (RANSAC) to screen 4 pieces of point pair information from the N pieces of point pair information, where the accuracy of the candidate external parameters determined based on the 4 pieces of point pair information is highest compared to the candidate external parameters determined by any 4 pieces of other point pair information in the N pieces of point pair information.

Therefore, in the embodiment of the present application, the upper computer 210 controls the calibration board 222 to move to a plurality of target positions through the calibration control mechanism 221, and obtains N point pair information of the first image point on the calibration board 222 in the process of moving the calibration board 222 to the plurality of target positions, so as to determine external parameters between the first device and the second device according to the N point pair information, thereby realizing automatic movement control of the calibration board, and improving calibration efficiency and calibration precision.

Fig. 6 is a flowchart of an external parameter calibration method 300 according to an embodiment of the present application. The implementation of the embodiment shown in fig. 6 may be the upper computer 210 in fig. 2 or a component (such as a chip or a chip system) of the upper computer 210. The method 300 includes:

s310, a control instruction is sent to the calibration control mechanism, and the control instruction is used for controlling the calibration plate to sequentially move to a plurality of target positions;

S320, in the process of controlling the calibration plate to sequentially move to a plurality of target positions, receiving N point pair information of a first image point on the calibration plate, wherein N is an integer greater than or equal to 4, the N point pair information corresponds to N target positions in the plurality of target positions one by one, the point pair information comprises first coordinate information and second coordinate information, the first coordinate information comprises coordinates of the first image point in a coordinate system of a first device when the calibration plate moves to the corresponding target position, and the second coordinate information comprises coordinates of the first image point in a coordinate system of a second device when the calibration plate moves to the corresponding target position;

s330, determining external parameters between the first device and the second device according to the N point pair information.

In some embodiments, the control instructions include a plurality of first control instructions for controlling the calibration plate to move from a current target position to a next target position, the sending control instructions to the calibration plate control mechanism including: transmitting the first control instruction to the control mechanism in response to a trigger event; wherein,

the triggering event includes at least one of:

Receiving first point pair information corresponding to a current target position of the calibration plate;

receiving information indicating that the calibration of the target position where the calibration plate is currently located fails;

a periodic trigger occasion is reached.

In some embodiments, the sending the first control instruction to the control mechanism in response to a trigger event includes: when the number of the received point pair information is smaller than a first preset value M, the calibration plate is controlled to move from the current target position to the next target position in response to the trigger event, and M is an integer greater than or equal to N.

In some embodiments, the method 300 further comprises: and ending the process of controlling the calibration plate to move when the number of the received point pair information is greater than or equal to a first preset value M, wherein M is an integer greater than or equal to N.

In some embodiments, the control instructions carry coordinates of the plurality of target locations.

In some embodiments, the control instruction carries a movement rule, where the movement rule includes a step size and/or a movement direction, and the movement rule is used to control the calibration plate to sequentially move to the plurality of target positions.

In some embodiments, the control instruction includes a plurality of first control instructions, where the first control instructions are used to control the calibration board to move from a current target position to a next target position, and coordinates of the next target position carried by the first control instructions are determined based on first point pair information corresponding to the current target position of the calibration board.

In some embodiments, the first device and the second device are each deployed on a first vehicle, and the plurality of target locations are each located in a detection zone of the first vehicle; the plurality of target positions are distributed on L detection planes, L is a positive integer, the L detection planes are perpendicular to the ground plane, and the L detection planes are parallel to each other; for each of the L detection planes, a plurality of target positions in the detection plane have different heights.

In some embodiments, when L is an integer greater than 1, the L detection planes are sequentially spaced apart by a second preset value in a direction perpendicular to the L detection planes.

In some embodiments, the determining the external parameters between the first device and the second device based on the N point pair information includes: according to the N point pair information, Q calibration groups are determined, wherein the Q calibration groups comprise N point pair information in the N point pair information, N is the number of the point pair information required for determining the external parameter, and the Q calibration groups are different from each other; for each calibration group in the Q calibration groups, determining an alternative external parameter corresponding to the calibration group; if q=1, using the alternative external parameter as an external parameter between the first device and the second device; or if Q is greater than 1, verifying the calibration accuracy of the alternative external parameters corresponding to the Q calibration groups according to the Q verification groups, determining the alternative external parameter with the highest calibration accuracy of the alternative external parameters corresponding to the Q calibration groups as the external parameter, wherein the Q verification groups are in one-to-one correspondence with the Q calibration groups, and N-N point pair information in one verification group and N point pair information in the corresponding calibration group form the N point pair information.

In some embodiments, the verifying the calibration accuracy of the alternative external parameters corresponding to the Q calibration groups according to the Q verification groups includes: and aiming at each verification group in the Q verification groups, obtaining N-N initial calibration accuracies according to N-N point pair information in the verification group and the alternative external parameters, and determining the calibration accuracy of the alternative external parameters according to the N-N initial calibration accuracies.

In some embodiments, the obtaining N-N initial calibration accuracies from the N-N point pair information in the verification set and the alternative external parameters includes: determining third coordinate information according to the first coordinate information and the alternative external parameters in the point pair information aiming at each point pair information in the N-N point pair information; and determining the initial calibration accuracy according to the coordinate deviation between the third coordinate information and the second coordinate information in the point pair information.

In some embodiments, the receiving N point pair information for the first image point on the calibration plate includes: the N point pair information transmitted by the control device of the first vehicle is received, and the control device of the first vehicle is connected with the first equipment and the second equipment.

In some embodiments, the calibration plate is circular in shape, and the first image point is a center of the calibration plate.

Fig. 7 is a flowchart of an external parameter calibration method 400 according to an embodiment of the present application. The implementation of the embodiment shown in fig. 7 may be the calibration control mechanism 211 of fig. 2 or a component (e.g., a chip or a system-on-chip) of the calibration control mechanism 211. The method 400 includes:

s410, responding to the periodic trigger time or the control instruction, and controlling the calibration plate to sequentially move to a plurality of target positions.

In some embodiments, the method 400 further comprises: and receiving the control instruction sent by the upper computer.

In some embodiments, the control calibration plate moves sequentially to a plurality of target positions, including: according to the movement rule, controlling the calibration plate to sequentially move to the target positions; wherein the movement rules comprise a step size and/or a movement direction.

In some embodiments, the control instruction carries the movement rule.

In some embodiments, the control calibration plate moves sequentially to a plurality of target positions, including: and controlling the calibration plate to sequentially move to the target positions according to the coordinates of the target positions.

In some embodiments, the control instruction includes a plurality of first control instructions, the coordinates of a next target position carried by the first control instructions, and the control calibration board moves to a plurality of target positions in sequence, including: and controlling the calibration plate to move from the current target position to the next target position according to one of the first control instructions.

In some embodiments, the control calibration plate moves sequentially to a plurality of target positions, including: and controlling the calibration control mechanism to move the calibration plate to the target positions in the horizontal direction and/or the vertical direction.

In some embodiments, the first device and the second device are each deployed on a first vehicle, and the plurality of target locations are each located in a detection zone of the first vehicle; the plurality of target positions are distributed on L detection planes, L is a positive integer, the L detection planes are parallel to each other, and the L detection planes are perpendicular to the ground plane; for each of the L detection planes, a plurality of target positions in the detection plane have different heights.

In some embodiments, when L is an integer greater than 1, the L detection planes are spaced apart by a second preset value in a direction perpendicular to the L detection planes.

Fig. 8 is a flowchart of an external parameter calibration method 500 according to an embodiment of the present application. The execution body of the embodiment shown in fig. 8 may be the control device 110 in fig. 1 or fig. 2, or a component (such as a chip or a chip system) in the control device 110. The method 500 includes:

s510, when the calibration plate moves to a target position, acquiring first detection data obtained by detecting the calibration plate by the first equipment and second detection data obtained by detecting the calibration plate by the second equipment;

S520, determining point pair information of a first image point on the calibration plate according to the first detection data and the second detection data, wherein the point pair information comprises first coordinate information and second coordinate information, the first coordinate information comprises coordinates of the first image point in a coordinate system of the first device when the calibration plate moves to the target position, the second coordinate information comprises coordinates of the first image point in a coordinate system of the second device when the calibration plate moves to the target position, and the point pair information is used for determining external parameters between the first device and the second device;

and S530, transmitting the point pair information.

In some embodiments, the determining the point pair information of the first image point on the calibration plate based on the first detection data and the second detection data includes: and if the pattern deviation between the calibration plate pattern reflected by the first detection data and the calibration plate pattern reflected by the second detection data is smaller than the preset deviation, determining the first coordinate information according to the first detection data, determining the second coordinate information according to the second detection data, and taking the first coordinate information and the second coordinate information as the point pair information.

It should be understood that each step in the vehicle control method provided in fig. 6 to 8 is described in the embodiment shown in fig. 1 to 5, and is not repeated here for brevity.

In various embodiments of the application, where terminology and/or descriptions of the various embodiments are consistent and may be referred to each other, unless specifically indicated as such and where logical conflict, features of different embodiments may be combined to form new embodiments in accordance with their inherent logical relationships.

Fig. 9 is a schematic structural diagram of an electronic device 600 according to an embodiment of the present application. The electronic device 600 shown in fig. 9 comprises a processor 610, from which the processor 610 may call and run a computer program to implement the method in an embodiment of the application.

In some embodiments, as shown in fig. 9, the electronic device 600 may also include a memory 620. Wherein the processor 610 may call and run a computer program from the memory 620 to implement the method in an embodiment of the application.

The memory 620 may be a separate device from the processor 610 or may be integrated into the processor 610.

In some embodiments, the electronic device 600 may also include an input interface 630. The processor 610 may control the input interface 630 to communicate with other devices or chips, and in particular, may acquire information or data sent by the other devices or chips.

In some embodiments, the electronic device 600 may also include an output interface 640. Wherein the processor 610 may control the output interface 640 to communicate with other devices or chips, and in particular, may output information or data to other devices or chips.

In some embodiments, the electronic device 600 may implement the corresponding flow of each method of the embodiments of the present application, which is not described herein for brevity.

In some embodiments, the device according to the embodiments of the present application may also be a chip. For example, a system-on-chip or a system-on-chip, etc.

It should be appreciated that the processor of an embodiment of the present application may be an integrated circuit chip having signal processing capabilities. In implementation, the steps of the above method embodiments may be implemented by integrated logic circuits of hardware in a processor or instructions in software form. The processor may be a general purpose processor, a digital signal processor (Digital Signal Processor, DSP), an application specific integrated circuit (Application Specific Integrated Circuit, ASIC), an off-the-shelf programmable gate array (Field Programmable Gate Array, FPGA) or other programmable logic device, discrete gate or transistor logic device, discrete hardware components. The disclosed methods, steps, and logic blocks in the embodiments of the present application may be implemented or performed. A general purpose processor may be a microprocessor or the processor may be any conventional processor or the like. The steps of the method disclosed in connection with the embodiments of the present application may be embodied directly in the execution of a hardware decoding processor, or in the execution of a combination of hardware and software modules in a decoding processor. The software modules may be located in a random access memory, flash memory, read only memory, programmable read only memory, or electrically erasable programmable memory, registers, etc. as well known in the art. The storage medium is located in a memory, and the processor reads the information in the memory and, in combination with its hardware, performs the steps of the above method.

It will be appreciated that the memory in embodiments of the application may be volatile memory or nonvolatile memory, or may include both volatile and nonvolatile memory. The nonvolatile Memory may be a Read-Only Memory (ROM), a Programmable ROM (PROM), an Erasable PROM (EPROM), an Electrically Erasable EPROM (EEPROM), or a flash Memory. The volatile memory may be random access memory (Random Access Memory, RAM) which acts as an external cache. By way of example, and not limitation, many forms of RAM are available, such as Static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double Data Rate SDRAM (Double Data Rate SDRAM), enhanced SDRAM (ESDRAM), synchronous DRAM (SLDRAM), and Direct RAM (DR RAM). It should be noted that the memory of the systems and methods described herein is intended to comprise, without being limited to, these and any other suitable types of memory.

It should be understood that the above memory is illustrative but not restrictive, and for example, the memory in the embodiments of the present application may be Static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDR SDRAM), enhanced SDRAM (ESDRAM), synchronous Link DRAM (SLDRAM), direct RAM (DR RAM), and the like. That is, the memory in embodiments of the present application is intended to comprise, without being limited to, these and any other suitable types of memory.

The embodiment of the application also provides a computer readable storage medium for storing a computer program.

In some embodiments, the computer readable storage medium may be applied to the processor in the embodiments of the present application, and the computer program causes the computer to execute the corresponding flow in each method in the embodiments of the present application, which is not described herein for brevity.

The embodiment of the application also provides a computer program product comprising computer program instructions.

In some embodiments, the computer program product may be applied to a processor in the embodiments of the present application, and the computer program instructions cause the computer to execute the corresponding processes in the methods in the embodiments of the present application, which are not described herein for brevity.

The embodiment of the application also provides a computer program.

In some embodiments, the computer program may be applied to a processor in the embodiments of the present application, where the computer program when executed on a computer causes the computer to perform corresponding processes in the methods in the embodiments of the present application, and for brevity, will not be described in detail herein.

Those of ordinary skill in the art will appreciate that the various illustrative elements and algorithm steps described in connection with the embodiments disclosed herein may be implemented as electronic hardware, or combinations of computer software and electronic hardware. Whether such functionality is implemented as hardware or software depends upon the particular application and design constraints imposed on the solution. Skilled artisans may implement the described functionality in varying ways for each particular application, but such implementation decisions should not be interpreted as causing a departure from the scope of the present application.

In the several embodiments provided by the present application, it should be understood that the disclosed systems, devices, and methods may be implemented in other manners. For example, the apparatus embodiments described above are merely illustrative, e.g., the division of the units is merely a logical function division, and there may be additional divisions when actually implemented, e.g., multiple units or components may be combined or integrated into another system, or some features may be omitted or not performed. Alternatively, the coupling or direct coupling or communication connection shown or discussed with each other may be an indirect coupling or communication connection via some interfaces, devices or units, which may be in electrical, mechanical or other form.

The units described as separate units may or may not be physically separate, and units shown as units may or may not be physical units, may be located in one place, or may be distributed on a plurality of network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

In addition, each functional unit in the embodiments of the present application may be integrated in one processing unit, or each unit may exist alone physically, or two or more units may be integrated in one unit.

The functions, if implemented in the form of software functional units and sold or used as a stand-alone product, may be stored in a computer-readable storage medium. For such understanding, the technical solution of the present application may be embodied essentially or in a part contributing to the prior art or in a part of the technical solution, in the form of a software product stored in a storage medium, comprising several instructions for causing a computer device (which may be a personal computer, a server, a network device, etc.) to perform all or part of the steps of the method according to the embodiments of the present application. And the aforementioned storage medium includes: a U-disk, a removable hard disk, a Read-Only Memory (ROM), a random access Memory (Random Access Memory, RAM), a magnetic disk, or an optical disk, or other various media capable of storing program codes.

The foregoing is merely illustrative of the present application, and the present application is not limited thereto, and any person skilled in the art will readily recognize that variations or substitutions are within the scope of the present application. Therefore, the protection scope of the present application shall be subject to the protection scope of the claims.

Claims

1. A method for calibrating external parameters, characterized in that it includes:

Send control commands to the calibration control mechanism, the control commands being used to control the calibration board to move sequentially to multiple target positions;

During the process of controlling the calibration board to move sequentially to multiple target positions, N point pairs of information of the first image point on the calibration board are received, where N is an integer greater than or equal to 4. The N point pairs of information correspond one-to-one with the N target positions among the multiple target positions. The point pairs of information include first coordinate information and second coordinate information. The first coordinate information includes the coordinates of the first image point in the coordinate system of the first device when the calibration board moves to the corresponding target position. The second coordinate information includes the coordinates of the first image point in the coordinate system of the second device when the calibration board moves to the corresponding target position.

Based on the N point pairs of information, the extrinsic parameters between the first device and the second device are determined.

2. The method according to claim 1, characterized in that the control command includes a plurality of first control commands, the first control commands being used to control the calibration board to move from its current target position to the next target position, and the step of sending the control command to the calibration board control mechanism includes:

In response to the triggering event, the first control command is sent to the control mechanism; wherein,

The triggering event includes at least one of the following:

Receive the first point pair information corresponding to the target position where the calibration board is currently located;

Received information indicating that the calibration board failed to calibrate at its current target location;

The periodic triggering time has been reached.

3. The method according to claim 2, wherein sending the first control command to the control mechanism in response to a triggering event comprises:

When the number of received point-to-point information is less than a first preset value M, in response to the triggering event, the calibration board is controlled to move from the current target position to the next target position, where M is an integer greater than or equal to N.

4. The method according to claim 2, characterized in that the method further comprises:

When the number of received point-to-point information is greater than or equal to a first preset value M, the process of controlling the movement of the calibration board ends, where M is an integer greater than or equal to N.

5. The method according to any one of claims 1 to 4, wherein the control command carries the coordinates of the plurality of target positions.

6. The method according to any one of claims 1 to 4, wherein the control command carries a movement rule, the movement rule including step size and/or movement direction, and the movement rule is used to control the calibration plate to move sequentially to the plurality of target positions.

7. The method according to any one of claims 1 to 4, wherein the control command includes a plurality of first control commands, the first control commands being used to control the calibration board to move from its current target position to the next target position, and the coordinates of the next target position carried by the first control command are determined based on the first point pair information corresponding to the target position of the calibration board.

8. The method according to any one of claims 1 to 7, wherein the first device and the second device are both deployed on the first vehicle, and the plurality of target locations are all located within the detection area of the first vehicle;

The multiple target locations are distributed across L detection planes, where L is a positive integer. All L detection planes are perpendicular to the ground plane and are parallel to each other. For each of the L detection planes, the multiple target locations within the detection planes have different heights.

9. The method according to claim 8, wherein when L is an integer greater than 1, the L detection planes are spaced apart by a second preset value in a direction perpendicular to the L detection planes.

10. The method according to any one of claims 1 to 9, characterized in that, determining the extrinsic parameters between the first device and the second device based on the N point-to-point information comprises:

Based on the N point pairs of information, Q calibration groups are determined. Each of the Q calibration groups includes n point pairs of information from the N point pairs of information, where n is the number of point pairs of information required to determine the extrinsic parameters. Furthermore, the Q calibration groups are all different from each other.

For each of the Q calibration groups, determine the alternative extrinsic parameters corresponding to the calibration group;

If Q = 1, then the alternative extrinsic parameter is used as the extrinsic parameter between the first device and the second device; or,

If Q > 1, then the calibration accuracy of the candidate extrinsic parameters corresponding to the Q calibration groups is verified according to the Q verification groups respectively. Then, the candidate extrinsic parameter with the highest calibration accuracy among the candidate extrinsic parameters corresponding to the Q calibration groups is determined as the extrinsic parameter. The Q verification groups correspond one-to-one with the Q calibration groups, and the N-n point pairs in a verification group and the n point pairs in a corresponding calibration group form the N point pairs.

11. The method according to claim 10, characterized in that, the step of verifying the calibration accuracy of the candidate extrinsic parameters corresponding to the Q calibration groups according to the Q verification groups includes:

For each of the Q verification groups, N-n initial calibration accuracies are obtained based on the N-n point pairs in the verification group and the candidate extrinsic parameters, and the calibration accuracy of the candidate extrinsic parameters is determined based on the N-n initial calibration accuracies.

12. The method according to claim 11, characterized in that, obtaining N-n initial calibration accuracies based on the N-n point pair information in the verification group and the candidate extrinsic parameters includes:

For each of the N-n point pairs, the third coordinate information is determined based on the first coordinate information in the point pair and the alternative extrinsic parameters;

The initial calibration accuracy is determined based on the coordinate deviation between the third coordinate information and the second coordinate information in the point pair information.

13. The method according to any one of claims 1 to 12, characterized in that receiving the N point pairs of information of the first image points on the calibration board includes:

The system receives the N point-to-point information sent by the control device of the first vehicle, which is connected to the first device and the second device.

14. The method according to any one of claims 1 to 13, wherein the calibration plate is circular in shape, and the first image point is the center of the calibration plate.

15. A method for calibrating external parameters, characterized in that it includes:

In response to periodic triggering or control commands, the control calibration board moves sequentially to multiple target positions.

16. The method according to claim 15, wherein the method further comprises:

Receive the control commands sent by the host computer.

17. The method according to claim 15 or 16, characterized in that the control calibration board is moved sequentially to a plurality of target positions, including:

According to the movement rules, the calibration board is controlled to move sequentially to the plurality of target positions; wherein...

The movement rules include step size and/or movement direction.

18. The method according to claim 17, wherein the control command carries the movement rule.

19. The method according to claim 15 or 16, characterized in that the control calibration board is moved sequentially to a plurality of target positions, including:

Based on the coordinates of the multiple target locations, the calibration plate is controlled to move sequentially to the multiple target locations.

20. The method according to claim 19, wherein the control command carries the coordinates of the plurality of target positions.

21. The method according to claim 15 or 16, characterized in that the control command includes a plurality of first control commands, the first control commands carrying the coordinates of the next target position, and the control calibration board sequentially moving to a plurality of target positions, including:

In response to one of the plurality of first control commands, the calibration board is controlled to move from its current target position to the next target position.

22. The method according to any one of claims 15 to 21, characterized in that the control calibration plate is moved sequentially to a plurality of target positions, comprising:

The calibration control mechanism moves the calibration plate to the plurality of target positions in the horizontal and/or vertical directions.

23. The method according to any one of claims 15 to 22, wherein the plurality of target locations are all located within the detection area of the first vehicle;

The multiple target locations are distributed across L detection planes, where L is a positive integer. The L detection planes are parallel to each other and all of them are perpendicular to the ground plane. For each of the L detection planes, the multiple target locations within the detection planes have different heights.

24. The method according to claim 23, wherein when L is an integer greater than 1, the L detection planes are spaced apart by a second preset value in the direction perpendicular to the L detection planes.

25. A method for calibrating external parameters, characterized in that it includes:

When the calibration board moves to the target position, the system acquires first detection data obtained by the first device detecting the calibration board, and second detection data obtained by the second device detecting the calibration board.

Based on the first detection data and the second detection data, point pair information of the first image point on the calibration board is determined. The point pair information includes first coordinate information and second coordinate information. The first coordinate information includes the coordinates of the first image point in the coordinate system of the first device when the calibration board moves to the target position. The second coordinate information includes the coordinates of the first image point in the coordinate system of the second device when the calibration board moves to the target position. The point pair information is used to determine the external parameters between the first device and the second device.

Send the point-to-point information.

26. The method according to claim 25, wherein determining the point pair information of the first image points on the calibration board based on the first detection data and the second detection data includes:

If the calibration board pattern reflected by the first detection data and the calibration board pattern reflected by the second detection data both have a pattern deviation from the calibration board that is less than a preset deviation, then the first coordinate information is determined based on the first detection data, and the second coordinate information is determined based on the second detection data. The first coordinate information and the second coordinate information are then used as the point pair information.

27. The method according to claim 25 or 26, wherein the calibration plate is circular in shape, and the first image point is the center of the calibration plate.

28. A host computer, characterized in that it comprises: a processor and a memory, the memory being used to store a computer program, the processor being used to call and run the computer program stored in the memory, and to perform the method as described in any one of claims 1 to 14.

29. A calibration control mechanism, characterized in that it comprises: a processor and a memory, the memory being used to store a computer program, the processor being used to call and run the computer program stored in the memory to perform the method as described in any one of claims 15 to 23.

30. A control device, characterized in that it comprises: a processor and a memory for storing a computer program, the processor for calling and running the computer program stored in the memory to perform the method as described in any one of claims 15 to 24.

31. An external parameter calibration system, characterized in that it comprises: a host computer as described in claim 28 and a calibration control device, wherein the calibration control device comprises a calibration control mechanism as described in claim 29.

32. The system according to claim 31, wherein the calibration control device further includes a calibration plate; the calibration control mechanism includes a movable device and a connecting device, one end of the connecting device is fixed to the movable device, and the other end of the connecting device is fixed to the calibration plate;

The movable device is used to control the calibration plate to move in the horizontal direction.

33. The system according to claim 32, wherein the movable device comprises a ground-based mobile device or a flying device.

34. The system according to claim 32 or 33, wherein the connecting device is a lifting device;

The lifting device is used to control the vertical movement of the calibration plate.

35. A calibration control device, characterized in that it comprises: a calibration control mechanism and a calibration plate as described in claim 29, wherein the calibration control mechanism comprises a movable device and a connecting device, one end of the connecting device is fixed to the movable device, and the other end of the connecting device is fixed to the calibration plate;

36. The device according to claim 35, wherein the movable device comprises a ground-based mobile device or a flying device.

37. The device according to claim 35 or 36, wherein the connecting device is a lifting device;

38. A chip, characterized in that it comprises: a processor for retrieving and executing computer instructions from a memory, causing a device having the chip mounted to perform the method as described in any one of claims 1 to 27.

39. A computer-readable storage medium, characterized in that it is used to store computer program instructions, said computer program causing a computer to perform the method as described in any one of claims 1 to 27.