CN119963660B

CN119963660B - External parameter calibration method, device and computer program product for photographing device

Info

Publication number: CN119963660B
Application number: CN202510444081.2A
Authority: CN
Inventors: 刘新阳; 肖俊苇; 刘新豪; 杨易华; 周玮
Original assignee: Sanechips Technology Co Ltd
Current assignee: Sanechips Technology Co Ltd
Priority date: 2025-04-10
Filing date: 2025-04-10
Publication date: 2025-07-15
Anticipated expiration: 2045-04-10
Also published as: CN119963660A

Abstract

The embodiment of the application provides an external parameter calibration method, device and computer program product for shooting equipment, which can comprise the steps of determining an overlapping area of a first input image and a second input image, wherein the first input image is a reference image, the second input image is an image under a preset visual angle of target shooting equipment, determining target characteristic points in the overlapping area, acquiring correction ranges of at least two scales set for each external parameter of the target shooting equipment, and correcting each external parameter of the target shooting equipment according to the target characteristic points and the correction ranges of each external parameter under the scales in order of the scales from large to small, wherein the correction result of each external parameter value under the previous scale is a correction standard of each external parameter value under the next scale, and the correction range of the next scale is a part of the correction range of the previous scale. Therefore, inconvenience brought to users by re-calibrating target shooting equipment returned to factories can be avoided.

Description

External parameter calibration method, device and computer program product for photographing device

Technical Field

This document relates to the field of camera external parameter calibration, and more particularly, to an external parameter calibration method, apparatus, and computer program product for a photographing apparatus.

Background

Under the use scene of a plurality of cameras, for example, in the long-term running process of an automobile, the problem that the external parameters of the cameras generate accumulated errors due to external force and external environment influence exists in a vehicle-mounted looking-around image system.

The common multi-camera external parameter recalibration (error correction) method is to return the hardware equipment to the manufacturer, and the manufacturer uses the calibration plate to recalibrate, so that the method has large limitation, is inconvenient for users and seriously affects the user experience.

Disclosure of Invention

The embodiment of the application provides an external parameter calibration method, equipment and a computer program product for shooting equipment, which are used for avoiding inconvenience brought to a user by returning hardware equipment to a manufacturer for calibration.

In order to solve the technical problems, the embodiment of the application is realized as follows:

In a first aspect, there is provided an external parameter calibration method for a photographing apparatus, the method comprising:

Determining an overlapping region of a first input image and a second input image, wherein the first input image is a reference image and the second input image is an image under a preset viewing angle of a target shooting device;

determining target feature points in the overlapping region;

acquiring correction ranges of at least two scales set for each external parameter of the target shooting equipment;

And correcting each external parameter value of the target shooting equipment according to the target characteristic point and the correction range of each external parameter under the scale according to the order of the scales from large to small, wherein the correction result of each external parameter value under the previous scale is a correction reference of each external parameter value under the next scale, and the correction range of the next scale is a part of the correction range of the previous scale.

In a second aspect, there is provided an external parameter calibration method for a photographing apparatus, the method comprising:

determining target feature points in the overlapping region;

Sequentially making i=1, 2, & gt..k, and repeating the outer reference correction step a after each assignment of i until a preset termination condition is met;

Acquiring an ith correction range set for each of a plurality of external parameters of the target shooting equipment, respectively selecting an ith correction value from the ith correction ranges corresponding to the plurality of external parameters to obtain a plurality of ith correction external parameter values corresponding to the plurality of external parameters, and correspondingly updating values of the plurality of external parameters of the target shooting equipment to the plurality of ith correction external parameter values in response to the pixel characteristics of the target characteristic point under the preset view angle of the target shooting equipment corrected according to the plurality of ith correction external parameter values, wherein the ith correction external parameter value of each external parameter is equal to the sum of the value of the external parameter before updating and the ith compensation value of the external parameter;

Wherein k is an integer of 2 or more, and for i=2, the first external parameter update for the i-th correction range is based on the last external parameter update result corresponding to the i-1-th correction range;

where, for i=2, the term "k, the i-th correction range is a part of the i-1-th correction range.

In a third aspect, there is provided an external parameter calibration apparatus for a photographing device, the apparatus comprising:

An overlapping region determining module, configured to determine an overlapping region of a first input image and a second input image, where the first input image is a reference image and the second input image is an image under a preset viewing angle of a target photographing apparatus;

a feature point determining module for determining a target feature point in the overlapping region;

A correction range acquisition module for acquiring correction ranges of at least two scales set for respective external parameters of the target photographing apparatus;

and the external parameter correction module is used for correcting each external parameter value of the target shooting equipment according to the target characteristic point and the correction range of each external parameter under the scale according to the order of the scale from large to small, wherein the correction result of each external parameter value under the previous scale is a correction reference of each external parameter value under the next scale, and the correction range of the next scale is a part of the correction range of the previous scale.

In a fourth aspect, there is provided an external parameter calibration apparatus for a photographing device, the apparatus comprising:

The external reference correction module is used for sequentially enabling i=1, 2, and the number of the external reference correction module is equal to the number of the external reference correction module, and repeating the external reference correction step a after each assignment of the i until a preset termination condition is met;

In a fifth aspect, there is provided an electronic device comprising:

A processor;

a memory for storing the processor-executable instructions;

wherein the processor is configured to execute the instructions to implement the method according to the first or second aspect.

In a sixth aspect, there is provided a computer readable storage medium, which when executed by a processor of an electronic device, causes the electronic device to perform the method of the first or second aspect.

In a seventh aspect, there is provided a computer program product comprising instructions which, when executed by a computer, perform the method of the first or second aspect.

In the embodiment of the application, the target characteristic points in the overlapping area can be automatically identified after the overlapping area of the reference image and the image of the target shooting equipment under the preset view angle is determined, and then the multi-scale correction is carried out on each external parameter of the target shooting equipment according to the target characteristic points and the correction ranges of different scales corresponding to each external parameter, so that the target shooting equipment does not need to be returned to a manufacturer for recalibration, the correction process is more convenient for a user, and the equipment use experience of the user can be improved.

Drawings

In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings that are required in the embodiments or the description of the prior art will be briefly described below, and it is obvious that the drawings in the following description are only some embodiments described in the present application, and other drawings can be obtained according to the drawings without inventive effort for a person skilled in the art.

Fig. 1 is a flowchart of an external parameter calibration method for a photographing apparatus according to an embodiment of the present application.

Fig. 2 is a view diagram of a vehicle-mounted looking-around camera system according to an embodiment of the present application.

Fig. 3 is a schematic diagram of a pixel window in the method for extracting angular feature points according to an embodiment of the present application.

Fig. 4 is a flowchart of an external parameter calibration method for a photographing apparatus according to an embodiment of the present application.

Fig. 5 is a schematic diagram of a target feature point extraction process according to an embodiment of the present application.

Fig. 6 is a flowchart of an external parameter calibration method for a photographing apparatus according to another embodiment of the present application.

Fig. 7 is a schematic structural view of an electronic device according to an embodiment of the present application.

Fig. 8 is a schematic structural diagram of an external parameter calibration device for a photographing apparatus according to an embodiment of the present application.

Fig. 9 is a schematic structural diagram of an external parameter calibration device for a photographing apparatus according to another embodiment of the present application.

Detailed Description

In order that those skilled in the art will better understand the technical solutions of the embodiments of the present application, the technical solutions of the embodiments of the present application will be clearly and completely described below with reference to the drawings in one or more embodiments of the present application, and it is apparent that the described embodiments are only some embodiments of the present application, not all embodiments. All other embodiments, which can be made by those skilled in the art based on the embodiments of the application without making any inventive effort, shall fall within the scope of protection of this document.

The terms first, second and the like in the description and in the claims, are used for distinguishing between similar objects and not necessarily for describing a particular sequential or chronological order. It is to be understood that the data so used may be interchanged where appropriate such that embodiments of the application may be practiced otherwise than as specifically illustrated or described herein. Furthermore, in the present application and in the claims, "and/or" means at least one of the connected objects, and the character "/" generally means that the associated object is an or relationship.

Although some camera external reference line calibration methods without returning to factories are also proposed in the related art, each of these methods has disadvantages and needs improvement. For example, the multi-camera external reference line calibration method based on image registration uses a large number of features of an overlapping area to perform registration correction, and the algorithm calculation amount and time consumption are large.

In order to avoid inconvenience brought to users by returning target shooting equipment to a manufacturer for calibration and to overcome at least one of the defects of a camera external reference line calibration method without returning to the manufacturer in the related art, the application provides an external reference calibration method, equipment and a computer program product for shooting equipment. The method may be performed by the electronic device or software installed in the electronic device. The electronic equipment comprises any one of intelligent equipment such as a smart phone, a personal computer (personal computer, PC), a notebook computer, a tablet personal computer, an electronic reader, a network television and a wearable device, but is not limited to the intelligent equipment.

The external parameter calibration method for the shooting equipment provided by the embodiment of the application can be applied to, but is not limited to, a multi-camera scene, such as a vehicle-mounted looking-around image system, an intelligent driving auxiliary system and the like. Under the condition that the multi-camera system is externally referred to receive the error image, the multi-camera system can be subjected to external parameter error correction online without returning to an after-sales workshop. As an example, the external parameter calibration method for a photographing device according to the embodiment of the present application may be used as an algorithm application or a software program in a vehicle-mounted looking-around software solution, and run on hardware such as a vehicle-mounted digital signal Processor (DIGITAL SIGNAL Processor, DSP), a graphics Processor (Graphic Processing Unit, GPU), a central processing unit (Central Processing Unit, CPU), and the like.

The following describes an external parameter calibration method for a photographing apparatus according to an embodiment of the present application with reference to the accompanying drawings.

As shown in fig. 1, an external parameter calibration method for a photographing apparatus according to an embodiment of the present application may include:

Step 101, determining an overlapping area of a first input image and a second input image, wherein the first input image is a reference image, and the second input image is an image under a preset viewing angle of a target photographing apparatus.

The target photographing apparatus is the apparatus to be corrected.

In some embodiments, the preset viewing angle may be a bird's eye view angle (Birds Eyes View, BEV), the first input image may be a bird's eye view converted from a first image captured by a reference capturing device, and the second input image may be a bird's eye view converted from a second image captured by the target capturing device, wherein capturing ranges of the reference capturing device and the target capturing device overlap, and the first image and the second image are captured by the reference capturing device and the target capturing device simultaneously from different viewing angles for the same scene.

Taking an in-vehicle looking around camera system as an example, the reference photographing apparatus may be one of the cameras, and the target photographing apparatus may be an adjacent one of the cameras. Fig. 2 shows a schematic view of the shooting range (field of view) of the in-vehicle looking around camera system including front-back, left-right, and four-way cameras. As shown in fig. 2, there is an overlapping of imaging fields between two four cameras, and the reference camera is used as a reference for correcting the external parameters of the right-view camera or the left-view camera (target shooting device), and then the reference camera is used as a reference for correcting the external parameters of the rear-view camera (target shooting device).

Optionally, in the case where the reference photographing apparatus and the target photographing apparatus are fish-eye cameras, before step 101, the method may further include:

De-distorting the first image and the second image;

Converting the de-distorted first image into the first input image;

and converting the undistorted second image into the second input image.

Further, under the condition that the preset visual angle is a bird's-eye view angle, converting the first image after distortion removal into an image under the bird's-eye view angle to obtain the first input image, and converting the second image after distortion removal into an image under the bird's-eye view angle to obtain the second input image.

In some embodiments, the de-distorting the first image and the second image may include de-distorting the first image with distortion parameters of a reference capture device (typically provided by a camera vendor) and internal parameters, and de-distorting the second image with distortion parameters of a target capture device (typically provided by a camera vendor) and internal parameters.

In some embodiments, the converting the first image after de-distortion into an image under the aerial view angle to obtain the first input image may include obtaining an aerial view image with an error, that is, the first input image, by using external parameters of the reference photographing device and aerial view angle initialization parameters.

In some embodiments, the converting the undistorted second image into an image under the aerial view angle to obtain the second input image may include obtaining an aerial view image with an error, that is, the second input image, by using initial external parameters and aerial view angle initialization parameters of the target photographing device.

The process of converting the de-distorted image into an image at the aerial view angle comprises the steps of based on the de-distorted image and camera internal parametersExternal parameters of cameraBird's eye view angle transformation matrixAll pixels of the undistorted image are converted into pixels under the aerial view angle, and the aerial view is obtained.

Wherein, for any distorted fisheye image pixel IThe pixel coordinate B of the pixel at the aerial view angle can be calculatedThe calculation formula is as follows:

Where z may take a constant, e.g Is an internal reference of the fish-eye camera,Is an external parameter of the fish-eye camera,Is a transformation parameter for converting the aerial view angle into the shooting view angle of the camera under the world coordinate system.

Optionally, the first input image and the second input image converted into the bird's eye view angle are cropped to obtain a required image area, and then an overlapping area of the two is calculated. The pixels of the two cut images are not zero in the overlapping area, so that according to the principle, the area where the pixels in the first input image and the second input image are not zero in the bird's eye view angle can be calculated, and the overlapping area is obtained. Optionally, preprocessing such as binarization and morphological operation is performed on the overlapped area, noise points with possibly zero pixel values are removed, a Mask (Mask) of the overlapped area is obtained, then etching operation is performed on the Mask image, and edge areas are removed, so that a clear and noiseless image of the overlapped area is obtained.

And 102, determining target feature points in the overlapped area.

Optionally, prior to step 102, the method may further comprise pre-processing, such as filtering, the overlapping region of the first input image to further eliminate noise of the overlapping region. Specifically, a 3×3 gaussian filter kernel is used to perform gaussian filtering on the overlapping region of the first input image, so as to reduce noise in the overlapping region.

Wherein the target feature points are sparse feature points in the overlapping region instead of all pixel points, e.g. the target feature points may be at least one of edge feature points and corner feature points in the overlapping region.

In some embodiments, the target feature points include edge feature points and angle feature points, and step 102 may include extracting edge feature points from the overlapping region of the first input image based on a Sobel operator, and extracting angle feature points from the overlapping region of the first input image based on a Harris feature angle feature point detection algorithm.

The edge feature extraction based on the Sobel operator comprises gradient calculation in the horizontal direction (X direction) and gradient calculation in the vertical direction (Y direction). The first-order Sobel operator in the X direction and the first-order operator in the Y direction are respectively:

The Harris feature angle feature point detection algorithm is to calculate the content difference value in the neighborhood region of the pixel to determine whether the pixel is an angle feature point, and generally refers to a pixel window with larger content difference from the surrounding. The difference value obtained by moving the pixel window in any direction on the image is called a window response value E, and the calculation formula is as follows:

as shown in fig. 3, the pixel window 31 is a window before moving The pixel window 32 is a shifted window,Indicating the direction of movement of the window,The weights of the different pixels are represented, typically by a gaussian weighting function. The above formula calculates the square of the gray value difference of each pixel before and after the movement, and sums the difference of the contents as two windows according to the weight of each pixel point. And judging E obtained by calculating a certain pixel position according to the set threshold value, if the E exceeds the set threshold value, the current pixel is the angular feature point position, otherwise, the current pixel is not the angular feature point position. The angular feature point location should be characterized by the current windowThe calculated difference value E is large moving in either direction.

And after the coordinates of the edge feature points and the coordinates of the angle feature points are obtained through calculation, removing the repeated coordinates, and obtaining a sparse coordinate set R of the target feature points.

It can be understood that 1) compared with the external parameter correction scheme of the look-around camera by utilizing lane line characteristics in the related art, the embodiment of the application can use any natural scene to extract the characteristics of the camera overlapping view angle area, is not limited to the lane lines, and can further reduce the limitation of external parameter calibration, and 2) does not use all pixels under the overlapping view angle to perform external parameter calibration, but extracts sparse edge characteristic points and angle characteristic points with larger gradients to perform external parameter calibration, so that the integral calculated amount in the whole calibration process can be reduced, the calculation resource consumption is reduced, and the calibration efficiency is improved.

In some embodiments, the external parameter calibration method for the photographing device provided by the application further comprises the steps of determining the brightness modulation coefficient according to the brightness information of the overlapped area in the first input image and the brightness information of the overlapped area in the second input image, wherein the brightness modulation coefficient can be used for eliminating brightness difference in two images subsequently, reducing errors caused by exposure of different cameras, and preventing the brightness difference from causing distortion of pixel characteristic distance, so that correction accuracy is prevented from being influenced.

In some embodiments, the determining the brightness modulation factor according to the brightness information of the overlapping area in the first input image and the brightness information of the overlapping area in the second input image may include determining a first statistical result of brightness of each pixel of the overlapping area in the first input image, determining a second statistical result of brightness of each pixel of the overlapping area in the second input image, and determining the brightness modulation factor according to the first statistical result and the second statistical result.

The first statistical result and the second statistical result are the same type of statistical result, for example, if the first statistical result is median, the second statistical result is median, and if the first statistical result is mean, the second statistical result is mean.

In one example, the first statistical result is a mean value of luminance of each pixel of the overlapping region in the first input image, and the second statistical result is a mean value of luminance of each pixel of the overlapping region in the second input image. Specifically, for each pixel in the overlapping region in the first input image or the second input image after masking, the luminance modulation factor luma_ratio is calculated by the following formula:

Luma_ration = Mean(image_f × Mask) / Mean(image_r × Mask)

Wherein image_f is the first input image, image_r is the second input image, mask is an overlapping region Mask, and Mean () is an operation of averaging all pixels of the region image.

Step 103, obtaining a correction range of at least two scales set for each external parameter of the target shooting device.

Under the same scale, the correction ranges of the same kind of external parameters can be the same or different.

And 104, correcting each external parameter value of the target shooting equipment according to the target characteristic point and the correction range of each external parameter under the scale according to the order of the scales from large to small, wherein the correction result of each external parameter value under the previous scale is a correction reference of each external parameter value under the next scale, and the correction range of the next scale is a part of the correction range of the previous scale.

For example, for a camera, its parameters include three translation parameter vectors and three rotation parameter vectors, where the three translation parameter vectors belong to the same class and the three rotation parameter vectors belong to the same class, and these six parameters can be expressed as, wherein,A vector containing three translation parameters is represented,Representing a vector containing three rotation parameters. In one example, "each of the external parameters" in step 103 refers to the six external parameters.

In order to improve the correction effect, the embodiment of the application sets at least two scale correction ranges for each external parameter respectively, and adopts repeated iterative correction from coarse scale to fine scale (from large scale to small scale), so that the external parameter correction with coarse granularity can be carried out in the early stage, the robustness of larger external parameter errors can be ensured, and the external parameter correction with fine granularity can be carried out in the later stage, and the accuracy of the correction result can be ensured.

As an example, assuming that correction ranges of 2 scales are set for the above six external parameters, a first external parameter correction is performed at a first scale with a larger scale (one external parameter correction includes multiple iterations), and then a second external parameter correction is performed at a second scale with a smaller scale, where the correction ranges of three rotation parameters are (-3, +3) degrees, the correction ranges of three translation parameters are (-0.1,0.1) meters, and the correction ranges of three rotation parameters are (-0.5, +0.5) degrees, and the correction ranges of three translation parameters are (-0.02,0.02) meters, at the second scale, it can be seen that the correction range of the second scale is a part of the correction range of the first scale for the same external parameter. Then, for each external reference correction process at the first scale, any offset value of the translational external referenceCompensation value of any rotation external parameterAfter the iterative search is completed (3, 3), updating each external parameter value of the target shooting equipment by utilizing the final result (optimal each external parameter correction value), and then carrying out iterative search of the external parameter under the second scale, namely, the correction result of each external parameter value under the previous scale is the correction reference of each external parameter value under the next scale, and for each external parameter correction process under the second scale, any compensation value of the translation external parameterCompensation value of any rotation external parameterAfter the iterative search is completed, the final result (optimal external correction values) is used to update the external parameters of the target photographing device. Through the correction strategy of the scale from thick to thin, the robustness of the algorithm to larger errors can be enhanced, and meanwhile, the correction precision is ensured.

In some embodiments, correcting the external parameter values of the target photographing device according to the target feature points and the correction ranges of the external parameters under the scales for each scale includes performing a first specified step in a circulating manner for each scale until a preset termination condition is met.

The preset termination condition may include, but is not limited to, at least one of reaching a preset iteration number upper limit, reaching a preset iteration time consuming upper limit, exhausting compensation values in the plurality of compensation value lists, and the like.

Wherein the first specifying step includes:

Respectively selecting one external parameter compensation value from the correction range of each external parameter under the scale to obtain a group of external parameter compensation values;

Determining a corrected extrinsic value of the target photographing apparatus according to the uncorrected extrinsic value of the target photographing apparatus and the set of extrinsic compensation values;

and updating the uncorrected extrinsic parameter of the target shooting equipment to the corrected extrinsic parameter in response to the pixel characteristics of the target characteristic point under the corrected preset viewing angle of the target shooting equipment meeting a preset correction condition.

And respectively selecting an external parameter compensation value from the correction range of each external parameter under the scale, wherein the external parameter compensation value comprises respectively randomly extracting an external parameter compensation value from a plurality of compensation value lists of each external parameter under the scale to obtain a group of external parameter compensation values, and one external parameter corresponds to one compensation value list determined according to the correction range of the external parameter under one scale. And extracting an external parameter compensation value from the compensation value list of each external parameter under the scale according to the iteration times under the scale to obtain a group of external parameter compensation values.

In some embodiments, before selecting one extrinsic compensation value from the correction range of each extrinsic parameter under the scale, the method further includes setting extrinsic compensation values according to a preset interval under the correction range corresponding to the scale, so as to obtain the multiple compensation value lists.

For example, as assumed above, two calibration ranges of the respective external parameters are set, and at the first scale, the calibration ranges of the three rotation parameters are (-3, +3) degrees, the calibration ranges of the three translation parameters are (-0.1,0.1) meters, and at the second scale, the calibration ranges of the three rotation parameters are (-0.5, +0.5) degrees, and the calibration ranges of the three translation parameters are (-0.02,0.02) meters. Then, for the first scale, an equally divided preset interval may be set, and if the preset interval is 0.1, a compensation value list corresponding to the rotation parameter may be calculated asAnd calculating the compensation value list corresponding to the translation parameters. In the six compensation value lists formed by 3 rotation compensation values and 3 translation compensation values, one compensation value is randomly extracted at a time to form a group of external parameter compensation values. Take uncorrected external parameter value asThe group of external parameter compensation values obtained by randomly extracting one compensation value from the six compensation value lists areFor example, the corrected extrinsic values may be expressed as。

In some embodiments, the updating the uncorrected extrinsic parameter of the target photographing apparatus to the corrected extrinsic parameter in response to the pixel feature of the target feature point at the corrected preset viewing angle of the target photographing apparatus satisfying a preset correction condition may include:

Acquiring a first pixel characteristic distance, wherein the first pixel characteristic distance is the pixel characteristic distance of the target characteristic point at the preset visual angle of the reference shooting equipment and the target shooting equipment before updating an external parameter value;

determining a second pixel characteristic distance of the target characteristic point at the preset viewing angles of the reference photographing device and the target photographing device based on the corrected external parameter value;

and in response to the second pixel characteristic distance being less than the first pixel characteristic distance, updating the uncorrected extrinsic value of the target capture device to the corrected extrinsic value.

Optionally, the method of FIG. 1 further comprises, responsive to the second pixel feature distance being greater than or equal to the first pixel feature distance, ignoring and/or recording the corrected extrinsic values, re-executing the first prescribed step, i.e., continuing with the next iteration.

In some embodiments, where the first specifying step is performed for the first time for a maximum scale, the obtaining a first pixel feature distance may include:

determining a pixel value of the target feature point in the first input image as a first pixel value;

Determining a pixel value of the target feature point in the second input image as a second pixel value;

and determining a first pixel characteristic distance according to the first pixel value and the second pixel value.

It should be noted that, for the first correction under the maximum scale, the first pixel feature distance can be calculated directly according to the pixel values of the target feature points in the first input image and the second input image, and may not be calculated according to the uncorrected extrinsic parameter value, but may be calculated according to the uncorrected extrinsic parameter value, and in other correction processes, if the first pixel feature distance is updated to the second pixel feature distance in the last correction process, the first pixel feature distance is the second pixel feature distance calculated last time, and no recalculation is required.

In some embodiments, the determining, based on the corrected extrinsic parameter value, a second pixel feature distance of the target feature point at the preset viewing angles of the reference photographing apparatus and the target photographing apparatus may include:

determining a third pixel value of the target feature point under the preset visual angle of the corrected target shooting equipment according to the corrected external parameter value;

and determining a second pixel characteristic distance of the target characteristic point at the preset visual angles of the reference shooting equipment and the target shooting equipment according to the first pixel value and the third pixel value.

As an example, the absolute value of the difference of the pixel values of two pixels may be taken as the characteristic distance of the two pixels. That is, the first pixel characteristic distance may be an absolute value of a difference between the first pixel value and the second pixel value, and the second pixel characteristic distance may be an absolute value of a difference between the first pixel value and the third pixel value. It will be appreciated that the pixel feature distance may represent an error between two pixels, i.e. the first pixel feature distance may represent a pixel value error of the target feature point at a preset viewing angle of the reference photographing apparatus and the target photographing apparatus before correction, and the second pixel feature distance may represent a pixel value error of the target feature point at a preset viewing angle of the reference photographing apparatus and the target photographing apparatus after correction, and if the error is reduced (i.e. loss is reduced) before and after correction, the correction is effective.

In other embodiments, the determining the first pixel characteristic distance from the first pixel value and the second pixel value includes determining the first pixel characteristic distance from the first pixel value, a brightness modulation factor, and the second pixel value, wherein the brightness modulation factor is determined from a brightness difference of the overlapping region in the first input image and the second input image. Correspondingly, the determining the second pixel characteristic distance of the target characteristic point under the preset visual angles of the reference shooting equipment and the target shooting equipment according to the first pixel value and the third pixel value comprises determining the second pixel characteristic distance of the target characteristic point under the preset visual angles of the reference shooting equipment and the target shooting equipment according to the first pixel value, the brightness modulation coefficient and the third pixel value.

It will be appreciated that introducing the brightness modulation factor in determining the first and second pixel feature distances may reduce errors introduced by different camera exposures to prevent differences in brightness from causing distortion in the pixel feature distances, thereby avoiding affecting the accuracy of the correction.

In some embodiments, the first specifying step may further include updating the first pixel feature distance to the second pixel feature distance in response to the second pixel feature distance being less than the first pixel feature distance.

In the above example, the iterative logic of each extrinsic parameter is based first on uncorrected extrinsic parametersCalculating a first pixel characteristic distance of the target characteristic point at the preset viewing angles of the reference photographing device and the target photographing device, the calculation formula can be:

Wherein, the Representing coordinates of the target feature point in the first input image,Representing coordinates of the target feature point at a preset viewing angle of the target photographing apparatus,Representing the pixel value in the first input image of the target feature point-the first pixel value,A second pixel value representing a pixel value of the target feature point at a preset viewing angle of the target photographing apparatus,Is the normalized weight coefficient of the weight of the model,Is the luminance modulation factor calculated previously.

Similarly, the correction values of each external reference are obtainedThen, mapping the coordinates of the target feature points under the corrected preset view angle of the target shooting equipment to obtain the coordinates of the target feature points under the corrected preset view angle of the target shooting equipmentAnd then calculating to obtain a second pixel characteristic distance by using the following formula:

at the time of calculating the first pixel characteristic distance And a second pixel feature distanceThe logic of updating the correction values of each external reference is that ifUpdating each external parameter value of the target shooting equipment to be a corrected external parameter valueThen updateOrder-makingIf (3)Ignoring and/or recordingAnd proceeds to the next iteration.

And repeatedly performing external parameter correction and characteristic distance comparison in the set correction iteration times. And finally, calculating to obtain a group of updated external parameter values, wherein the external parameter value combination with the minimum characteristic distance is the optimal result of current round parameter correction.

According to the external parameter correction method for the shooting equipment, the target characteristic points in the overlapping area can be automatically identified after the overlapping area of the reference image and the image of the target shooting equipment under the preset view angle is determined, then the external parameters of the target shooting equipment are subjected to multi-scale correction according to the target characteristic points and correction ranges of different scales corresponding to the external parameters, and the target shooting equipment is not required to be returned to be recalibrated, so that the correction process is more convenient for a user, and the equipment use experience of the user can be improved.

In addition, compared with the external parameter correction scheme of the panoramic camera by utilizing lane line characteristics in the related art, the external parameter correction method for the photographing equipment provided by the embodiment of the application has the advantages that 1) the external parameter correction scheme of the panoramic camera can be used for extracting the characteristics of the camera overlapping view angle area by using any natural scene, is not limited to lane lines, and therefore, the limitation of external parameter calibration can be further reduced, 2) the external parameter calibration is carried out by not using all pixels under the overlapping view angle, but extracting sparse edge characteristic points and angle characteristic points with larger gradients, so that the whole calculated amount in the whole calibration process can be reduced, the calculation resource consumption is reduced, the calibration efficiency is improved, and 3) the brightness modulation coefficient is introduced when the first pixel characteristic distance and the second pixel characteristic distance are determined, the errors introduced by different camera exposures can be reduced, the distortion of the pixel characteristic distances caused by brightness differences can be prevented, and the influence on the correction precision can be avoided.

Fig. 4 shows a flowchart of an external parameter calibration method for a photographing apparatus according to an embodiment of the present application. The external parameter calibration method for the photographing apparatus according to the present application will be described again with reference to fig. 4.

Taking a fisheye camera common to a panoramic system as an example of a multi-view camera, as shown in fig. 4, the method may include 1) inputting multi-view images, such as inputting a first image captured by a reference camera and a second image captured by a target camera, 2) importing distortion parameters, internal parameters and initial external parameters of the multi-view camera, 3) de-distorting the first image and the second image, converting the de-distorted first image and image into a bird's eye view to obtain the first input image and the second input image, 4) extracting target feature points, as shown in fig. 5, and the method may specifically include determining an overlapping area, image exposure coordination (such as determining a brightness modulation coefficient), overlap area preprocessing, calculation of edge feature points and angle feature points, and feature table screening (such as removing the repeated coordinates), 5) multi-scale external parameter searching and correction, and loss (such as pixel feature of the target feature points under the preset viewing angle of the target photographing device after correction) and external parameter updating, and 6) outputting the result. The detailed implementation procedures of 1) to 6) described herein can refer to the corresponding descriptions above, and are not repeated here,

A method for calibrating an external parameter for a photographing apparatus according to the present application will be described from another point of view by way of an embodiment shown in fig. 6.

As shown in fig. 6, an external parameter calibration method for a photographing apparatus according to another embodiment of the present application may include:

Step 601, determining an overlapping area of a first input image and a second input image, wherein the first input image is a reference image, and the second input image is an image under a preset viewing angle of a target shooting device;

step 602, determining target feature points in the overlapping area.

The specific implementation procedures of step 601 and step 602 correspond to the specific implementation procedures of step 101 and step 102 in the embodiment shown in fig. 1, and the same technical effects can be achieved, and the description will not be repeated here.

Step 603, let i=1, 2, &.. and repeating the outer reference correction step a after each assignment of i until a preset termination condition is met.

The step a of external parameter correction may include obtaining an ith correction range set for each of a plurality of external parameters of the target photographing apparatus, respectively selecting an external parameter compensation value from the ith correction ranges corresponding to the plurality of external parameters, obtaining a plurality of ith correction external parameter values corresponding to the plurality of external parameters, and updating values of the plurality of external parameters of the target photographing apparatus to the plurality of ith correction external parameter values in response to a pixel characteristic of the target characteristic point under the preset viewing angle of the target photographing apparatus corrected according to the plurality of ith correction external parameter values satisfying a preset correction condition, wherein the ith correction external parameter value of each external parameter is equal to a sum of a value of the external parameter before updating and the ith compensation value of the external parameter.

As an example, assuming that k=2, i.e., correction ranges of 2 scales are set for the above six extrinsic parameters, first, let i=1 perform the first extrinsic correction in the 1 st correction range of larger scale (one extrinsic correction includes multiple iterations), then let i=2 perform the second extrinsic correction in the 2 nd correction range of smaller scale, where the 1 st correction range of three rotation parameters is (-3, +3) degrees, the correction ranges of three translation parameters are (-0.1,0.1) meters, the 2 nd correction range of three rotation parameters is (-0.5, +0.5) degrees, and the 2 nd correction range of three translation parameters is (-0.02,0.02) meters, it can be seen that the 2 nd correction range is a part of the 1 st correction range for the same extrinsic parameter. Then, for each external reference correction process under the correction range 1, any offset value of the translation external referenceCompensation value of any rotation external parameterAfter completion of the iterative search (3, 3), the final result (optimal values of the extrinsic parameters) is used to update the extrinsic parameters of the target camera, and then an iterative search of the extrinsic parameters is performed in the 2 nd correction range, i.e. the i-th correction range is a part of the i-1 th correction range, for one extrinsic correction process in the 2 nd correction range, the compensation value of any translational extrinsic parameter is usedCompensation value of any rotation external parameterAfter the iterative search is completed, the final result (optimal external correction values) is used to update the external parameters of the target photographing device. The robustness of the algorithm to larger errors can be enhanced through a correction strategy of the correction range from thick to thin, and meanwhile, the correction precision is ensured.

The selecting an extrinsic compensation value from the ith correction range corresponding to the extrinsic parameters may include randomly extracting an extrinsic compensation value from a plurality of ith compensation value lists corresponding to the extrinsic parameters, to obtain a plurality of ith correction extrinsic values corresponding to the extrinsic parameters, where one extrinsic parameter corresponds to an ith compensation value list determined according to the ith correction range of the extrinsic parameter.

In some embodiments, for a list of i-th compensation values for which the extrinsic parameters correspond to a list of i-th compensation values determined according to an i-th correction range for the extrinsic parameters, the list of i-th compensation values is a series of arithmetic differences determined according to the i-th correction range.

For example, as assumed above, k=2, i.e., two scale correction ranges are set for each external parameter, respectively, in the first scale, the 1 st correction range of the three rotation parameters is (-3, +3) degrees, the 1 st correction range of the three translation parameters is (-0.1,0.1) meters, and in the second scale, the 2 nd correction range of the three rotation parameters is (-0.5, +0.5) degrees, and the 2 nd correction range of the three translation parameters is (-0.02,0.02) meters. Then, for the 1 st correction range, an equally divided preset interval may be set, and if the preset interval is 0.1, a compensation value list corresponding to the rotation parameter may be calculated asAnd calculating the compensation value list corresponding to the translation parameters. And randomly extracting one compensation value at each time in a six compensation value list formed by 3 rotation compensation values and 3 translation compensation values to obtain six ith correction external parameter values corresponding to the six external parameters. Take uncorrected external parameter value asThe six i-th correction external parameter values obtained by randomly extracting one compensation value from the six compensation value lists are respectivelyFor example, the corrected extrinsic values may be expressed as。

Wherein the responding to the pixel characteristic of the target feature point under the preset view angle of the target shooting equipment corrected according to the ith correction external parameter value meeting a preset correction condition, correspondingly updating the values of the multiple external parameters of the target shooting equipment to the multiple ith correction external parameter values comprises the following steps:

determining a second pixel characteristic distance of the target characteristic point at the preset viewing angles of the reference photographing device and the target photographing device based on the i-th correction extrinsic values;

and in response to the second pixel characteristic distance being less than the first pixel characteristic distance, correspondingly updating values of the plurality of external parameters of the target shooting device to the plurality of ith corrected external parameter values.

For i=1, and the extrinsic correction step a is performed for the first time, the obtaining a first pixel feature distance includes:

Wherein the determining, based on the plurality of i-th corrected extrinsic values, a second pixel feature distance of the target feature point at the preset viewing angles of the reference photographing apparatus and the target photographing apparatus includes:

Determining a third pixel value of the target feature point under the preset view angle of the target shooting equipment according to the i-th correction external parameter values;

Optionally, the outlier correction step a may further include updating the first pixel feature distance to the second pixel feature distance in response to the second pixel feature distance being less than the first pixel feature distance.

The foregoing describes certain embodiments of the present application. Other embodiments are within the scope of the following claims. In some cases, the actions or steps recited in the claims can be performed in a different order than in the embodiments and still achieve desirable results. In addition, the processes depicted in the accompanying figures do not necessarily require the particular order shown, or sequential order, to achieve desirable results. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

Fig. 7 is a schematic structural view of an electronic device according to an embodiment of the present application. Referring to fig. 7, at the hardware level, the electronic device includes a processor, and optionally an internal bus, a network interface, and a memory. The Memory may include a Memory, such as a Random-Access Memory (RAM), and may further include a non-volatile Memory (non-volatile Memory), such as at least 1 disk Memory. Of course, the electronic device may also include hardware required for other services.

The processor, network interface, and memory may be interconnected by an internal bus, which may be an ISA (Industry Standard Architecture ) bus, a PCI (PERIPHERAL COMPONENT INTERCONNECT, peripheral component interconnect standard) bus, or EISA (Extended Industry Standard Architecture ) bus, among others. The buses may be classified as address buses, data buses, control buses, etc. For ease of illustration, only one bi-directional arrow is shown in FIG. 7, but not only one bus or type of bus.

And the memory is used for storing programs. In particular, the program may include program code including computer-operating instructions. The memory may include memory and non-volatile storage and provide instructions and data to the processor.

The processor reads the corresponding computer program from the nonvolatile memory to the memory and then runs the computer program to form the external parameter calibration device on the logic level. The processor is used for executing the programs stored in the memory and is specifically used for executing the following operations:

determining target feature points in the overlapping region;

Or the processor reads the corresponding computer program from the nonvolatile memory to the memory and then runs the computer program to form the external parameter calibration device on the logic level. The processor is used for executing the programs stored in the memory and is specifically used for executing the following operations:

determining target feature points in the overlapping region;

The method performed by the external reference calibration device disclosed in the embodiment of fig. 7 of the present application may be applied to a processor or implemented by a processor. The processor may be an integrated circuit chip having signal processing capabilities. In implementation, the steps of the above method may be performed by integrated logic circuits of hardware in a processor or by instructions in the form of software. The Processor may be a general-purpose Processor including a central processing unit (Central Processing Unit, CPU), a network Processor (Network Processor, NP), etc., or may be a digital signal Processor (DIGITAL SIGNAL Processor, DSP), application SPECIFIC INTEGRATED Circuit (ASIC), field-Programmable gate array (Field-Programmable GATE ARRAY, FPGA) or other Programmable logic device, discrete gate or transistor logic device, discrete hardware components. The disclosed methods, steps, and logic blocks in the embodiments of the present application may be implemented or performed. A general purpose processor may be a microprocessor or the processor may be any conventional processor or the like. The steps of the method disclosed in connection with the embodiments of the present application may be embodied directly in the execution of a hardware decoding processor, or in the execution of a combination of hardware and software modules in a decoding processor. The software modules may be located in a random access memory, flash memory, read only memory, programmable read only memory, or electrically erasable programmable memory, registers, etc. as well known in the art. The storage medium is located in a memory, and the processor reads the information in the memory and, in combination with its hardware, performs the steps of the above method.

The electronic device may also execute the method of fig. 1 or fig. 6, and implement the functions of the external parameter calibration device in the embodiment shown in fig. 1 or fig. 6, which is not described herein again.

Of course, other implementations, such as a logic device or a combination of hardware and software, are not excluded from the electronic device of the present application, that is, the execution subject of the following processing flows is not limited to each logic unit, but may be hardware or a logic device.

Embodiments of the present application also provide a computer-readable storage medium storing one or more programs, the one or more programs comprising instructions, which when executed by a portable electronic device comprising a plurality of target application programs, enable the portable electronic device to perform the method of the embodiments shown in fig. 1 or 6.

The embodiments of the present application also provide a computer program product comprising instructions which, when executed by a computer, perform a method of calibrating a parameter as shown in fig. 1 or 6.

Fig. 8 is a schematic structural diagram of an external parameter calibration device 800 according to an embodiment of the present application. Referring to fig. 8, in a software implementation, the external parameter calibration device 800 may include an overlap region determination module 801, a feature point determination module 802, a correction range acquisition module 803, and an external parameter correction module 804.

An overlapping area determining module 801 is configured to determine an overlapping area of a first input image and a second input image, where the first input image is a reference image and the second input image is an image under a preset viewing angle of a target photographing apparatus.

Wherein the target photographing apparatus is the apparatus to be corrected.

Taking an in-vehicle looking around camera system as an example, the reference photographing apparatus may be one of the cameras, and the target photographing apparatus may be an adjacent one of the cameras.

Optionally, the apparatus 800 may further include a de-distortion module and an image perspective conversion module.

And the de-distortion module is used for de-distorting the first image and the second image before determining the overlapping area of the first input image and the second input image.

The image visual angle conversion module is used for converting the first image after de-distortion into the first input image and converting the second image after de-distortion into the second input image.

Further, the image view angle conversion module is used for converting the first image after distortion removal into an image under the bird's-eye view angle to obtain the first input image when the preset view angle is the bird's-eye view angle, and converting the second image after distortion removal into the image under the bird's-eye view angle to obtain the second input image.

In some embodiments, the de-distortion module is specifically operable to de-distort the first image using the distortion parameters of the reference capture device (typically provided by the camera vendor) and the internal parameters and to de-distort the second image using the distortion parameters of the target capture device (typically provided by the camera vendor) and the internal parameters.

In some embodiments, the image perspective conversion module is specifically operable to:

Obtaining an aerial view angle image with errors, namely a first input image, by utilizing external parameters and aerial view angle initialization parameters of reference shooting equipment;

And obtaining an error-bearing aerial view angle image, namely a second input image, by utilizing the initial external parameters and aerial view angle initialization parameters of the target shooting equipment.

Optionally, the apparatus 800 may further include a cropping module configured to crop the first input image and the second input image converted into the bird's eye view angle to obtain a desired image area, and then calculate an overlapping area of the two. The pixels of the two cut images are not zero in the overlapping area, so that according to the principle, the area where the pixels in the first input image and the second input image are not zero in the bird's eye view angle can be calculated, and the overlapping area is obtained. Optionally, the apparatus 800 may further include an overlap region enhancement module, configured to perform binarization and morphological operations on the overlap region, and reject noise points with possibly zero pixel values to obtain a Mask (Mask) of the overlap region, and then perform corrosion operations on the Mask image, and reject edge regions to obtain a clear and noiseless image of the overlap region.

A feature point determining module 802, configured to determine a target feature point in the overlapping region.

Optionally, the apparatus 800 may further comprise an overlap region preprocessing module, configured to perform preprocessing, such as filtering, on the overlap region of the first input image before determining the target feature point in the overlap region, so as to further eliminate noise of the overlap region.

Wherein the target feature points are sparse feature points in the overlapping region, for example, the target feature points may be at least one of edge feature points and corner feature points in the overlapping region.

In some embodiments, the target feature points include edge feature points and angle feature points, and feature point determination module 802 is operable to extract edge feature points from the overlapping region of the first input image based on a Sobel operator, and to extract angle feature points from the overlapping region of the first input image based on a Harris feature angle feature point detection algorithm.

In some embodiments, the apparatus 800 may further include an exposure coordination module configured to determine the brightness modulation factor according to brightness information of the overlapping region in the first input image and brightness information of the overlapping region in the second input image, where the brightness modulation factor may be used to subsequently eliminate brightness differences in the two images, and reduce errors introduced by exposure of different cameras, so as to prevent the brightness differences from causing distortion of the pixel feature distance, thereby avoiding affecting correction accuracy.

A correction range obtaining module 803 is configured to obtain a correction range of at least two scales set for each external parameter of the target capturing apparatus.

And an extrinsic parameter correction module 804, configured to correct, for each scale, the extrinsic parameter values of the target photographing device according to the target feature point and the correction range of each extrinsic parameter under the scale according to the order of the scale from large to small, where the correction result of each extrinsic parameter under the previous scale is a correction reference of each extrinsic parameter under the next scale, and the correction range of the next scale is a part of the correction range of the previous scale.

In order to improve the correction effect, the embodiment of the application sets at least two scale correction ranges for each external parameter respectively, and adopts repeated iterative correction from coarse scale to fine scale (from large scale to small scale), so that the external parameter correction with coarse granularity can be carried out in the early stage, the robustness of larger external parameter errors is ensured, and the external parameter correction with fine granularity is carried out in the later stage, and the accuracy of the correction result can be ensured.

In some embodiments, the outer correction module 804 may be configured to loop through the first specified step for each of the scales until a preset termination condition is met.

The preset termination condition may include, but is not limited to, at least one of reaching a preset iteration number upper limit, reaching a preset iteration time consuming upper limit, and the like.

Wherein the first specifying step may include:

In some embodiments, the apparatus further includes a compensation value list setting module, configured to, before selecting one external parameter compensation value from the correction ranges of the external parameters under the scale, set an external parameter compensation value according to a preset interval respectively under the correction range corresponding to the scale for each external parameter, to obtain the plurality of compensation value lists.

Optionally, the extrinsic correction module 804 may be further configured to re-execute the first specified step, i.e. continue the next iteration, by ignoring and/or recording the corrected extrinsic values in response to the second pixel feature distance being greater than or equal to the first pixel feature distance.

In some embodiments of the present invention, in some embodiments,

In the case where the first specifying step is performed for the first time for a maximum scale, the acquiring a first pixel feature distance may include:

As an example, the absolute value of the difference of the pixel values of two pixels may be taken as the characteristic distance of the two pixels. That is, the first pixel characteristic distance may be an absolute value of a difference between the first pixel value and the second pixel value, and the second pixel characteristic distance may be an absolute value of a difference between the first pixel value and the third pixel value. It will be appreciated that the pixel feature distance may represent an error between two pixels, i.e. the first pixel feature distance may represent a pixel value error of the target feature point at a preset viewing angle of the reference photographing apparatus and the target photographing apparatus before correction, and the second pixel feature distance may represent a pixel value error of the target feature point at a preset viewing angle of the reference photographing apparatus and the target photographing apparatus after correction, and if the error is reduced before and after correction, the correction is effective.

The external parameter calibration device 800 for a photographing apparatus provided in the embodiment of the present application may also execute the method of fig. 1, implement the functions of the embodiment shown in fig. 1, and achieve the same technical effects, which are not described herein in detail.

Fig. 9 is a schematic structural diagram of an external parameter calibration device 900 according to an embodiment of the present application. Referring to fig. 9, in a software embodiment, the external parameter calibration device 900 may include an overlap region determination module 901, a feature point determination module 902, and an external parameter correction module 903.

An overlapping area determining module 901, configured to determine an overlapping area of a first input image and a second input image, where the first input image is a reference image and the second input image is an image under a preset viewing angle of a target photographing apparatus.

A feature point determining module 902, configured to determine a target feature point in the overlapping area.

The specific implementation procedures of the overlapping area determining module 901 and the feature point determining module 902 correspond to the specific implementation procedures of the overlapping area determining module 801 and the feature point determining module 802 in the embodiment shown in fig. 8, and the same technical effects can be achieved, and the description will not be repeated here.

The outer reference correction module 903 is configured to sequentially make i=1, 2, & gt..once, k, and repeat the outer reference correction step a after each assignment of i until a preset termination condition is met.

The external parameter calibration device 900 for a photographing apparatus provided in the embodiment of the present application may also execute the method of fig. 6, implement the functions of the embodiment shown in fig. 6, and achieve the same technical effects, which are not described herein in detail.

In summary, the foregoing description is only of the preferred embodiments of the present application, and is not intended to limit the scope of the present application. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application should be included in the protection scope of the present application.

The system, apparatus, module or unit set forth in the above embodiments may be implemented in particular by a computer chip or entity, or by a product having a certain function. One typical implementation is a computer. In particular, the computer may be, for example, a personal computer, a laptop computer, a cellular telephone, a camera phone, a smart phone, a personal digital assistant, a media player, a navigation device, an email device, a game console, a tablet computer, a wearable device, or a combination of any of these devices.

Computer readable media, including both non-transitory and non-transitory, removable and non-removable media, may implement information storage by any method or technology. The information may be computer readable instructions, data structures, modules of a program, or other data. Examples of storage media for a computer include, but are not limited to, phase change memory (PRAM), static Random Access Memory (SRAM), dynamic Random Access Memory (DRAM), other types of Random Access Memory (RAM), read Only Memory (ROM), electrically Erasable Programmable Read Only Memory (EEPROM), flash memory or other memory technology, compact disc read only memory (CD-ROM), digital Versatile Discs (DVD) or other optical storage, magnetic cassettes, magnetic tape magnetic disk storage or other magnetic storage devices, or any other non-transmission medium, which can be used to store information that can be accessed by a computing device. Computer-readable media, as defined herein, does not include transitory computer-readable media (transmission media), such as modulated data signals and carrier waves.

It should also be noted that the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising one does not exclude the presence of other like elements in a process, method, article, or apparatus that comprises an element.

The embodiments of the present application are described in a progressive manner, and the same and similar parts of the embodiments are all referred to each other, and each embodiment is mainly described in the differences from the other embodiments. In particular, for system embodiments, since they are substantially similar to method embodiments, the description is relatively simple, as relevant to see a section of the description of method embodiments.

Claims

1. A method for calibrating external parameters of a photographing device, characterized in that the method comprises:

Determine an overlapping area between a first input image and a second input image, wherein the first input image is a reference image and the second input image is an image at a preset viewing angle of a target shooting device;

Determine the target feature point in the overlapping area;

Obtain k scale correction ranges for each extrinsic parameter setting of the target shooting device, where k is an integer greater than or equal to 2, and the correction range is a value range of the extrinsic parameter compensation value of the target shooting device;

In the order of the scales from large to small, for each scale, the extrinsic parameter values of the target shooting device are corrected according to the target feature points and the correction range of each extrinsic parameter at the scale, wherein the correction result of the extrinsic parameter values at the previous scale is the correction reference of the extrinsic parameter values at the next scale, and the correction range of the next scale is a part of the correction range of the previous scale.

2. The method according to claim 1, characterized in that, for each scale, correcting each extrinsic parameter value of the target shooting device according to the target feature point and the correction range of each extrinsic parameter at the scale comprises:

For each of the scales, the first specified step is executed cyclically until a preset termination condition is met;

The first specifying step includes:

Selecting an external parameter compensation value from the correction range of each external parameter under the scale to obtain a set of external parameter compensation values;

Determining a corrected extrinsic parameter value of the target shooting device according to the uncorrected extrinsic parameter value of the target shooting device and the set of extrinsic parameter compensation values;

In response to the pixel feature of the target feature point at the preset viewing angle of the target shooting device after correction satisfying a preset correction condition, the uncorrected extrinsic parameter value of the target shooting device is updated to the corrected extrinsic parameter value.

3. The method according to claim 2, characterized in that the step of selecting an external parameter compensation value from the correction range of each external parameter at the scale comprises:

From the multiple compensation value lists of each extrinsic parameter at the scale, one extrinsic parameter compensation value is randomly selected to obtain a group of extrinsic parameter compensation values, wherein, at one scale, one extrinsic parameter corresponds to a compensation value list determined according to the correction range of the extrinsic parameter.

4. The method according to claim 3, characterized in that before selecting an external parameter compensation value from the correction range of each external parameter at the scale, the method further comprises:

For each of the external parameters within the correction range corresponding to the scale, external parameter compensation values are set according to preset intervals to obtain the multiple compensation value lists.

5. The method according to claim 3, wherein the preset termination condition includes at least one of the following:

The preset upper limit of iterations is reached;

The preset iteration time limit is reached;

The compensation values in the plurality of compensation value lists are exhausted.

6. The method according to claim 2, characterized in that, in response to the pixel feature of the target feature point at the preset viewing angle of the target shooting device after correction satisfying a preset correction condition, updating the uncorrected extrinsic parameter value of the target shooting device to the corrected extrinsic parameter value comprises:

Acquire a first pixel feature distance, wherein the first pixel feature distance is a pixel feature distance of the target feature point under the preset viewing angle of the target shooting device before the reference shooting device and the updating of the external parameter value;

Based on the corrected extrinsic parameter value, determining a second pixel feature distance of the target feature point at the preset viewing angles of the reference shooting device and the target shooting device;

In response to the second pixel characteristic distance being smaller than the first pixel characteristic distance, the uncorrected extrinsic parameter value of the target photographing device is updated to the corrected extrinsic parameter value.

7. The method according to claim 6, characterized in that, when the first specifying step is performed for the first time for the maximum scale, obtaining the first pixel feature distance comprises:

Determine a pixel value of the target feature point in the first input image as a first pixel value;

Determine a pixel value of the target feature point in the second input image as a second pixel value;

A first pixel feature distance is determined according to the first pixel value and the second pixel value.

8. The method according to claim 7, characterized in that the step of determining, based on the corrected extrinsic parameter value, a second pixel feature distance of the target feature point at the preset viewing angles of the reference shooting device and the target shooting device comprises:

Determine, according to the corrected extrinsic parameter value, a third pixel value of the target feature point at the preset viewing angle of the corrected target shooting device;

A second pixel feature distance of the target feature point at the preset viewing angles of the reference shooting device and the target shooting device is determined according to the first pixel value and the third pixel value.

9. The method according to claim 8, characterized in that determining the first pixel feature distance according to the first pixel value and the second pixel value comprises:

A first pixel feature distance is determined according to the first pixel value, a brightness modulation coefficient and the second pixel value, wherein the brightness modulation coefficient is determined according to a brightness difference of the overlapping area between the first input image and the second input image.

10. The method according to claim 9, characterized in that the determining, according to the first pixel value and the third pixel value, a second pixel feature distance of the target feature point at the preset viewing angles of the reference shooting device and the target shooting device comprises:

A second pixel feature distance of the target feature point at the preset viewing angles of the reference shooting device and the target shooting device is determined according to the first pixel value, the brightness modulation coefficient and the third pixel value.

11. The method according to claim 9, characterized in that before determining the first pixel feature distance according to the first pixel value, the brightness modulation coefficient and the second pixel value, the method further comprises:

The brightness modulation coefficient is determined according to the brightness information of the overlapping area in the first input image and the brightness information of the overlapping area in the second input image.

12. The method according to claim 11, characterized in that determining the brightness modulation coefficient according to the brightness information of the overlapping area in the first input image and the brightness information of the overlapping area in the second input image comprises:

Determine a first statistical result of the brightness of each pixel in the overlapping area in the first input image;

Determine a second statistical result of the brightness of each pixel in the overlapping area in the second input image;

The brightness modulation coefficient is determined according to the first statistical result and the second statistical result.

13. The method according to any one of claims 6 to 12, characterized in that the first specifying step further comprises:

In response to the second pixel feature distance being smaller than the first pixel feature distance, the first pixel feature distance is updated to the second pixel feature distance.

14. The method according to any one of claims 1-12 is characterized in that the preset perspective is a bird's-eye view perspective, the first input image is a bird's-eye view converted from a first image taken by a reference shooting device, and the second input image is a bird's-eye view converted from a second image taken by the target shooting device.

15. The method according to claim 14, wherein the reference photographing device and the target photographing device are fisheye cameras, and before determining the overlapping area between the first input image and the second input image, the method further comprises:

dedistorting the first image and the second image;

Converting the dedistorted first image into the first input image;

The dedistorted second image is converted into the second input image.

16. The method according to any one of claims 1 to 12, characterized in that the target feature points include edge feature points and corner feature points, wherein determining the target feature points in the overlapping area comprises:

Extracting edge feature points from the overlapping area of the first input image based on a Sobel operator;

Corner feature points are extracted from the overlapping area of the first input image based on the Harris feature corner feature point detection algorithm.

17. A method for calibrating external parameters of a photographing device, characterized in that the method comprises:

Determine the target feature point in the overlapping area;

Let i = 1, 2, ..., k in sequence, and repeat the external parameter correction step a after assigning a value to i each time until the preset termination condition is met;

Wherein, the extrinsic parameter correction step a comprises: obtaining an i-th correction range set for each external parameter of a plurality of external parameters of the target shooting device, selecting an external parameter compensation value from the i-th correction range corresponding to each of the plurality of external parameters, respectively, to obtain a plurality of i-th corrected external parameter values corresponding to each of the plurality of external parameters, in response to the pixel features of the target feature point at the preset viewing angle of the target shooting device after correction according to the plurality of i-th corrected external parameter values satisfying a preset correction condition, updating the values of the plurality of external parameters of the target shooting device to the plurality of i-th corrected external parameter values, wherein the i-th corrected external parameter value of each external parameter is equal to the sum of the value of the external parameter before the update and the i-th compensation value of the external parameter;

Wherein, k is an integer greater than or equal to 2, and for i=2, ..., k, the first external parameter update for the i-th correction range is based on the last external parameter update result corresponding to the i-1-th correction range;

Wherein, for i=2, ..., k, the i-th correction range is a part of the i-1-th correction range.

18. The method according to claim 17, characterized in that selecting an external parameter compensation value from the i-th correction range corresponding to the multiple external parameters respectively to obtain multiple i-th correction external parameter values corresponding to the multiple external parameters respectively comprises:

From the multiple i-th compensation value lists corresponding to the multiple external parameters, one external parameter compensation value is randomly selected respectively to obtain multiple i-th corrected external parameter values corresponding to each of the multiple external parameters, wherein one external parameter corresponds to an i-th compensation value list determined according to the i-th correction range of the external parameter.

19. The method according to claim 18, characterized in that

For one of the external parameters, an i-th compensation value list is determined according to the i-th correction range of the external parameter, and the i-th compensation value list is an arithmetic progression determined according to the i-th correction range.

20. The method according to claim 17, characterized in that in response to the pixel features of the target feature points at the preset viewing angle of the target shooting device after correction according to the multiple i-th correction extrinsic parameter values satisfying a preset correction condition, updating the values of the multiple extrinsic parameters of the target shooting device to the multiple i-th correction extrinsic parameter values accordingly comprises:

Based on the multiple i-th corrected extrinsic parameter values, determining a second pixel feature distance of the target feature point at the preset viewing angles of the reference shooting device and the target shooting device;

In response to the second pixel characteristic distance being smaller than the first pixel characteristic distance, the values of the multiple extrinsic parameters of the target shooting device are correspondingly updated to the multiple i-th corrected extrinsic parameter values.

21. The method according to claim 20, characterized in that, for i=1, and the external parameter correction step a is performed for the first time, the obtaining of the first pixel feature distance comprises:

22. The method according to claim 21, characterized in that the determining, based on the plurality of i-th corrected extrinsic parameter values, a second pixel feature distance of the target feature point at the preset viewing angles of the reference shooting device and the target shooting device comprises:

Determining, according to the plurality of i-th corrected extrinsic parameter values, a third pixel value of the target feature point at the preset viewing angle of the target shooting device;

23. The method according to any one of claims 20 to 22, characterized in that the external parameter correction step a further comprises:

24. An electronic device, comprising:

processor;

a memory for storing instructions executable by the processor;

The processor is configured to execute the instructions to implement the method as claimed in any one of claims 1 to 23.

25. A computer-readable storage medium, characterized in that when the instructions in the storage medium are executed by a processor of an electronic device, the electronic device is enabled to execute the method as claimed in any one of claims 1 to 23.

26. A computer program product comprising instructions, wherein when a computer runs the instructions of the computer program product, the computer performs the method according to any one of claims 1 to 23.