CN111246100A - Calibration method, device and electronic device for anti-shake parameters - Google Patents

Abstract

The application relates to an anti-shake parameter calibration method and device, electronic equipment and a computer readable storage medium. The method comprises the steps of controlling a camera and a reference calibration object to perform relative motion, shooting the reference calibration object, and acquiring at least two frames of target images; respectively acquiring corresponding feature points from at least two frames of target images; acquiring motion information of each characteristic point; determining at least two cost values of the anti-shake parameters based on the motion information of the corresponding feature points in the at least two frames of target images; and calibrating the anti-shake parameters according to at least two cost values of the anti-shake parameters. The method and the device, the electronic equipment and the computer readable storage medium can improve the accuracy of calibrating the anti-shake parameters.

Description

Calibration method, device and electronic device for anti-shake parameters

technical field

The present application relates to the field of imaging technologies, and in particular, to a method, device, electronic device, and computer-readable storage medium for calibrating anti-shake parameters.

Background technique

With the development of shooting equipment such as smartphones and video cameras, shooting techniques have emerged. When shooting, in order to shoot clearer and better-looking pictures or videos, it is usually necessary to perform anti-shake processing on camera devices such as smartphones and cameras. In order to make the photographing device perform anti-shake processing more accurately, each anti-shake parameter of the photographing device needs to be calibrated when the photographing device leaves the factory.

However, the traditional calibration method of anti-shake parameters has the problem of inaccurate calibration.

SUMMARY OF THE INVENTION

Embodiments of the present application provide a method, device, electronic device, and computer-readable storage medium for calibrating anti-shake parameters, which can improve the accuracy of anti-shake parameter calibration.

A method for calibrating anti-shake parameters, comprising:

Controlling the relative movement between the camera and the reference calibration object, and photographing the reference calibration object to obtain at least two frames of target images;

respectively obtain corresponding feature points from at least two frames of the target image;

acquiring motion information of each of the feature points;

Determine at least two cost values of the anti-shake parameter based on the motion information of the corresponding feature points in the at least two frames of the target image;

The anti-shake parameter is calibrated according to at least two of the cost values of the anti-shake parameter.

A device for calibrating anti-shake parameters, comprising:

a shooting module, configured to control the relative movement between the camera and the reference calibration object, and shoot the reference calibration object to obtain at least two frames of target images;

a feature point acquiring module, configured to acquire corresponding feature points from at least two frames of the target image respectively;

a motion information acquisition module for acquiring motion information of each of the feature points;

A cost value determination module, configured to determine at least two cost values of the anti-shake parameter based on the motion information of the corresponding feature points in the at least two frames of the target image;

A calibration module, configured to calibrate the anti-shake parameters according to at least two of the cost values of the anti-shake parameters.

An electronic device includes a memory and a processor, wherein a computer program is stored in the memory, and when the computer program is executed by the processor, the processor executes the steps of the above-mentioned method for calibrating anti-shake parameters.

A computer-readable storage medium having a computer program stored thereon, the computer program implementing the steps of the above-described method when executed by a processor.

The calibration method, device, electronic device and computer-readable storage medium for the above-mentioned anti-shake parameters control the relative movement between the camera and the reference calibration object, and shoot the reference calibration object to obtain at least two frames of target images; from at least two frames of target images Obtain the corresponding feature points in the target image respectively; obtain the motion information of each feature point; based on the motion information of the corresponding feature points in at least two frames of target images, it is possible to know the anti-shake between at least two frames of target images of the reference calibration object. Therefore, at least two cost values of the anti-shake parameters are determined; according to the at least two cost values of the anti-shake parameters, the anti-shake parameters can be calibrated more accurately.

Description of drawings

In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the following briefly introduces the accompanying drawings required for the description of the embodiments or the prior art. Obviously, the drawings in the following description are only These are some embodiments of the present application. For those of ordinary skill in the art, other drawings can also be obtained based on these drawings without any creative effort.

1 is a schematic diagram of an image processing circuit in one embodiment;

2 is a flowchart of a method for calibrating anti-shake parameters in one embodiment;

Fig. 3 is a flow chart of steps to determine the cost value of anti-shake parameters in one embodiment;

4 is a flowchart of steps in one embodiment to determine a target shooting strategy;

5 is a structural block diagram of an apparatus for calibrating anti-shake parameters in one embodiment;

FIG. 6 is a schematic diagram of the internal structure of an electronic device in one embodiment.

Detailed ways

In order to make the purpose, technical solutions and advantages of the present application more clearly understood, the present application will be described in further detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application, but not to limit the present application.

It will be understood that the terms "first", "second", etc. used in this application may be used herein to describe various elements, but these elements are not limited by these terms. These terms are only used to distinguish a first element from another element. For example, a first image could be referred to as a second image, and, similarly, a second image could be referred to as a first image, without departing from the scope of this application. Both the first image and the second image are images, but they are not the same image.

The embodiments of the present application also provide an electronic device. The above electronic device includes an image processing circuit, and the image processing circuit may be implemented by hardware and/or software components, and may include various processing units that define an ISP (Image Signal Processing, image signal processing) pipeline. FIG. 1 is a schematic diagram of an image processing circuit in one embodiment. As shown in FIG. 1 , for the convenience of description, only various aspects of the image processing technology related to the embodiments of the present application are shown.

As shown in FIG. 1 , the image processing circuit includes an ISP processor 140 and a control logic 150 . Image data captured by imaging device 110 is first processed by ISP processor 140 , which analyzes the image data to capture image statistics that can be used to determine and/or control one or more parameters of imaging device 110 . Imaging device 110 may include a camera having one or more lenses 112 and an image sensor 114 . Image sensor 114 may include an array of color filters (eg, Bayer filters), image sensor 114 may obtain light intensity and wavelength information captured with each imaging pixel of image sensor 114 and provide a set of raw materials that may be processed by ISP processor 140 . image data. The sensor 120 (eg, a gyroscope) may provide the acquired image processing parameters (eg, anti-shake parameters) to the ISP processor 140 based on the sensor 120 interface type. The sensor 120 interface may utilize a SMIA (Standard Mobile Imaging Architecture) interface, other serial or parallel camera interfaces, or a combination of the above interfaces.

Additionally, image sensor 114 may also send raw image data to sensor 120 , sensor 120 may provide raw image data to ISP processor 140 based on the sensor 120 interface type, or sensor 120 may store the raw image data in image memory 130 .

The ISP processor 140 processes raw image data pixel by pixel in various formats. For example, each image pixel may have a bit depth of 8, 10, 12, or 14 bits, and the ISP processor 140 may perform one or more image processing operations on the raw image data, collecting statistical information about the image data. Among them, the image processing operations can be performed with the same or different bit depth precision.

The ISP processor 140 may also receive image data from the image memory 130 . For example, the sensor 120 interface sends the raw image data to the image memory 130, and the raw image data in the image memory 130 is provided to the ISP processor 140 for processing. The image memory 130 may be a part of a memory device, a storage device, or an independent dedicated memory in an electronic device, and may include a DMA (Direct Memory Access, direct memory access) feature.

When receiving raw image data from the image sensor 114 interface or from the sensor 120 interface or from the image memory 130, the ISP processor 140 may perform one or more image processing operations, such as temporal filtering. The processed image data may be sent to image memory 130 for additional processing before being displayed. The ISP processor 140 may also receive processed data from the image memory 130 on which to perform image data processing in the original domain and in the RGB and YCbCr color spaces. In addition, the output of the ISP processor 140 may also be sent to the image memory 130 . In one embodiment, image memory 130 may be configured to implement one or more frame buffers.

The step of processing the image data by the ISP processor 140 includes: performing VFE (Video Front End, video front-end) processing and CPP (Camera Post Processing, camera post-processing) processing on the image data. VFE processing of image data may include correcting the contrast or brightness of the image data, modifying the digitally recorded lighting state data, performing compensation processing on the image data (such as white balance, automatic gain control, gamma correction, etc.), modifying the image data. filtering, etc. CPP processing of image data may include scaling the image, providing preview frames and recording frames to each path. Among them, CPP can use different codecs to process preview frames and record frames.

Statistics determined by the ISP processor 140 may be sent to the control logic 150 unit. For example, the statistics may include image sensor 114 statistics such as auto exposure, auto white balance, auto focus, flicker detection, black level compensation, lens 112 shading correction, and the like. Control logic 150 may include a processor and/or microcontroller executing one or more routines (eg, firmware) that may determine control parameters of imaging device 110 and ISP processing based on received statistics control parameters of the controller 140. For example, imaging device 110 control parameters may include sensor 120 control parameters (eg, gain, integration time for exposure control), camera flash control parameters, lens 112 control parameters (eg, focal length for focusing or zooming), or a combination of these parameters. ISP control parameters may include gain levels and color correction matrices for automatic white balance and color adjustment (eg, during RGB processing), and lens 112 shading correction parameters.

In one embodiment, the electronic device controls the imaging device (camera) 110 to perform relative movement between the camera and the reference calibration object, and controls the imaging device (camera) 110 to shoot the reference calibration object through the control logic 150 to obtain at least two frames of the target image, and at least two frames of target images are sent to the ISP processor 140 through the imaging device (camera) 110 . After receiving the at least two frames of target images, the ISP processor 140 obtains corresponding feature points respectively from the at least two frames of target images, and obtains motion information of each feature point. The motion information of each feature point may be acquired by the sensor 120 and sent to the ISP processor 140 . Based on the motion information of the corresponding feature points in the at least two target images, the ISP processor 140 can know the anti-shake situation between at least two target images of the reference calibration object, thereby determining at least two cost values of the anti-shake parameters. ; The anti-shake parameters can be calibrated more accurately according to at least two cost values of the anti-shake parameters.

FIG. 2 is a flowchart of a method for calibrating anti-shake parameters in one embodiment. As shown in FIG. 2 , the method for calibrating anti-shake parameters includes steps 202 to 210 .

Step 202 , controlling the camera to perform relative movement with the reference calibration object, and photographing the reference calibration object to acquire at least two frames of target images.

The camera is installed on the electronic device, and can be installed on the front side of the electronic device as a front camera, or on the back side of the electronic device as a rear camera, and can also be installed in other places of the electronic device. The number of cameras installed on the electronic device may be one or more. The type of any camera can be one of a depth camera, a telephoto camera, a wide-angle camera, a laser camera, a millimeter-wave camera, and the like.

It should be pointed out that the camera described in this application may refer to both a lens and a camera module. The camera module includes components such as lenses, image sensors, and motors.

The reference calibration object refers to the reference object used to calibrate the anti-shake parameters. For example, the reference calibration object may be at least one of a calibration plate, a calibration picture, a person, and the like.

Optionally, the camera can be controlled to remain still, and the reference calibration object can be controlled to move; the reference calibration object can also be controlled to remain stationary and the camera can be controlled to move; the reference calibration object and the camera can also be controlled to move at the same time, and the reference calibration object can be controlled to move at the same time. The speed of movement is not the same as the speed of movement of the camera.

Further, both the reference calibration object and the camera can move from at least one of the three moving degrees of freedom, front, back, left and right, and up and down, and can also move from roll (roll angle), pitch (pitch angle), and yaw (yaw angle). At least one of the three rotational degrees of freedom rotates, and it is also possible to move from both a movement degree of freedom and a rotation from a rotational degree of freedom.

When a relative movement is performed between the control camera and the reference calibration object, the electronic device controls the camera to photograph the reference calibration object to acquire at least two frames of target images.

Step 204, respectively acquiring corresponding feature points from at least two frames of target images.

Feature points refer to the points in the target image where certain features exist. For example, when the reference calibration object is a calibration plate, the feature point can be the corner point between the black square of the calibration plate and the adjacent white square in the target image; when the reference calibration object is a person, the feature point can be the character's eyes, nose, etc.

In each frame of the target image, the number of acquired feature points is not limited, and may be one or multiple. In each target image, feature points correspond. For example, the feature points obtained from target image 1 are A1 and B1, the feature points obtained from target image 2 are A2 and B2, the feature point A1 corresponds to the feature point A2, and the feature point B1 corresponds to the feature point B2; for another example, the target The feature points acquired in image 1 are eyes, and the feature points acquired in target image 2 are the corresponding eyes.

Step 206, acquiring motion information of each feature point.

The motion information of the feature point can be the coordinate position of the feature point in the target image; it can also be the motion direction of the feature point when shooting, and the motion direction can be obtained by the gyroscope in the electronic device; it can also be the feature point in the The movement speed when shooting, the movement speed can be obtained by the attitude sensor in the electronic device.

Step 208: Determine at least two cost values of the anti-shake parameter based on the motion information of the corresponding feature points in the at least two frames of the target image.

Anti-shake parameters refer to parameters used for camera anti-shake during camera shooting. The anti-shake parameters can include the angular velocity data collected by the gyroscope, the sampling frequency of the gyroscope, the zero drift value and the temperature drift value of the gyroscope, the delay between the time stamp of the collection of the gyroscope and the time stamp of the captured image, and the readout of the image sensor. At least one of the time of the image, the rotation matrix between the gyroscope module and the image sensor, and the like. The cost value is used to indicate the degree of difference between the target image actually shot with the anti-shake parameter and the target image that should be shot with the anti-shake parameter. The larger the cost value, the less accurate the anti-shake parameters; the smaller the cost value, the more accurate the anti-shake parameters.

Two frames of target images are selected from at least two frames of target images, and based on the motion information of the corresponding feature points in the selected two frames of target images, a cost value of the anti-shake parameter can be determined. Another two target images are selected from the at least two target images, and another cost value of the anti-shake parameter can be determined based on the motion information of the corresponding feature points in the selected other two target images. The more cost values are determined, the more accurate the anti-shake parameters can be calibrated.

In step 210, the anti-shake parameter is calibrated according to at least two cost values of the anti-shake parameter.

Calibration mainly refers to the use of standard measuring instruments to test whether the accuracy (precision) of the instruments used meets the standards. To calibrate the anti-shake parameters, that is, to calibrate the anti-shake parameters. It is understandable that, before the electronic device leaves the factory, the anti-shake parameters of the electronic device need to be calibrated, so that the user can obtain a clearer and more stable image or video when shooting with the electronic device.

The above-mentioned method for calibrating anti-shake parameters, controls the relative movement between the camera and the reference calibration object, and shoots the reference calibration object to obtain at least two frames of target images; respectively obtains corresponding feature points from the at least two frames of target images; obtains each The motion information of the feature points; based on the motion information of the corresponding feature points in at least two frames of target images, it is possible to know the anti-shake situation between at least two frames of target images of the reference calibration object, thereby determining at least two of the anti-shake parameters. Cost value; the anti-shake parameter can be calibrated more accurately according to at least two cost values of the anti-shake parameter.

In one embodiment, as shown in FIG. 3 , based on the motion information of corresponding feature points in at least two frames of target images, at least two cost values of anti-shake parameters are determined, including:

Step 302: Determine a first image and a second image from at least two frames of target images; the shooting time of the second image is later than the shooting time of the first image, the feature points of the first image are used as the first feature points, and the second image is taken as the first feature point. The feature point is used as the second feature point.

The first image and the second image may be two adjacent frames of target images, or may be separated from target images by a preset number of frames.

Step 304: Obtain reference values of anti-shake parameters between the first image and the second image.

The reference value of the anti-shake parameter is used to perform anti-shake processing on the second image to obtain the second image after anti-shake processing; more accurate anti-shake parameters can be determined by comparing the second image after anti-shake processing with the first image . The anti-shake processing may include at least one of EIS (Electric Image Stabilization, electronic image stabilization) and OIS (Optical Image Stabilization, optical image stabilization).

Specifically, the shooting time of the first image and the shooting time of the second image are obtained; the first reference value of the anti-shake parameter is obtained according to the shooting time of the first image, and the second reference of the anti-shake parameter is obtained according to the shooting time of the second image value; the reference value of the anti-shake parameter between the first image and the second image is determined based on the first reference value and the second reference value.

Optionally, difference processing may be performed on the first reference value and the second reference value, and the obtained difference value may be used as the reference value of the anti-shake parameter between the first image and the second image; the first reference value may also be obtained. The first weight of , and the second weight of the second reference value, multiply the first reference value and the first weight to obtain the first product, multiply the second reference value and the second weight to obtain the second product, and then multiply the Difference processing is performed on the first product and the second product, and the obtained difference is used as a reference value of the anti-shake parameter between the first image and the second image. The manner of acquiring the reference value of the anti-shake parameter between the first image and the second image is not limited.

Step 306: Determine the cost value of the anti-shake parameter based on the reference value of the anti-shake parameter, the motion information of the first feature point and the motion information of the second feature point.

The cost value is used to indicate the degree of difference between the target image actually shot with the anti-shake parameter and the target image that should be shot with the anti-shake parameter. The larger the cost value, the less accurate the anti-shake parameters; the smaller the cost value, the more accurate the anti-shake parameters.

Step 308: Change the reference value of the anti-shake parameter, obtain a new reference value of the anti-shake parameter, and return to the previous step until the determined cost value of the anti-shake parameter is less than the first generation value threshold, or the obtained anti-shake parameter is less than the value of the first generation value. The amount of the cost value reaches the first amount threshold.

Changing the reference value of the anti-shake parameter can increase the reference value of the anti-shake parameter, or decrease the reference value of the anti-shake parameter. After obtaining the reference value of the new anti-shake parameter, return to the previous step, that is, determine the cost value of the anti-shake parameter based on the reference value of the anti-shake parameter, the motion information of the first feature point and the motion information of the second feature point, The cost value of the new anti-shake parameter can be obtained.

In this embodiment, the first image and the second image are determined from at least two frames of target images; the shooting time of the second image is later than the shooting time of the first image, and the feature points of the first image are used as the first feature points, and the second image is taken as the first feature point. The feature point of the two images is used as the second feature point; the reference value of the anti-shake parameter between the first image and the second image is obtained; based on the reference value of the anti-shake parameter, the motion information of the first feature point and the second feature point Motion information to determine the cost value of the anti-shake parameter; change the reference value of the anti-shake parameter to obtain a new reference value of the anti-shake parameter, and return to the previous step until the determined cost value of the anti-shake parameter is less than the first generation value threshold , or until the number of the obtained cost values of the anti-shake parameters reaches the first number threshold. By changing the reference value of the anti-shake parameter to obtain the cost value of multiple anti-shake parameters, more accurate anti-shake parameters can be determined, so that the anti-shake parameters can be calibrated more accurately.

In one embodiment, determining the cost value of the anti-shake parameter based on the reference value of the anti-shake parameter, the motion information of the first feature point and the motion information of the second feature point includes: based on the reference value of the anti-shake parameter and the second feature point The motion information of the feature point, the second feature point is projected and transformed to obtain a new second feature point and the motion information of the new second feature point; the motion information of the new second feature point and the corresponding first feature The motion information of the points is compared to determine the cost value of the anti-shake parameters.

Projection transformation is the process of transforming the coordinates of a point in one image to the coordinates of a point in another image. When the anti-shake processing is performed on the second image, the anti-shake parameters are used to perform projection transformation on the second feature points in the second image to obtain new second feature points.

Similarly, the motion information of the new second feature point can be the coordinate position of the new second feature point in the new second image; it can also be the motion direction of the new second feature point when shooting, the The movement direction can be obtained by a gyroscope in the electronic device; it can also be the movement speed of the new second feature point when shooting, and the movement speed can be obtained by an attitude sensor in the electronic device.

It can be understood that the second feature point in the second image corresponds to the first feature point in the first image, and a new second feature point is obtained by performing projection transformation on the second feature point. When the motion information of the point is the same as the motion information of the corresponding first feature point, the cost value of the anti-shake parameter is the smallest, and the reference value of the anti-shake parameter for the projection transformation is the most accurate; and when the new second feature band The greater the difference between your motion information and the motion information of the corresponding first feature point, the greater the cost value of the anti-shake parameter, and the less accurate the reference value of the anti-shake parameter for performing the projection transformation.

For example, when the coordinates of the new second feature point are the same as the coordinates of the corresponding first feature point, the cost value of the anti-shake parameter is the smallest, and the reference value of the anti-shake parameter for the projection transformation is the most accurate; The larger the difference between the coordinates of the feature point and the coordinates of the corresponding first feature point, the greater the cost value of the anti-shake parameter, and the less accurate the reference value of the anti-shake parameter for performing the projection transformation.

The cost value can be calculated by the following cost function:

是第一图像t中的特征点i的运动信息，

是第二图像t+1中相对应的特征点i的运动信息，

是对第二特征点

进行投影变换之后得到新的第二特征点的运动信息。where J is the cost value,

is the motion information of the feature point i in the first image t,

is the motion information of the corresponding feature point i in the second image t+1,

is the second feature point

After the projection transformation is performed, the motion information of the new second feature point is obtained.

In this embodiment, based on the reference value of the anti-shake parameter and the motion information of the second feature point, a projection transformation is performed on the second feature point to obtain a new second feature point and motion information of the new second feature point; By comparing the motion information of the new second feature point with the motion information of the corresponding first feature point, a more accurate cost value of the anti-shake parameter can be determined.

In one embodiment, calibrating the anti-shake parameter according to at least two cost values of the anti-shake parameter includes: determining a target cost value from the at least two cost values, and using a reference value corresponding to the target cost value as the anti-shake parameter The target value of the anti-shake parameter; the anti-shake parameter is calibrated according to the target value of the anti-shake parameter.

Optionally, the smallest cost value may be determined as the target cost value from the at least two cost values, and the next smallest cost value may also be determined as the target cost value, which is not limited thereto.

After the target cost value is determined, the reference value of the anti-shake parameter corresponding to the target cost value, that is, the target value of the anti-shake parameter is the final value for calibrating the anti-shake parameter, that is, when the user is shooting, the The value of the anti-shake parameter is the target value of the anti-shake parameter.

In one embodiment, determining the target cost value from the at least two cost values includes: comparing the at least two cost values, and taking the smallest cost value as the target cost value.

The reference value of the anti-shake parameter corresponding to the smallest cost value is the most accurate, and the smallest cost value is used as the target cost value, so that the anti-shake parameter can be calibrated according to the target value of the most accurate anti-shake parameter.

In other embodiments, other cost values of at least two cost values can also be used as the target cost value, such as taking the next smallest cost value as the target cost value, and taking the cost value at the median position as the target cost value, etc., Not limited to this.

in other embodiments. The target cost value can also be determined from the at least two cost values by a gradient descent method, a least square method, a particle swarm algorithm, or the like.

In one embodiment, the above-mentioned method further includes: determining a new first image and a new second image from at least two frames of target images, and returning a reference value for obtaining the anti-shake parameter between the first image and the second image. step until the determined cost value of the anti-shake parameter is less than the second cost value threshold, or the determined number of the first images reaches the second number threshold.

After the first image and the second image are determined from the at least two frames of target images, one or more cost values corresponding to the anti-shake parameter may be determined according to the first image and the second image. After the first image and the second image are processed, a new first image and a new second image can be determined from at least two frames of target images, and then determined according to the new first image and the new second image. One or more cost values corresponding to the anti-shake parameter, until the determined cost value of the anti-shake parameter is less than the second generation value threshold, or the determined number of first images reaches the second number threshold.

In this embodiment, determining the cost value corresponding to the anti-shake parameter according to different first images and second images can improve the universality and stability of the cost value corresponding to the determined anti-shake parameter, and increase the anti-shake parameter. With more cost values of parameters, a more accurate target cost value can be determined from more cost values, so that the anti-shake parameters can be calibrated more accurately.

In one embodiment, as shown in Figure 4, the above method further includes:

Step 402, when the number of anti-shake parameters is at least two, determine the target shooting strategy.

The anti-shake parameters can include the angular velocity data collected by the gyroscope, the sampling frequency of the gyroscope, the zero drift value and the temperature drift value of the gyroscope, the delay between the time stamp of the collection of the gyroscope and the time stamp of the captured image, and the readout of the image sensor. At least one of the time of the image, the rotation matrix between the gyroscope module and the image sensor, and the like.

The target shooting strategy refers to the strategy when shooting the reference calibration object. The target shooting strategy may include setting the shooting duration, shooting frame rate, shooting imaging field of view, and the like. For example, the target shooting strategy may be to shoot a video with a duration of 5 seconds and a shooting frame rate of 10 frames per second.

Step 404 , when calibrating any one of the at least two anti-shake parameters, the target shooting strategy is used to shoot the reference calibration object.

The traditional anti-shake parameter calibration method usually formulates different shooting strategies for different anti-shake parameters, which has the problem of low calibration efficiency. In this embodiment, when the number of anti-shake parameters is at least two, when calibrating any one of the at least two anti-shake parameters, the target shooting strategy is used to shoot the reference calibration object, which can avoid The switching of shooting strategies during calibration with different anti-shake parameters improves the efficiency of calibration.

In one embodiment, photographing a reference calibrator to obtain at least two frames of target images includes: photographing a reference calibrator to obtain at least two frames of candidate images; and determining at least two adjacent frames of target images from the at least two frames of candidate images.

A candidate image refers to an image obtained by shooting. The target image refers to an image determined from candidate images. If at least two adjacent target images are determined from the at least two candidate images, then the shooting scene between the two adjacent target images is closer, and the corresponding feature points in the target image can be determined more accurately, so that the corresponding feature points in the target image can be more accurately determined. Anti-shake parameters are calibrated.

In another embodiment, candidate images separated by a preset number of frames may also be determined from the at least two candidate images as the at least two target images.

In other embodiments, the above method for calibrating anti-shake parameters can also be applied to other automated tests in the factory or the calibration of other parameters in the factory, to further improve the efficiency of the factory assembly line.

It should be understood that although the steps in the flowcharts of FIG. 2 to FIG. 4 are shown in sequence according to the arrows, these steps are not necessarily executed in the sequence shown by the arrows. Unless explicitly stated herein, the execution of these steps is not strictly limited to the order, and these steps may be performed in other orders. Moreover, at least a part of the steps in FIG. 2 to FIG. 4 may include multiple sub-steps or multiple stages. These sub-steps or stages are not necessarily executed and completed at the same time, but may be executed at different times. These sub-steps or stages may be executed at different times. The order of execution of the stages is also not necessarily sequential, but may be performed alternately or alternately with other steps or sub-steps of other steps or at least a portion of a stage.

FIG. 5 is a structural block diagram of an apparatus for calibrating anti-shake parameters according to an embodiment. As shown in FIG. 5 , a device 500 for calibrating anti-shake parameters includes: a shooting module 502, a feature point acquisition module 504, a motion information acquisition module 506, a cost value determination module 508, and a calibration module 510, wherein:

The shooting module 502 is configured to control the relative movement between the camera and the reference calibration object, and shoot the reference calibration object to obtain at least two frames of target images.

The feature point obtaining module 504 is configured to obtain corresponding feature points respectively from at least two frames of target images.

The motion information acquisition module 506 is configured to acquire motion information of each feature point.

The cost value determination module 508 is configured to determine at least two cost values of the anti-shake parameter based on the motion information of the corresponding feature points in the at least two frames of target images.

The calibration module 510 is configured to calibrate the anti-shake parameter according to at least two cost values of the anti-shake parameter.

The above-mentioned anti-shake parameter calibration device controls the relative movement between the camera and the reference calibration object, and shoots the reference calibration object to obtain at least two frames of target images; respectively obtains corresponding feature points from the at least two frames of target images; obtains each The motion information of the feature points; based on the motion information of the corresponding feature points in at least two frames of target images, it is possible to know the anti-shake situation between at least two frames of target images of the reference calibration object, thereby determining at least two of the anti-shake parameters. Cost value; the anti-shake parameter can be calibrated more accurately according to at least two cost values of the anti-shake parameter.

In one embodiment, the cost value determination module 508 is further configured to determine the first image and the second image from at least two frames of target images; the shooting time of the second image is later than the shooting time of the first image, and the The feature point is used as the first feature point, and the feature point of the second image is used as the second feature point; the reference value of the anti-shake parameter between the first image and the second image is obtained; based on the reference value of the anti-shake parameter, the first feature point and the motion information of the second feature point to determine the cost value of the anti-shake parameter; change the reference value of the anti-shake parameter to obtain a new reference value of the anti-shake parameter, and return to the previous step until the determined anti-shake parameter The cost value of is less than the first generation value threshold, or the number of obtained anti-shake parameter cost values reaches the first number threshold.

In one embodiment, the above-mentioned cost value determination module 508 is further configured to perform projective transformation on the second feature point based on the reference value of the anti-shake parameter and the motion information of the second feature point to obtain a new second feature point and a new motion information of the second feature point; comparing the motion information of the new second feature point with the motion information of the first feature point to determine the cost value of the anti-shake parameter.

In one embodiment, the calibration module 510 is further configured to determine a target cost value from at least two cost values, and use a reference value corresponding to the target cost value as the target value of the anti-shake parameter; Anti-shake parameters are calibrated.

In one embodiment, the above-mentioned calibration module 510 is further configured to compare at least two cost values, and use the smallest cost value as the target cost value.

In one embodiment, the above-mentioned cost value determination module 508 is further configured to determine a new first image and a new second image from at least two frames of target images, and return to perform anti-shake between acquiring the first image and the second image The reference value step of the parameter, until the determined cost value of the anti-shake parameter is less than the second cost value threshold, or the determined number of the first images reaches the second number threshold.

In one embodiment, the apparatus for calibrating anti-shake parameters further includes a target shooting strategy determination module for determining a target shooting strategy when the number of anti-shake parameters is at least two; for any of the at least two anti-shake parameters When one is calibrating, the target shooting strategy is used to shoot the reference calibration object.

In one embodiment, the above-mentioned photographing module 502 is further configured to photograph a reference calibration object to obtain at least two frames of candidate images; and to determine at least two adjacent frames of target images from the at least two frames of candidate images.

The division of each module in the above-mentioned anti-shake parameter calibration device is only used for illustration. full or partial functionality.

For the specific definition of the apparatus for calibrating the anti-shake parameter, reference may be made to the definition of the method for calibrating the anti-shake parameter above, which will not be repeated here. Each module in the above-mentioned anti-shake parameter calibration device may be implemented in whole or in part by software, hardware, and combinations thereof. The above modules can be embedded in or independent of the processor in the computer device in the form of hardware, or stored in the memory in the computer device in the form of software, so that the processor can call and execute the operations corresponding to the above modules.

FIG. 6 is a schematic diagram of the internal structure of an electronic device in one embodiment. As shown in FIG. 6, the electronic device includes a processor and a memory connected by a system bus. Among them, the processor is used to provide computing and control capabilities to support the operation of the entire electronic device. The memory may include non-volatile storage media and internal memory. The nonvolatile storage medium stores an operating system and a computer program. The computer program can be executed by the processor to implement an anti-shake parameter calibration method provided by the following embodiments. Internal memory provides a cached execution environment for operating system computer programs in non-volatile storage media. The electronic device may be any terminal device such as a mobile phone, a tablet computer, a PDA (Personal Digital Assistant), a POS (Point of Sales, a sales terminal), a vehicle-mounted computer, and a wearable device.

The implementation of each module in the apparatus for calibrating anti-shake parameters provided in the embodiments of the present application may be in the form of a computer program. The computer program can be run on a terminal or server. The program modules constituted by the computer program can be stored on the memory of the electronic device. When the computer program is executed by the processor, the steps of the methods described in the embodiments of the present application are implemented.

Embodiments of the present application also provide a computer-readable storage medium. One or more non-volatile computer-readable storage media containing computer-executable instructions, when the computer-executable instructions are executed by one or more processors, cause the processors to perform the method of calibrating anti-shake parameters. step.

A computer program product containing instructions, when run on a computer, causes the computer to perform a method for calibrating anti-shake parameters.

Any reference to a memory, storage, database, or other medium as used herein may include non-volatile and/or volatile memory. Nonvolatile memory may include read only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), or flash memory. Volatile memory may include random access memory (RAM), which acts as external cache memory. By way of illustration and not limitation, RAM is available in various forms such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDR SDRAM), enhanced SDRAM (ESDRAM), synchronous Link (Synchlink) DRAM (SLDRAM), Memory Bus (Rambus) Direct RAM (RDRAM), Direct Memory Bus Dynamic RAM (DRDRAM), and Memory Bus Dynamic RAM (RDRAM).

The above-mentioned embodiments only represent several embodiments of the present application, and the descriptions thereof are relatively specific and detailed, but should not be construed as a limitation on the scope of the patent of the present application. It should be noted that, for those skilled in the art, without departing from the concept of the present application, several modifications and improvements can be made, which all belong to the protection scope of the present application. Therefore, the scope of protection of the patent of the present application shall be subject to the appended claims.

Claims (11)

1. a calibration method of anti-shake parameter, is characterized in that, comprises: Controlling the relative movement between the camera and the reference calibration object, and photographing the reference calibration object to obtain at least two frames of target images; respectively obtain corresponding feature points from at least two frames of the target image; acquiring motion information of each of the feature points; Determine at least two cost values of the anti-shake parameter based on the motion information of the corresponding feature points in the at least two frames of the target image; The anti-shake parameter is calibrated according to at least two of the cost values of the anti-shake parameter. 2. The method according to claim 1, wherein, determining at least two cost values of anti-shake parameters based on the motion information of corresponding feature points in at least two frames of the target image, comprising: A first image and a second image are determined from at least two frames of the target image; the shooting time of the second image is later than the shooting time of the first image, and the feature points of the first image are used as the first feature points , the feature points of the second image are used as the second feature points; obtaining the reference value of the anti-shake parameter between the first image and the second image; Determine the cost value of the anti-shake parameter based on the reference value of the anti-shake parameter, the motion information of the first feature point and the motion information of the second feature point; Change the reference value of the anti-shake parameter to obtain a new reference value of the anti-shake parameter, and return to the previous step until the determined cost value of the anti-shake parameter is less than the first generation value threshold, or the obtained anti-shake parameter The number of cost values of the dithering parameter reaches the first number threshold. 3 . The method according to claim 2 , wherein the determination of the The cost value of anti-shake parameters, including: Based on the reference value of the anti-shake parameter and the motion information of the second feature point, projective transformation is performed on the second feature point to obtain a new second feature point and new motion information of the second feature point ; Comparing the motion information of the new second feature point with the motion information of the first feature point, the cost value of the anti-shake parameter is determined. 4 . The method according to claim 2 , wherein the calibrating the anti-shake parameters according to at least two of the cost values of the anti-shake parameters comprises: 5 . Determine a target cost value from at least two of the cost values, and use the reference value corresponding to the target cost value as the target value of the anti-shake parameter; The anti-shake parameter is calibrated according to the target value of the anti-shake parameter. 5. The method according to claim 4, wherein the determining a target cost value from at least two of the cost values comprises: At least two of the cost values are compared, and the smallest cost value is taken as the target cost value. 6. The method according to claim 2, wherein the method further comprises: Determine a new first image and a new second image from at least two frames of the target image, and return to executing the step of obtaining the reference value of the anti-shake parameter between the first image and the second image, until The determined cost value of the anti-shake parameter is less than the second cost value threshold, or the determined number of first images reaches the second number threshold. 7. The method of claim 1, wherein the method further comprises: When the number of the anti-shake parameters is at least two, determine the target shooting strategy; When calibrating any one of the at least two anti-shake parameters, the target shooting strategy is used to shoot the reference calibration object. 8. The method according to claim 1, wherein the photographing of the reference calibration object to obtain at least two frames of target images comprises: photographing the reference calibration object to obtain at least two candidate images; At least two adjacent target images are determined from the at least two candidate images. 9. A calibration device for anti-shake parameters, characterized in that, comprising: a shooting module, used to control the relative movement between the camera and the reference calibration object, and shoot the reference calibration object to obtain at least two frames of target images; a feature point acquiring module, configured to acquire corresponding feature points from at least two frames of the target image respectively; a motion information acquisition module for acquiring motion information of each of the feature points; A cost value determination module, configured to determine at least two cost values of the anti-shake parameter based on the motion information of the corresponding feature points in the at least two frames of the target image; A calibration module, configured to calibrate the anti-shake parameters according to at least two of the cost values of the anti-shake parameters. 10. An electronic device comprising a memory and a processor, wherein a computer program is stored in the memory, and when the computer program is executed by the processor, the processor is made to execute any one of claims 1 to 8 The steps of the method for calibrating anti-shake parameters. 11. A computer-readable storage medium on which a computer program is stored, wherein when the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 8 are implemented.

Images