CN111372000A - Video anti-shake method and apparatus, electronic device and computer-readable storage medium - Google Patents

Abstract

The application relates to a video anti-shake method, which comprises the following steps: acquiring first fuzzy information of an image frame of a video to be processed, wherein the first fuzzy information is used for representing the fuzzy degree of the image frame of the video to be processed; determining anti-shake information according to first fuzzy information of an image frame of the video to be processed, wherein the anti-shake information is used for representing the anti-shake degree of the video to be processed; determining anti-shake parameters of the video to be processed according to the anti-shake information; and carrying out anti-shake processing on the video to be processed according to the anti-shake parameters. The application also discloses a video anti-shake device, an electronic device and a computer readable storage medium. The method and the device reduce the influence of motion blur on the video and improve the shooting quality of the video.

Description

Video anti-shake method and apparatus, electronic device and computer-readable storage medium

technical field

The present application relates to the technical field of electronic devices, and in particular, to a video anti-shake method and device, an electronic device, and a computer-readable storage medium.

Background technique

In recent years, with the rapid development of electronic device technology, many electronic devices support video shooting, and the video shooting quality has become one of the important indicators for evaluating electronic devices. There are many factors that affect the shooting quality of video, such as resolution, saturation, sharpness, etc. Among them, image sharpness is a very important factor.

During the exposure time of the electronic device, when the subject moves or the electronic device shakes, it can cause motion blur in the video. Motion blur is a common phenomenon that occurs when shooting with electronic equipment, which is manifested as blur, blur or smear when the subject is imaged. The reason for the formation of motion blur is: due to the movement of the subject or the shaking of the electronic device, the imaging position of the subject on the sensor changes, resulting in a blurring effect. Motion blur affects the clarity of your video, which in turn affects the quality of your video's capture.

SUMMARY OF THE INVENTION

Embodiments of the present application provide a video anti-shake method, device, electronic device, and computer-readable storage medium, which can reduce the influence of motion blur on video and improve the shooting quality of video.

A video stabilization method, comprising:

acquiring first blur information of the image frame of the video to be processed, where the first blur information is used to represent the blur degree of the image frame of the video to be processed;

Determine anti-shake information according to the first blur information of the image frame of the video to be processed, where the anti-shake information is used to represent the anti-shake degree of the video to be processed;

Determine the anti-shake parameter of the video to be processed according to the anti-shake information;

Perform anti-shake processing on the video to be processed according to the anti-shake parameter.

A video anti-shake device, comprising:

an acquisition module, configured to acquire first blur information of the image frame of the video to be processed, where the first blur information is used to represent the blur degree of the image frame of the video to be processed;

a determining module, configured to determine anti-shake information according to the first blur information of the image frame of the to-be-processed video, where the anti-shake information is used to represent the anti-shake degree of the to-be-processed video;

The determining module is further configured to determine the anti-shake parameter of the video to be processed according to the anti-shake information;

A processing module, configured to perform anti-shake processing on the video to be processed according to the anti-shake parameter.

An electronic device, comprising a memory and a processor, wherein a computer program is stored in the memory, and when the computer program is executed by the processor, the processor executes the steps of any one of the video anti-shake methods .

A computer-readable storage medium on which a computer program is stored, and when the computer program is executed by a processor, causes the processor to execute the steps of any one of the video anti-shake methods.

The above-mentioned video anti-shake method, device, electronic device, and computer-readable storage medium obtain first blurring information of an image frame of the video to be processed, and the first blurring information is used to represent the blurring degree of the image frame of the video to be processed. The first blur information of the image frame of the video determines the anti-shake information, and the anti-shake information is used to represent the anti-shake degree of the video to be processed. Shake processing, in this way, not only reduces the impact of motion blur on the video, but also improves the shooting quality of the video, and adjusts the degree of shaking of the video according to the degree of blurring of the video, which realizes the stability of the video at the user's perception level.

Description of drawings

In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the following briefly introduces the accompanying drawings required for the description of the embodiments or the prior art. Obviously, the drawings in the following description are only These are some embodiments of the present application. For those of ordinary skill in the art, other drawings can also be obtained based on these drawings without any creative effort.

1 is a flowchart of a video anti-shake method in one embodiment;

FIG. 2 is a detailed flow chart of obtaining first fuzzy information in one embodiment;

3 is a detailed flow chart of determining anti-shake information in one embodiment;

4 is a flowchart of a video stabilization method in another embodiment;

5 is a schematic diagram of a refinement process for obtaining jitter information in one embodiment;

6 is a detailed flow chart of determining anti-shake information in another embodiment;

7 is a flowchart of a video anti-shake method in yet another embodiment;

8 is a structural block diagram of a video anti-shake device in one embodiment;

FIG. 9 is a block diagram of the internal structure of an electronic device in one embodiment.

Detailed ways

In order to make the purpose, technical solutions and advantages of the present application more clearly understood, the present application will be described in further detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application, but not to limit the present application.

The electronic device in the embodiment of the present application may be a mobile terminal, such as a mobile phone, a camera, a video camera, a tablet computer, a notebook computer, a wearable device, and the like. The electronic device is provided with a camera, and the camera is used for taking videos, photos, and the like. Specifically, the electronic device acquires first blurring information of the image frame of the video to be processed, where the first blurring information is used to represent the blurring degree of the image frame of the video to be processed. Next, the electronic device determines anti-shake information according to the first blur information of the image frame of the video to be processed, where the anti-shake information is used to represent the anti-shake degree of the video to be processed. Next, the electronic device determines the anti-shake parameter of the video to be processed according to the anti-shake information. Next, the electronic device performs anti-shake processing on the video to be processed according to the anti-shake parameter.

FIG. 1 is a flowchart of a video anti-shake method in one embodiment. The video anti-shake method shown in FIG. 1 can be applied to the above electronic equipment, including:

Step 102: Obtain first blur information of the image frame of the video to be processed, where the first blur information is used to represent the blur degree of the image frame of the video to be processed.

The video to be processed refers to a video to be subjected to anti-shake processing, and optionally, the video to be processed may be a video currently being shot by the electronic device. An image frame is the smallest unit of video, and the image frame of the video to be processed is used to represent each frame of the video to be processed.

The first blur information is used to represent the blur degree of the image frame of the video to be processed. The blur degree of the image frame refers to the reverse deviation degree of the sharpness of the image frame from the preset sharpness. Sharpness refers to the clarity of each detail shadow and its boundary in an image frame. Taking the preset definition as the standard definition, when the definition of an image frame is higher than or equal to the preset definition, the image frame is determined to be clear, and when the definition of an image frame is lower than the preset definition, the Image frames are blurry.

In one embodiment, the first blur information may be sharpness. In another embodiment, the first blur information may be a blur level, for example, as the definition of the image frame becomes lower and lower, the blur level becomes higher and higher.

In one embodiment, the definition of any frame in the image frames of the video to be processed may be acquired, and the first blur information is determined according to the definition of the arbitrary frame. In another embodiment, the sharpness of any number of frames in the image frames of the video to be processed may be acquired, and the first blur information is determined according to the average value of the sharpness of the arbitrary several frames. The arbitrary several frames may be any number of image frames of the video to be processed within a preset time period.

In one embodiment, the first blur information of the image frame of the video to be processed may be acquired through an image sharpness detection method. The image sharpness detection method may be: Brenner gradient function, Laplacian gradient function, variance function, energy gradient function, and the like. The Brenner gradient function determines the sharpness of the image frame by calculating the square of the grayscale difference between two adjacent pixels. The Laplacian gradient function calculates the convolution of the Laplacian operator at each pixel point, and accumulates it pixel by pixel to determine the sharpness of the image frame. The variance function determines the sharpness of the image frame by the average gray value of all pixels. The energy gradient function determines the sharpness of the image frame by the gray value of each pixel point.

Step 104: Determine anti-shake information according to the first blur information of the image frame of the video to be processed, where the anti-shake information is used to represent the anti-shake degree of the video to be processed.

Among them, the anti-shake information is used to represent the anti-shake degree of the video to be processed. Optionally, the anti-shake information may be an anti-shake level, such as anti-shake level one, anti-shake level two, anti-shake level three, and the like. The relationship between the anti-shake level and the anti-shake degree can be set according to the actual application, for example, the higher the anti-shake level, the higher the anti-shake degree, or the higher the anti-shake level, the lower the anti-shake degree, etc.

Specifically, when a user watches a video, the user's perception of the blur of the video is related to the jitter of the video. For example, in a shaking video, the user is often not easily aware of the blurred performance of the video, while in a stable video, the blurred performance of the video is easily perceived by the user. Further, under different degrees of shaking, the user's perception of the blurred appearance of the video is different. Therefore, adjusting the shaking degree of the video according to the blurring degree of the video can suppress the blurring performance of the video at the user's perception level.

In one embodiment, the first blur information is inversely proportional to the anti-shake information. That is, the higher the blur degree of the image frame of the video to be processed, the lower the anti-shake degree of the video to be processed.

In one embodiment, before determining the anti-shake information according to the first blur information of the image frame of the video to be processed, the method further includes: detecting whether the first blur information of the image frame of the video to be processed satisfies the adjustment condition, when the video to be processed When the first blur information of the image frame satisfies the adjustment condition, the step of determining anti-shake information according to the first blur information of the image frame of the video to be processed is performed. The adjustment condition is used to detect whether the definition of the image frame of the video to be processed is lower than the predetermined definition, and when the definition of the image frame of the video to be processed is lower than the predetermined definition, determine the first blur information of the image frame of the video to be processed If the adjustment conditions are met, the anti-shake information is determined according to the first blur information of the image frame of the video to be processed; when the definition of the image frame of the video to be processed is higher than or equal to the predetermined definition, the anti-shake processing of the video to be processed may not be subject to the first blur. The effect of fuzzy information. Wherein, the predetermined definition can be set according to the actual application, and it can be lower than the preset definition.

Step 106: Determine anti-shake parameters of the video to be processed according to the anti-shake information.

The anti-shake parameter is used to perform anti-shake processing on the video to be processed. The anti-shake parameter can be any parameter applied in the process of anti-shake processing of the video to be processed.

In one embodiment, the mapping relationship between the anti-shake information and the anti-shake parameter may be preset, and the anti-shake parameter is determined according to the anti-shake information and the mapping relationship. It can be understood that the mapping relationship between the anti-shake information and the anti-shake parameter is obtained through a large number of experiments, and the mapping relationship can make the blurring performance of the image frame of the video within a reasonable range.

Step 108: Perform anti-shake processing on the video to be processed according to the anti-shake parameter.

In one embodiment, the anti-shake processing may be EIS (Electric Image Stabilization) electronic image stabilization, OIS (Optical image stabilization) optical image stabilization, etc., or a combination of EIS electronic image stabilization and OIS optical image stabilization. EIS electronic image stabilization is to use the degree of shaking of the shooting terminal to dynamically adjust the sensitivity, shutter or software to do blur correction. OIS optical image stabilization refers to the setting of optical components in the shooting terminal, such as the setting of the lens, to avoid or reduce the shooting terminal shake phenomenon that occurs in the process of capturing optical signals.

Specifically, the basic principle of the anti-shake processing is to transform each frame of image from the projection of the imaging plane corresponding to the actual camera posture to the projection of the imaging plane corresponding to the corresponding virtual smooth camera posture. The camera attitude refers to the camera attitude, and the change of the camera attitude can be obtained through the data generated by the gyroscope. Changes in camera pose over a period of time can form a camera pose path. If it is a stable video, the actual camera pose path of the video presents a smooth curve. If it is a jittery video, the actual camera pose path of the video presents a jittery curve. The actual camera pose path of the shaken video is smoothed to obtain a virtual smooth camera pose path, that is, the anti-shake processing is completed.

In the video anti-shake method in this embodiment, the first blur information of the image frame of the video to be processed is obtained, and the first blur information is used to represent the blur degree of the image frame of the video to be processed. The blur information determines the anti-shake information, and the anti-shake information is used to represent the anti-shake degree of the video to be processed. Determine the anti-shake parameters of the to-be-processed video according to the anti-shake information, and perform anti-shake processing on the to-be-processed video according to the anti-shake parameters. The impact of motion blur on the video is improved, the shooting quality of the video is improved, and the degree of shaking of the video is adjusted according to the degree of blurring of the video, which realizes the stability of the video at the level of user perception.

In one embodiment, as shown in FIG. 2 , the acquiring the first blur information of the image frame of the video to be processed includes:

Step 202: Obtain second blur information and third blur information of the image frame of the video to be processed, where the second blur information is blur information generated by the movement of the shooting object, and the third blur information is caused by the movement of the shooting terminal. The resulting fuzzy information.

Wherein, the second blur information is blur information generated by the motion of the photographed object, the photographed object refers to the photographed object, and the photographed object motion means that the photographed object is not in a stationary state during the photographing process. The third blur information is blur information generated by the motion of the photographing terminal. The photographing terminal may be an electronic device for photographing, and the motion of the photographing terminal means that the electronic device for photographing is not in a stationary state during the photographing process.

In one embodiment, the second blur information and the third blur information may be sharpness. In another embodiment, the second blur information and the third blur information may be blur levels. For example, as the definition of the image frame is lower and lower, the blur level is higher and higher.

In one embodiment, the manner of acquiring the second blur information and the third blur information of the image frame of the video to be processed may be: respectively acquiring the second blur information according to the feature information of the second blur information and the feature information of the third blur information information and third vague information. The feature information of the second blur information means that the shooting object is moving, but the background where the shooting object is located is in a static state; the feature information of the third blur information means that both the shooting object and the background where the shooting object is located are in a moving state.

Step 204: Determine the first fuzzy information according to the second fuzzy information and the third fuzzy information.

In one embodiment, the first blur information may include second blur information and third blur information.

In one embodiment, a mapping relationship between the second blur information, the third blur information, and the anti-shake information may be preset, and the anti-shake information is determined according to the second blur information, the third blur information, and the mapping relationship.

In the video anti-shake method in this embodiment, the second blur information and third blur information of the image frame of the video to be processed are obtained, the second blur information is blur information generated by the movement of the shooting object, and the third blur information is obtained by the shooting terminal. For the blur information generated by motion, the first blur information is determined according to the second blur information and the third blur information. In this way, the first blur information is refined into the second blur information and the third blur information, so that the anti-shake information can be determined more accurately. .

In one embodiment, as shown in FIG. 3 , the determining of the anti-shake information according to the first blur information of the image frame of the video to be processed includes:

Step 302: Obtain a pre-stored first mapping relationship between the first blur information and the anti-shake information.

The first mapping relationship is used to represent the mapping relationship between the first fuzzy information and the anti-shake information.

Step 304: Determine the anti-shake information according to the first fuzzy information and the first mapping relationship.

Specifically, a mapping relationship between the first fuzzy information and the anti-shake information is preset, and the anti-shake information is determined according to the first fuzzy information and the mapping relationship. It can be understood that the mapping relationship between the first fuzzy information and the anti-shake information is obtained through a large number of experiments. Specifically, the first blur information may be clarity (the higher the clarity, the lower the blur degree), or the first blur information may be the blur level (the relationship between the blur level and the blur degree can be set according to the actual application, For example, the higher the blur level, the higher the blur degree); the anti-shake information can be the anti-shake level (the relationship between the anti-shake level and the anti-shake degree can be set according to the actual application, for example, the higher the anti-shake level, the higher the anti-shake degree. high). The mapping relationship between the first blur information and the anti-shake information may be the mapping relationship between the sharpness and the anti-shake level, or the mapping relationship between the blur level and the anti-shake level. The mapping relationship between the first blur information and the anti-shake information is used to represent the relationship between the blur degree and the anti-shake degree. In one embodiment, the first blur information is inversely proportional to the anti-shake information, that is, the higher the blur degree of the image frame of the video to be processed, the lower the anti-shake degree of the video to be processed. The video anti-shake method in this embodiment acquires a first mapping relationship between the pre-stored first blur information and the anti-shake information, determines the anti-shake information according to the first blur information and the first mapping relationship, and determines the anti-shake information according to the blur degree of the video. Adjust the degree of jitter of the video, so that the blurred performance of the video can be suppressed at the user's perception level.

In an embodiment, before determining the anti-shake information according to the first blur information of the image frame of the video to be processed, the method further includes: acquiring the jitter information of the image frame of the video to be processed, the jitter information is used for Indicates the degree of jitter of the video to be processed.

Among them, the shaking information is used to represent the shaking degree of the video to be processed. The degree of shaking of the video to be processed refers to the degree of deviation of the shaking state of the video to be processed relative to the still state.

In one embodiment, the jitter information may be a jitter level, for example, as the jitter state of the video becomes more and more serious, the jitter level becomes higher and higher.

In one embodiment, the jitter information of the video to be processed within a predetermined time may be acquired. The predetermined time can be set according to the actual application, such as unit time.

In one embodiment, determining the anti-shake information according to the first blur information of the image frame of the video to be processed includes: determining the anti-shake information according to the first blur information and the jitter information. Specifically, the first blur information and the shaking information can reflect the blurry performance visually perceived by the user, and the anti-shake information can be more accurately determined according to the first blur information and the shaking information. That is, the visually perceived blurring performance of the user may be a function constructed by the first blurring information and the shaking information. Assuming that the visually perceived blurring performance by the user is B ₁ , the first blurring information is B ₂ , and the shaking information is Y, then B ₁ =f ₁ (B ₂ , Y).

The video anti-shake method in this embodiment obtains the jitter information of the image frame of the video to be processed, the jitter information is used to represent the jitter degree of the to-be-processed video, and the anti-shake information is determined according to the first blur information and the jitter information. In this way, according to the video The degree of blurring and the actual degree of shaking adjust the degree of shaking of the video, and realize the stability of the video at the user's perception level.

In one embodiment, as shown in FIG. 4 , the acquiring the jitter information of the image frame of the video to be processed includes:

Step 402: Acquire a shaking parameter of the image frame of the video to be processed, where the shaking parameter includes at least one of a shaking frequency and a shaking amplitude.

Wherein, the jitter parameter includes at least one of a jitter frequency and a jitter amplitude. The jitter frequency and the jitter amplitude can be detected by the gyroscope.

Step 404: Determine the jitter information according to the jitter parameter.

In one embodiment, the average amplitude of video jitter in a predetermined time and the average frequency of video jitter in the predetermined time may be obtained, and the jitter information is determined according to the average amplitude and average frequency of video jitter in the predetermined time.

In the video anti-shake method in this embodiment, the jitter parameters of the image frames of the video to be processed are obtained, the jitter parameters include at least one of the jitter frequency and the jitter amplitude, and the jitter information is determined according to the jitter parameters, so that the jitter information can be accurately determined.

In one embodiment, as shown in FIG. 5 , the determining the jitter information according to the jitter parameter includes:

Step 502: Acquire a pre-stored second mapping relationship between the jitter parameter and the jitter information.

The second mapping relationship is used to represent the mapping relationship between the jitter parameter and the jitter information. Specifically, the jitter parameter includes at least one of a jitter frequency and a jitter amplitude, and the jitter information can be a function constructed by an average amplitude of video jitter in a predetermined time and an average amplitude of video jitter in a predetermined time. The average amplitude of video jitter is A, and the average amplitude of video jitter in a predetermined time is F, then Y=f ₂ (A, F). Wherein, the predetermined time may be 1-10 seconds, etc., and the function may be a linear function or the like. Step 504: Determine the jitter information according to the jitter parameter and the second mapping relationship.

Specifically, the mapping relationship between the jitter parameters and the jitter information is preset, and the jitter information is determined according to the jitter parameters and the mapping relationship. It can be understood that the mapping relationship between the jitter parameters and the jitter information is obtained through a large number of experiments.

The video anti-shake method in this embodiment acquires the second mapping relationship between the pre-stored jitter parameters and the jitter information, and determines the jitter information according to the jitter parameters and the second mapping relationship, so as to accurately determine the jitter information. In one embodiment, as shown in FIG. 6 , the determining of the anti-shake information according to the first blur information and the jitter information includes:

Step 602: Acquire a pre-stored third mapping relationship between the first blur information, the jitter information, and the anti-shake information.

The third mapping relationship is used to represent the mapping relationship between the first blur information, the jitter information, and the anti-shake information. The first blur information may be clarity (the higher the clarity, the lower the blur degree), or the first blur information may be the blur level (the relationship between the blur level and the blur level may be set according to practical applications, for example, the blur level The higher the degree of blurring, the higher the degree of blur); the jitter information can be the jitter level (for example, the more serious the jitter state of the video, the higher the jitter level); the anti-shake information can be the anti-shake level (the relationship between the anti-shake level and the degree of anti-shake). It can be set according to the actual application, for example, the higher the anti-shake level, the higher the anti-shake degree). The mapping relationship between the first blur information, the jitter information, and the anti-shake information may be the mapping relationship between the sharpness, the anti-shake level, and the anti-shake level, or the mapping between the blur level, the anti-shake level, and the anti-shake level. relation.

In one embodiment, the higher the degree of blurring of the image frame of the video to be processed, the higher the degree of jitter, and the lower the degree of anti-shake of the video to be processed; the higher the degree of blurriness of the image frame of the video to be processed, the lower the degree of jitter, The lower the degree of anti-shake of the video to be processed; the lower the blur degree of the image frame of the video to be processed, the higher the degree of jitter, the higher the degree of anti-shake of the video to be processed; the lower the degree of blur of the image frame of the video to be processed, the higher the degree of jitter The lower the degree, the higher the degree of stabilization of the video to be processed.

Step 604: Determine the anti-shake information according to the first blur information, the jitter information, and the third mapping relationship.

Specifically, a mapping relationship between the first blur information, the jitter information and the anti-shake information is preset, and the anti-shake information is determined according to the first blur information, the jitter information, and the mapping relationship. It can be understood that the mapping relationship among the first blur information, the jitter information and the anti-shake information is obtained through a large number of experiments, and the mapping relationship can make the blur performance of the image frame of the video within a reasonable range.

The video anti-shake method in this embodiment acquires the pre-stored first blur information, the third mapping relationship between the jitter information and the anti-shake information, and determines the anti-shake information according to the first blur information, the jitter information and the third mapping relationship , in this way, the degree of shaking of the video is adjusted according to the degree of blurring of the video and the actual degree of shaking, and the stability of the video is realized at the level of user perception.

In one embodiment, as shown in FIG. 7 , performing anti-shake processing on the video to be processed according to the anti-shake parameter includes:

Step 702, acquiring the camera internal parameter matrix.

Among them, the camera internal parameter is used as a key parameter to participate in the anti-shake calculation, and the camera internal parameter includes the camera optical center parameter.

Step 704: Perform projection transformation on the image frame according to the camera internal parameter matrix and the anti-shake parameter, so as to perform anti-shake processing on the video to be processed.

The anti-shake parameter may be a rotation matrix.

Specifically, for EIS electronic image stabilization, the Euler angle is first converted into a quaternion; then smoothing is performed, and the result of the smoothing is the conversion quaternion from the actual camera pose of each frame to the virtual camera pose; then Convert the quaternion to a rotation matrix to describe the transformation of each image frame; then combine the rotation matrix and the camera internal parameter matrix to generate a transformation matrix, which is used for the projection transformation of the image frame; finally, the image frame is projected through the transformation matrix Transform to achieve anti-shake of the video's image frames.

In the video anti-shake method in this embodiment, the camera internal parameter matrix is obtained, and the image frame is projected and transformed according to the camera internal parameter matrix and anti-shake parameters, so as to perform anti-shake processing on the video to be processed. In this way, the video is anti-shake according to EIS electronic anti-shake. Shaking processing reduces the impact of motion blur on the video, improves the shooting quality of the video, and adjusts the degree of shaking of the video according to the degree of blurring of the video, which realizes the stability of the video at the level of user perception.

In a specific embodiment, a video anti-shake method includes:

First, obtain first blur information of the image frame of the video to be processed, where the first blur information is used to represent the blur degree of the image frame of the video to be processed;

Next, obtain the average amplitude of the video jitter in a unit time and the average frequency of the video jitter in the unit time, and determine the jitter information according to the average amplitude and the average frequency of the video jitter in the unit time;

Further, the mapping relationship between the pre-stored first fuzzy information, the jitter information and the anti-shake information is obtained, and the anti-shake information is determined according to the first fuzzy information, the jitter information and the mapping relationship, and the anti-shake information is used to represent the to-be-processed The degree of anti-shake of the video, the first blur information is inversely proportional to the anti-shake information;

Next, obtain the mapping relationship between the pre-stored anti-shake information and the anti-shake parameter, and determine the anti-shake parameter according to the anti-shake information and the mapping relationship;

Further, the camera internal parameter matrix is obtained, and the image frame is projected and transformed according to the camera internal parameter matrix and the anti-shake parameter, so as to perform anti-shake processing on the video to be processed.

The video anti-shake method in this embodiment performs anti-shake processing on the video according to the EIS electronic anti-shake, reduces the influence of motion blur on the video, improves the shooting quality of the video, and performs anti-shake processing on the video according to the blur degree of the video. Adjustment to achieve video stabilization at the user perception level.

It should be understood that although the steps in the flowcharts of FIGS. 1-7 are shown in sequence according to the arrows, these steps are not necessarily executed in the sequence shown by the arrows. Unless explicitly stated herein, the execution of these steps is not strictly limited to the order, and these steps may be performed in other orders. Moreover, at least a part of the steps in FIGS. 1-7 may include multiple sub-steps or multiple stages. These sub-steps or stages are not necessarily executed and completed at the same time, but may be executed at different times. These sub-steps or stages are not necessarily completed at the same time. The order of execution of the steps is not necessarily sequential, but may be performed alternately or alternately with other steps or at least part of sub-steps or stages of other steps.

FIG. 8 is a structural block diagram of a video anti-shake apparatus 800 according to an embodiment. As shown in FIG. 8 , a video anti-shake device 800 includes: an acquisition module 802 , a determination module 804 and a processing module 806 . in:

an obtaining module 802, configured to obtain first blur information of the image frame of the video to be processed, where the first blur information is used to represent the blur degree of the image frame of the video to be processed;

A determination module 804, configured to determine anti-shake information according to the first blur information of the image frame of the video to be processed, the anti-shake information is used to represent the anti-shake degree of the to-be-processed video, the first blur information Inversely proportional to the anti-shake information;

The determining module 804 is further configured to determine the anti-shake parameter of the video to be processed according to the anti-shake information;

The processing module 806 is configured to perform anti-shake processing on the video to be processed according to the anti-shake parameter.

The video anti-shake device 800 in this embodiment obtains first blur information of the image frame of the video to be processed, and the first blur information is used to represent the blur degree of the image frame of the video to be processed. The blur information determines the anti-shake information, and the anti-shake information is used to represent the anti-shake degree of the video to be processed. Determine the anti-shake parameters of the to-be-processed video according to the anti-shake information, and perform anti-shake processing on the to-be-processed video according to the anti-shake parameters. The impact of motion blur on the video is improved, the shooting quality of the video is improved, and the degree of shaking of the video is adjusted according to the degree of blurring of the video, which realizes the stability of the video at the level of user perception.

In one embodiment, the acquiring module 802 is further configured to: acquire second blur information and third blur information of the image frame of the video to be processed, where the second blur information is blur generated by the movement of the photographing object information, the third blur information is blur information generated by the motion of the photographing terminal, and the first blur information is determined according to the second blur information and the third blur information.

The video anti-shake device 800 in this implementation obtains second blur information and third blur information of the image frame of the video to be processed, the second blur information is blur information generated by the movement of the shooting object, and the third blur information is generated by the shooting terminal For the blur information generated by the motion, the first blur information is determined according to the second blur information and the third blur information. In this way, the first blur information is refined into the second blur information and the third blur information, so that the anti-shake information can be determined more accurately. .

In one embodiment, the determining module 804 is further configured to: obtain a pre-stored first mapping relationship between the first blur information and the anti-shake information, according to the first blur information and the The first mapping relationship determines the anti-shake information.

The video anti-shake device 800 in this embodiment acquires a first mapping relationship between the pre-stored first blur information and the anti-shake information, determines the anti-shake information according to the first blur information and the first mapping relationship, and determines the anti-shake information according to the blur degree of the video. Adjust the degree of jitter of the video, so that the blurred performance of the video can be suppressed at the user's perception level.

In one embodiment, the obtaining module 802 is further configured to: obtain the shaking information of the image frame of the video to be processed, where the shaking information is used to represent the shaking degree of the video to be processed; the determining module 804 , which is further configured to: determine the anti-shake information according to the first blur information and the jitter information.

The video anti-shake device 800 in this implementation obtains the jitter information of the image frame of the video to be processed, the jitter information is used to represent the degree of jitter of the to-be-processed video, and the anti-shake information is determined according to the first blur information and the jitter information. In this way, according to the video The degree of blurring and the actual degree of shaking adjust the degree of shaking of the video, and realize the stability of the video at the user's perception level.

In one embodiment, the obtaining module 802 is further configured to: obtain a jitter parameter of the image frame of the video to be processed, where the jitter parameter includes at least one of a jitter frequency and a jitter amplitude, and is determined according to the jitter parameter the jitter information.

The video anti-shake device 800 in this embodiment acquires the jitter parameter of the image frame of the video to be processed, the jitter parameter includes at least one of the jitter frequency and the jitter amplitude, and determines the jitter information according to the jitter parameter, so that the jitter information can be accurately determined.

In one embodiment, the obtaining module 802 is further configured to: obtain a pre-stored second mapping relationship between the jitter parameter and the jitter information, and determine according to the jitter parameter and the second mapping relationship the jitter information.

The video anti-shake device 800 in this embodiment obtains a second mapping relationship between the pre-stored jitter parameters and the jitter information, and determines the jitter information according to the jitter parameters and the second mapping relationship, so as to accurately determine the jitter information.

In one embodiment, the obtaining module 802 is further configured to: obtain a pre-stored third mapping relationship between the first blur information, the jitter information and the anti-shake information, according to the first The blur information, the jitter information, and the third mapping relationship determine the anti-shake information.

The video anti-shake apparatus 800 in this embodiment acquires the pre-stored first blur information, the third mapping relationship between the jitter information and the anti-shake information, and determines the anti-shake information according to the first blur information, the jitter information and the third mapping relationship , in this way, the degree of shaking of the video is adjusted according to the degree of blurring of the video and the actual degree of shaking, and the stability of the video is realized at the level of user perception.

In one embodiment, the processing module 806 is further configured to: acquire a camera intrinsic parameter matrix, and perform projection transformation on the image frame according to the camera intrinsic parameter matrix and the anti-shake parameter, so as to perform a projection transformation on the video to be processed. Anti-shake processing.

The video anti-shake device 800 in this implementation obtains the camera internal parameter matrix, and performs projection transformation on the image frame according to the camera internal parameter matrix and anti-shake parameters, so as to perform anti-shake processing on the video to be processed. In this way, the video is anti-shake according to the EIS electronic anti-shake. Shaking processing reduces the impact of motion blur on the video, improves the shooting quality of the video, and adjusts the degree of shaking of the video according to the degree of blurring of the video, which realizes the stability of the video at the level of user perception.

For specific limitations on the video anti-shake device, reference may be made to the above limitations on the video anti-shake method, which will not be repeated here. Each module in the above video anti-shake device may be implemented in whole or in part by software, hardware and combinations thereof. The above modules can be embedded in or independent of the processor in the computer device in the form of hardware, or stored in the memory in the computer device in the form of software, so that the processor can call and execute the operations corresponding to the above modules.

FIG. 9 is a schematic diagram of the internal structure of an electronic device in one embodiment. As shown in FIG. 9, the electronic device includes a processor and a memory connected by a system bus. Among them, the processor is used to provide computing and control capabilities to support the operation of the entire electronic device. The memory may include non-volatile storage media and internal memory. The nonvolatile storage medium stores an operating system and a computer program. The computer program can be executed by the processor to implement a video anti-shake method provided by the above embodiments. Internal memory provides a cached execution environment for operating system computer programs in non-volatile storage media. The electronic device may be a mobile phone, a tablet computer, a personal digital assistant or a wearable device, and the like.

The implementation of each module in the video anti-shake apparatus provided in the embodiments of the present application may be in the form of a computer program. The computer program can be run on a terminal or server. The program modules constituted by the computer program can be stored in the memory of the terminal or the server. When the computer program is executed by the processor, the steps of the methods described in the embodiments of the present application are implemented.

Embodiments of the present application also provide a computer-readable storage medium. One or more non-volatile computer-readable storage media containing computer-executable instructions, when executed by one or more processors, cause the processors to perform the steps of the video stabilization method.

A computer program product containing instructions, when run on a computer, causes the computer to perform a video stabilization method.

Any reference to a memory, storage, database, or other medium as used herein may include non-volatile and/or volatile memory. Nonvolatile memory may include read only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), or flash memory. Volatile memory may include random access memory (RAM), which acts as external cache memory. By way of illustration and not limitation, RAM is available in various forms such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDR SDRAM), enhanced SDRAM (ESDRAM), synchronous Link (Synchlink) DRAM (SLDRAM), Memory Bus (Rambus) Direct RAM (RDRAM), Direct Memory Bus Dynamic RAM (DRDRAM), and Memory Bus Dynamic RAM (RDRAM).

The above-mentioned embodiments only represent several embodiments of the present application, and the descriptions thereof are relatively specific and detailed, but should not be construed as a limitation on the scope of the patent of the present application. It should be pointed out that for those skilled in the art, without departing from the concept of the present application, several modifications and improvements can be made, which all belong to the protection scope of the present application. Therefore, the scope of protection of the patent of the present application shall be subject to the appended claims.

Claims (12)

1. A video anti-shake method, comprising:

acquiring first fuzzy information of an image frame of a video to be processed, wherein the first fuzzy information is used for representing the fuzzy degree of the image frame of the video to be processed;

determining anti-shake information according to first fuzzy information of an image frame of the video to be processed, wherein the anti-shake information is used for representing the anti-shake degree of the video to be processed;

determining anti-shake parameters of the video to be processed according to the anti-shake information;

and carrying out anti-shake processing on the video to be processed according to the anti-shake parameters.

2. The method of claim 1, wherein the obtaining first blur information of image frames of the video to be processed comprises:

acquiring second fuzzy information and third fuzzy information of an image frame of the video to be processed, wherein the second fuzzy information is fuzzy information generated by the motion of a shooting object, and the third fuzzy information is fuzzy information generated by the motion of a shooting terminal;

and determining the first fuzzy information according to the second fuzzy information and the third fuzzy information.

3. The method according to claim 1, wherein the first blur information and the anti-shake information are in a first mapping relationship.

4. The method of claim 1, wherein determining anti-shake information from first blur information of image frames of the video to be processed comprises:

acquiring a first mapping relation between the pre-stored first fuzzy information and the anti-shake information;

and determining the anti-shake information according to the first fuzzy information and the first mapping relation.

5. The method of claim 1, wherein before determining anti-shake information from first blur information of image frames of the video to be processed, further comprising:

acquiring jitter information of an image frame of the video to be processed, wherein the jitter information is used for representing the jitter degree of the video to be processed;

the determining anti-shake information according to the first blur information of the image frame of the video to be processed includes:

and determining the anti-shake information according to the first fuzzy information and the shake information.

6. The method according to claim 5, wherein the obtaining the dithering information of the image frames of the video to be processed comprises:

acquiring jitter parameters of image frames of the video to be processed, wherein the jitter parameters comprise at least one of jitter frequency and jitter amplitude;

and determining the jitter information according to the jitter parameters.

7. The method of claim 6, wherein the determining the jitter information according to the jitter parameter comprises:

acquiring a second mapping relation between the pre-stored jitter parameters and the jitter information;

and determining the jitter information according to the jitter parameter and the second mapping relation.

8. The method according to claim 5, wherein the determining the anti-shake information according to the first blur information and the shake information comprises:

acquiring prestored first fuzzy information, and a third mapping relation between the jitter information and the anti-jitter information;

and determining the anti-shake information according to the first fuzzy information, the shake information and the third mapping relation.

9. The method according to claim 1, wherein the anti-shake processing the video to be processed according to the anti-shake parameters comprises:

acquiring a camera internal reference matrix;

and performing projection transformation on the image frame according to the camera internal parameter matrix and the anti-shake parameters so as to perform anti-shake processing on the video to be processed.

10. A video anti-shake apparatus, comprising:

the device comprises an acquisition module, a processing module and a processing module, wherein the acquisition module is used for acquiring first fuzzy information of an image frame of a video to be processed, and the first fuzzy information is used for representing the fuzzy degree of the image frame of the video to be processed;

the determining module is used for determining anti-shake information according to first fuzzy information of an image frame of the video to be processed, wherein the anti-shake information is used for representing the anti-shake degree of the video to be processed;

the determining module is further configured to determine anti-shake parameters of the video to be processed according to the anti-shake information;

and the processing module is used for carrying out anti-shake processing on the video to be processed according to the anti-shake parameters.

11. An electronic device comprising a memory and a processor, the memory having stored therein a computer program that, when executed by the processor, causes the processor to perform the steps of the video anti-shake method according to any of claims 1-9.

12. A computer-readable storage medium, on which a computer program is stored, which, when being executed by a processor, carries out the steps of the video anti-shake method according to any one of claims 1 to 9.

Images