CN111861900A - Video image matting and noise reduction method and device - Google Patents
Video image matting and noise reduction method and device Download PDFInfo
- Publication number
- CN111861900A CN111861900A CN202010504351.1A CN202010504351A CN111861900A CN 111861900 A CN111861900 A CN 111861900A CN 202010504351 A CN202010504351 A CN 202010504351A CN 111861900 A CN111861900 A CN 111861900A
- Authority
- CN
- China
- Prior art keywords
- video frame
- frame image
- current video
- noise reduction
- matting
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T5/00—Image enhancement or restoration
- G06T5/70—Denoising; Smoothing
-
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T5/00—Image enhancement or restoration
- G06T5/20—Image enhancement or restoration using local operators
-
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/10—Segmentation; Edge detection
- G06T7/11—Region-based segmentation
-
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/10—Segmentation; Edge detection
- G06T7/194—Segmentation; Edge detection involving foreground-background segmentation
-
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/10—Image acquisition modality
- G06T2207/10016—Video; Image sequence
-
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20024—Filtering details
- G06T2207/20032—Median filtering
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Picture Signal Circuits (AREA)
Abstract
The present disclosure relates to a video image matting and noise reduction method and device, wherein the video image matting and noise reduction method comprises obtaining pixel point noise reduction matting information of a previous video frame image and pixel point matting information of a current video frame image; determining a pixel point damping coefficient of the current video frame image based on the pixel point keying information of the current video frame image; determining the noise reduction keying information of the pixel points of the current video frame image based on the noise reduction keying information of the pixel points of the previous video frame image, the keying information of the pixel points of the current video frame image and the damping coefficient of the pixel points of the current video frame image; and based on the noise reduction and image matting information of the pixel points of the current video frame image, carrying out noise reduction and image matting processing on the current video frame image to obtain a noise reduction and image matting image of the current video frame image. By the method and the device, the signal-to-noise ratio of the keying result of the current video frame image is improved, and the problems of delay, smear and the like of the picture of the current video frame image are avoided.
Description
Technical Field
The disclosure relates to the technical field of image matting and noise reduction, in particular to a method and a device for video image matting and noise reduction.
Background
Matting is a technique for identifying the region of a specific object in a picture. The image matting result of one frame of image is a single-channel image, wherein the pixel points with higher brightness correspond to foreground elements in the original image, and the pixel points with lower brightness correspond to background elements in the original image. Through the single-channel graph, which pixel points in the image are foreground and which pixel points are background can be distinguished.
Because a large amount of noise often exists in the image subjected to image matting processing, in order to avoid the interference of the noise on the image application effect, time domain noise reduction is often adopted to perform noise reduction processing on the image at present. However, the use of temporal noise reduction can cause problems such as picture delay, smearing, etc.
Disclosure of Invention
In order to overcome the problems in the prior art, the present disclosure provides a method and an apparatus for video matting and noise reduction.
In a first aspect, an embodiment of the present disclosure provides a method for video matting and noise reduction. The video image matting and noise reduction method comprises the following steps: acquiring noise reduction and image matting information of pixel points of a previous video frame image and pixel point image matting information of a current video frame image; determining a pixel point damping coefficient of the current video frame image based on the pixel point keying information of the current video frame image; determining the noise reduction keying information of the pixel points of the current video frame image based on the noise reduction keying information of the pixel points of the previous video frame image, the keying information of the pixel points of the current video frame image and the damping coefficient of the pixel points of the current video frame image; and based on the noise reduction and image matting information of the pixel points of the current video frame image, carrying out noise reduction and image matting processing on the current video frame image to obtain a noise reduction and image matting image of the current video frame image.
In one embodiment, the pixel matting information of the current video frame image represents the probability that the corresponding pixel in the current video frame image is a foreground or a background; if the probability that the pixel point keying information of the current video frame image is the foreground is higher, the pixel point damping coefficient of the current video frame image is higher, and if the probability that the pixel point keying information of the current video frame image is the background is higher, the pixel point damping coefficient of the current video frame image is higher.
In another embodiment, the pixel point damping coefficient of the current video frame image is determined based on the pixel point keying information of the current video frame image, and the determination is realized by the following formula:
wherein k isijThe damping coefficient of a pixel point of the current video frame image is obtained; sijFor the keying information of the pixel point of the current video frame image, S is more than or equal to 0ijLess than or equal to 1; γ is a first parameter.
In a further embodiment, the first parameter γ is defined by: gamma is more than or equal to 2 and less than or equal to 4.
In another embodiment, determining the noise reduction and matting information of the pixel point of the current video frame image based on the noise reduction and matting information of the pixel point of the previous video frame image, the noise reduction and matting information of the pixel point of the current video frame image, and the damping coefficient of the pixel point of the current video frame image includes: determining a first weight of noise reduction keying information of a pixel point of a previous video frame image and a second weight of the keying information of the pixel point of the current video frame image based on a pixel point damping coefficient of the current video frame image; and based on the first weight and the second weight, carrying out weighted summation on the noise reduction and keying information of the pixel points of the previous video frame image and the pixel point keying information of the current video frame image to obtain the noise reduction and keying information of the pixel points of the current video frame image.
In another embodiment, determining a first weight of noise reduction keying information of a pixel of a previous video frame image and a second weight of noise reduction keying information of the pixel of a current video frame image based on a pixel damping coefficient of the current video frame image comprises: taking the pixel point damping coefficient of the current video frame image as a second weight of the pixel point keying information of the current video frame image; and determining a first weight based on the second weight, wherein the sum of the first weight and the second weight is 1.
In yet another embodiment, pixel matting information for a current video frame image is determined from a neural network model.
In another embodiment, the method for video matting and noise reduction further comprises: determining noise reduction and image matting information of pixel points of a preset video frame image; and if the current video frame image is the first frame image, taking the pixel point noise reduction and matting information of the preset video frame image as the pixel point noise reduction and matting information of the previous video frame image.
In another embodiment, based on the noise reduction and image matting information of the pixel point of the current video frame image, the noise reduction and image matting processing is performed on the current video frame image to obtain a noise reduction and image matting image of the current video frame image, including: based on the pixel point noise reduction and image matting information of the current video frame image, image matting and noise reduction processing is carried out on the current video frame image to obtain a first noise reduction image of the current video frame image; and carrying out median filtering processing on the first noise reduction image to obtain a noise reduction image matting image of the current video frame image.
In a second aspect, the disclosed embodiments provide a video image matting and noise reduction device. The video image matting and noise reduction device comprises an acquisition module, a processing module and a processing module, wherein the acquisition module is used for acquiring pixel point noise reduction and matting information of a previous video frame image and pixel point matting information of a current video frame image; the determining module is used for determining a pixel damping coefficient of the current video frame image based on the pixel matting information of the current video frame image; the method comprises the steps of obtaining pixel point noise reduction and matting information of a current video frame image, and determining the pixel point noise reduction and matting information of the current video frame image based on the pixel point noise reduction and matting information of a previous video frame image, the pixel point matting information of the current video frame image and a pixel point damping coefficient of the current video frame image; and the processing module is used for carrying out noise reduction and image matting processing on the current video frame image based on the noise reduction and image matting information of the pixel points of the current video frame image to obtain a noise reduction and image matting image of the current video frame image.
In a third aspect, an embodiment of the present disclosure provides an electronic device, where the electronic device includes: a memory to store instructions; and the processor is used for calling the instructions stored in the memory to execute the video matting and noise reduction method in the first aspect or any one implementation manner of the first aspect.
In a fourth aspect, the present disclosure provides a computer-readable storage medium, where the computer-readable storage medium stores computer-executable instructions, and when executed by a processor, the computer-executable instructions perform the video matting and denoising method described in the first aspect or any one of the implementation manners of the first aspect.
The utility model provides a video keying and denoising method, which dynamically sets a pixel damping coefficient for each pixel of the current video frame image by reasonably utilizing the pixel denoising keying information of the previous video frame image and the pixel keying information of the current video frame image so as to perform denoising treatment on the keying result of the current video frame image, thereby improving the keying result signal to noise ratio of the current video frame image and avoiding the problems of delay, smear and the like of the picture of the current video frame image.
Drawings
The above and other objects, features and advantages of the embodiments of the present disclosure will become readily apparent from the following detailed description read in conjunction with the accompanying drawings. Embodiments of the present disclosure are illustrated by way of example, and not by way of limitation, in the figures of the accompanying drawings and in which:
fig. 1 shows a flowchart of a method for video matting noise reduction provided by an embodiment of the present disclosure;
Fig. 2 is a flowchart illustrating a step of determining noise reduction matting information for a pixel point of a current video frame image in a video matting noise reduction method provided by an embodiment of the present disclosure;
FIG. 3 is a flowchart illustrating a step of obtaining a noise-reduced matte image of a current video frame image in a video matte noise-reduction method according to an embodiment of the present disclosure;
fig. 4 is a schematic diagram illustrating a video matting noise reduction apparatus provided by an embodiment of the present disclosure;
fig. 5 shows a schematic diagram of an electronic device provided by an embodiment of the present disclosure.
Detailed Description
The principles and spirit of the present disclosure will be described with reference to a number of exemplary embodiments. It is understood that these embodiments are given solely for the purpose of enabling those skilled in the art to better understand and to practice the present disclosure, and are not intended to limit the scope of the present disclosure in any way.
It should be noted that, although the expressions "first", "second", etc. are used herein to describe different modules, steps, data, etc. of the embodiments of the present disclosure, the expressions "first", "second", etc. are merely used to distinguish between different modules, steps, data, etc. and do not indicate a particular order or degree of importance. Indeed, the terms "first," "second," and the like are fully interchangeable.
At present, the image Matting processing of the image is often realized by adopting a neural network algorithm, an optical flow method, a Matting algorithm or a color key technology.
For continuous video frame images, the keying result of each video frame image is a single-channel image, that is, a foreground pixel point or a background pixel point in an original image corresponding to each pixel point can be represented by a brightness value or a gray value of the pixel point in the video frame image, so that the keying processing of the video frame image is realized. However, the video frame image after the image matting processing also has the problem of low signal-to-noise ratio.
The present disclosure provides a video image matting and noise reduction method, which can improve the signal-to-noise ratio of the image matting result of the current video frame image and avoid the problems of delay, smear, etc. of the picture of the current video frame image.
Fig. 1 shows a flowchart of a method for video matting and noise reduction provided by an embodiment of the present disclosure.
In an exemplary embodiment of the present disclosure, as shown in fig. 1, the video matting noise reduction method includes step S101, step S102, step S103, and step S104. The steps will be described separately below.
In step S101, noise reduction and matting information of a pixel point of a previous video frame image and pixel point matting information of a current video frame image are obtained.
Matting is a technique for identifying the area of a particular object in an image. Because the image is composed of the pixel points, the keying processing of the specific object in the image can be realized by identifying the region where each pixel point in the image is located.
In one embodiment, the video frame image may be subjected to matting processing by a neural network model. The identification of the area where each pixel point in the image is located is realized through the possibility of identifying the position of each pixel point in the video frame image. The pixel matting information of the current video frame image is information for identifying the position of the pixel. For example, the probability that the pixel point is located in the foreground region of the current video frame image or the probability that the pixel point is located in the background region of the current video frame image.
And the pixel point denoising and matting information of the previous video frame image is the pixel point matting information of the previous video frame image which is subjected to denoising processing.
In step S102, a pixel damping coefficient of the current video frame image is determined based on the pixel matting information of the current video frame image.
In step S103, based on the noise reduction and matting information of the pixel point of the previous video frame image, the pixel point matting information of the current video frame image, and the pixel point damping coefficient of the current video frame image, the noise reduction and matting information of the pixel point of the current video frame image is determined.
And for the next video frame image, the determined pixel point noise reduction keying information of the current video frame image is the pixel point noise reduction keying information of the previous video frame image.
Because the pixel point keying information of the current video frame image is the information for identifying the position of the pixel point, namely, the probability information for representing the position of the pixel point in the foreground or background area, the pixel point will probably be in the area between the foreground and the background. If the pixel point cannot be judged to belong to the foreground region or the background region, the noise appears in the keying result of the current video frame image.
If the pixel point can be judged to belong to the foreground area or the background area, the larger the damping coefficient of the pixel point of the current video frame image is. Furthermore, when determining the noise reduction and keying information of the pixel point of the current video frame image, more reference can be made to the noise reduction and keying information of the pixel point of the current video frame image, and relatively less reference can be made to the noise reduction and keying information of the pixel point of the previous video frame image.
In step S104, based on the noise reduction and image matting information of the pixel points of the current video frame image, the current video frame image is subjected to noise reduction and image matting processing to obtain a noise reduction and image matting image of the current video frame image.
The video image matting and denoising method provided by the disclosure reprocesses a current video frame image based on pixel point denoising and matting information of a previous video frame image, pixel point matting information of a current video frame image and a pixel point damping coefficient of the current video frame image to obtain pixel point denoising and matting information of the current video frame image so as to realize denoising processing of a matting result of the current video frame image. The method and the device have the advantages that the noise reduction keying information of the pixel point of the previous video frame image and the pixel point keying information of the current video frame image are reasonably utilized, the pixel point damping coefficient is dynamically set for each pixel point of the current video frame image, the noise reduction processing is carried out on the keying result of the current video frame image, the signal to noise ratio of the keying result of the current video frame image is improved, and the problems of delay, trailing and the like of the picture of the current video frame image are avoided.
In an exemplary embodiment of the present disclosure, the pixel matting information of the current video frame image represents the probability that the corresponding pixel in the current video frame image is a foreground or a background.
If the probability that the keying information of the pixel points of the current video frame image is foreground is higher, the pixel point damping coefficient of the current video frame image is higher; and if the probability that the keying information of the pixel points of the current video frame image is the background is higher, the larger the damping coefficient of the pixel points of the current video frame image is.
If the probability that the pixel keying information of the current video frame image is foreground is smaller and the probability that the pixel keying information is background is smaller, the pixel damping coefficient of the current video frame image is smaller.
In an exemplary embodiment of the present disclosure, determining a pixel damping coefficient of a current video frame image based on pixel matting information of the current video frame image may be implemented by the following formula:
wherein k isijAnd the damping coefficient of the pixel point of the current video frame image. SijFor the keying information of the pixel point of the current video frame image, S is more than or equal to 0ijLess than or equal to 1. γ is a first parameter.
Pixel matting information S of current video frame imageijInformation identifying the location of the pixel. In an embodiment, the identification of the position of each pixel point in the video frame image can be realized in a digital manner.
The value range for marking the position of each pixel point is [0,1 ]]I.e. 0. ltoreq.SijLess than or equal to 1. Wherein S isijThe pixel point with larger value corresponds to the foreground pixel point in the original image, SijAnd the pixel points with smaller values correspond to background pixel points in the original image.
When S isijWhen the value is 1, the probability that the pixel point corresponds to a foreground pixel point in the original image is the largest, namely the pixel point keying information S of the current video frame image is represented ijThe probability of being foreground is the greatest, at which time the corresponding damping coefficient kijAnd max.
When S isijWhen the value is 0, the probability that the pixel point corresponds to the background pixel point in the original image is the largest, namely the pixel point keying information S of the current video frame image is representedijThe probability of being the background is the maximum, at which time the corresponding damping coefficient kijAnd max.
When S isijWhen the value is 0.5, it cannot be determined whether the pixel point corresponds to a foreground pixel point in the original image or corresponds to a background pixel point in the original image, that is, the pixel point keying information S representing the current video frame imageijThe probability of being foreground is minimal and the probability of being background is minimal, at which point the corresponding damping coefficient kijAnd minimum.
In an embodiment, the pixel matting information s of the current video frame image can be obtained through calculation of a neural network modelij(0≤Sij≤1)。
Pixel point image matting information based on current video frame imageThe keying result of the current video frame image can be obtained
Wherein 0 is less than or equal to Sij≤1。
Calculating the corresponding pixel point damping coefficient k of the current video frame image by respectively calculating all pixel points of the current video frame imageijAnd obtaining a damping coefficient matrix containing all pixel points of the current video frame image
Pixel point noise reduction and image matting information F based on previous video frame image ijThe noise reduction and image matting result of the previous video frame image can be obtained
Based on the matting and denoising result F of the previous video certificate image, the matting result S of the current video frame image and the damping coefficient matrix K of the current video frame image, the denoising matting result F' of the current video frame image can be determined. Wherein, the pixel point noise reduction keying information of all the current video frame images forms the noise reduction keying result F' of the current video frame images.
In an exemplary embodiment of the present disclosure, the first parameter γ may be defined by:
2≤γ≤4。
if the first parameter gamma is too small, the noise reduction effect of the image matting result of the current video frame image is poor; if the first parameter γ is too large, the image matting result of the current video frame image after noise reduction will have a problem of image delay.
Fig. 2 shows a flowchart of a step of determining noise reduction matting information for a pixel point of a current video frame image in a video matting noise reduction method provided by an embodiment of the present disclosure.
In an exemplary embodiment of the present disclosure, as shown in fig. 2, the step of determining the noise reduction and matting information of the pixel point of the current video frame image includes step S201 and step S202 based on the noise reduction and matting information of the pixel point of the previous video frame image, the noise reduction and matting information of the pixel point of the current video frame image, and the damping coefficient of the pixel point of the current video frame image. Step S201 and step S202 will be described separately below.
In step S201, a first weight of the noise reduction keying information of the pixel point of the previous video frame image and a second weight of the pixel point keying information of the current video frame image are determined based on the pixel point damping coefficient of the current video frame image.
In step S202, based on the first weight and the second weight, weighted summation is performed on the noise reduction and keying information of the pixel point of the previous video frame image and the pixel point keying information of the current video frame image to obtain the noise reduction and keying information of the pixel point of the current video frame image.
And adjusting the contribution of the noise reduction and matting information of the pixel points of the previous video frame image and the contribution of the noise reduction and matting information of the pixel points of the current video frame image to the noise reduction and matting information of the pixel points of the current video frame image through the first weight and the second weight. The method comprises the steps of setting a pixel damping coefficient for each pixel of a current video frame image by reasonably utilizing noise reduction and keying information of the pixel of a previous video frame image and pixel keying information of the current video frame image, thereby eliminating noise of the keying information of the pixel of the current video frame image on the premise of not generating delay and smear, namely obtaining noise reduction and keying information of the pixel of the current video frame image with high signal-to-noise ratio.
In an exemplary embodiment of the present disclosure, determining a first weight of noise reduction keying information of a pixel of a previous video frame image and a second weight of the keying information of the pixel of the current video frame image based on a pixel damping coefficient of the current video frame image comprises the following steps.
And taking the pixel point damping coefficient of the current video frame image as a second weight of the pixel point keying information of the current video frame image.
Determining a first weight based on a second weight, wherein the sum of the first weight and the second weight is 1.
Because the damping coefficient k of the pixel points of all the current video frame imagesijForm the currentA damping coefficient matrix K of the video frame image; the noise reduction and image matting information of the pixel points of all the previous video frame images forms a noise reduction and image matting result F of the previous video frame images; pixel matting information s of all current video frame imagesijAnd forming a keying result S of the current video frame image. Therefore, for the keying result S of the current video frame image, the second weight of the keying result S is the damping coefficient matrix K; for the noise reduction matting result F of the previous video frame image, the first weight of the noise reduction matting result F is a damping coefficient matrix (1-K).
Noise reduction and image matting result of current video frame image 
Wherein,
which is the EntryProduct (chinese interpreted as fractional multiplication), is an expression of a matrix operation that represents the multiplication of each corresponding element between matrices.
Since the second weight of the matting result S for the current video frame image is the damping coefficient matrix K. The damping coefficient matrix K is according to the pixel point damping coefficient of the current video frame image
And is determined.
Therefore, if the pixel point keying information s of the current video frame imageijThe closer to 1 or the closer to 0, that is, when it can be more accurately determined that the pixel point of the current video frame image corresponds to the foreground pixel point in the original image or corresponds to the background pixel point in the original image, the larger the second weight of the matting result S of the current video frame image is. Namely, the denoising matting result F' of the current video frame image can refer to the matting result S of the current video frame image more.
If the pixel point keying information s of the current video frame imageijThe closer to 0.5, the more the current video frame image is, the more the foreground pixel point or the background pixel point in the original image corresponding to the pixel point of the current video frame image can not be accurately judged, and the current video frame image isThe smaller the second weight of the matting result S of a frame image is, and the larger the first weight of the noise reduction matting result F of the previous video frame image is. That is, the noise reduction matting result F' of the current video frame image can refer to the noise reduction matting result F of the previous video frame image more.
Through the embodiment, the denoising matting result F of the previous video frame image and the matting result S of the current video frame image can be reasonably utilized to denoise the matting result of the current video frame image, so that the signal to noise ratio of the matting result of the current video frame image is improved, and the problems of delay, smear and the like of the picture of the current video frame image are avoided.
In an exemplary embodiment of the present disclosure, the video matting and denoising method further includes determining denoising matting information of pixels of a preset video frame image.
And if the current video frame image is the first frame image, taking the pixel point noise reduction and matting information of the preset video frame image as the pixel point noise reduction and matting information of the previous video frame image.
In an embodiment, if the current video frame image is the first frame image and the current video frame image is to obtain the foreground region through image matting processing, the noise reduction and image matting information of the pixel points of all the preset video frame images can be made to correspond to the information of the positions of the foreground pixel points. If the current video frame image is the first frame image and the current video frame image is to obtain the background region through image matting processing, the noise reduction and image matting information of all the pixel points of the preset video frame image can be made to correspond to the information of the positions of the background pixel points.
In an embodiment, the value range of the position of each pixel point is identified as [0,1], and the pixel point with a larger value corresponds to a foreground pixel point in the original image. If the value of the pixel point is 1, the pixel point can be represented to be a foreground pixel point in the original image; if the value of the pixel point is 0, it can be indicated that the pixel point is corresponding to a background pixel point in the original image.
If the current video frame image is the first frame image and the current video frame image is subjected to image matting processing to obtain a foreground region, reducing the pixel points of the previous video frame imageNoise keying information FijThe value is 1, and the noise reduction and image matting information F of the pixel points of all the previous video frame imagesijNoise reduction and image matting result for forming previous video frame image
If the current video frame image is the first frame image and the current video frame image is subjected to image matting processing to obtain a background area, the noise reduction and image matting information F of the pixel point of the previous video frame imageijThe values are all 0, and the noise reduction and image matting information F of the pixel points of all the previous video frame imagesijNoise reduction and image matting result for forming previous video frame image
Fig. 3 shows a flowchart of a step of obtaining a noise reduction matte image of a current video frame image in a video matte noise reduction method provided by an embodiment of the present disclosure.
In an exemplary embodiment of the present disclosure, as shown in fig. 3, performing noise reduction and image matting processing on a current video frame image based on noise reduction and image matting information of a pixel point of the current video frame image to obtain a noise reduction and image matting image of the current video frame image includes step S301 and step S302, and step S301 and step S302 will be described below respectively.
In step S301, based on the noise reduction matting information of the pixel points of the current video frame image, the current video frame image is subjected to matting noise reduction processing to obtain a first noise reduction image of the current video frame image.
In step S302, a median filtering process is performed on the first noise-reduced image to obtain a noise-reduced image-matting image of the current video frame image.
By carrying out median filtering processing on the first noise reduction image, the signal to noise ratio of the matting result of the current video frame image can be further ensured, so that the denoising effect of the matting result of the current video frame image is better.
Based on the same inventive concept, the embodiment of the disclosure also provides a video image matting and noise reduction device.
Fig. 4 shows a schematic diagram of a video matting noise reduction apparatus provided by an embodiment of the present disclosure.
In an exemplary embodiment of the present disclosure, as shown in fig. 4, the video matting noise reduction apparatus includes an acquisition module 201, a determination module 202, and a processing module 203. The acquisition module 201, the determination module 202 and the processing module 203 will be described separately below.
The obtaining module 201 is configured to obtain noise reduction keying information of a pixel point of a previous video frame image and pixel point keying information of a current video frame image.
The determining module 202 is configured to determine a pixel damping coefficient of the current video frame image based on pixel matting information of the current video frame image; and the method is used for determining the pixel point noise reduction keying information of the current video frame image based on the pixel point noise reduction keying information of the previous video frame image, the pixel point keying information of the current video frame image and the pixel point damping coefficient of the current video frame image.
The processing module 203 is configured to perform noise reduction and image matting processing on the current video frame image based on the noise reduction and image matting information of the pixel point of the current video frame image, so as to obtain a noise reduction and image matting image of the current video frame image.
In an exemplary embodiment of the present disclosure, the pixel matting information of the current video frame image represents the probability that the corresponding pixel in the current video frame image is a foreground or a background; if the probability that the pixel point keying information of the current video frame image is the foreground is higher, the pixel point damping coefficient of the current video frame image is higher, and if the probability that the pixel point keying information of the current video frame image is the background is higher, the pixel point damping coefficient of the current video frame image is higher.
In an exemplary embodiment of the present disclosure, the determining module 202 determines the pixel damping coefficient of the current video frame image based on the pixel matting information of the current video frame image according to the following formula:
wherein k isijThe damping coefficient of a pixel point of the current video frame image is obtained; sijFor the keying information of the pixel point of the current video frame image, S is more than or equal to 0ijLess than or equal to 1; γ is a first parameter.
In an exemplary embodiment of the present disclosure, the determining module 202 defines the first parameter γ by:
2≤γ≤4。
in an exemplary embodiment of the disclosure, the determining module 202 is further configured to: determining a first weight of noise reduction keying information of a pixel point of a previous video frame image and a second weight of the keying information of the pixel point of the current video frame image based on a pixel point damping coefficient of the current video frame image; and based on the first weight and the second weight, carrying out weighted summation on the noise reduction and keying information of the pixel points of the previous video frame image and the pixel point keying information of the current video frame image to obtain the noise reduction and keying information of the pixel points of the current video frame image.
In an exemplary embodiment of the disclosure, the determining module 202 is further configured to: taking the pixel point damping coefficient of the current video frame image as a second weight of the pixel point keying information of the current video frame image; and determining a first weight based on the second weight, wherein the sum of the first weight and the second weight is 1.
In an exemplary embodiment of the present disclosure, the pixel matting information of the current video frame image is determined according to a neural network model.
In an exemplary embodiment of the present disclosure, the determining module 202 is further configured to determine noise reduction and keying information of pixel points of a preset video frame image; and if the current video frame image is the first frame image, taking the pixel point noise reduction and matting information of the preset video frame image as the pixel point noise reduction and matting information of the previous video frame image.
In an exemplary embodiment of the present disclosure, the processing module 203 is configured to perform image matting and noise reduction processing on a current video frame image based on noise reduction and image matting information of a pixel point of the current video frame image to obtain a first noise reduction image of the current video frame image; and carrying out median filtering processing on the first noise reduction image to obtain a noise reduction image matting image of the current video frame image.
Fig. 5 illustrates an electronic device 30 provided by an embodiment of the present disclosure. As shown in fig. 5, an embodiment of the present disclosure provides an electronic device 30, where the electronic device 30 includes a memory 310, a processor 320, and an Input/Output (I/O) interface 330. The memory 310 is used for storing instructions. A processor 320 for calling the instructions stored in the memory 310 to execute the method for video matting noise reduction of the present disclosure. The processor 320 is connected to the memory 310 and the I/O interface 330, respectively, for example, via a bus system and/or other connection mechanism (not shown). The memory 310 may be used to store programs and data, including the programs for video matting and noise reduction involved in the embodiments of the present disclosure, and the processor 320 executes various functional applications and data processing of the electronic device 30 by running the programs stored in the memory 310.
In the embodiment of the present disclosure, the processor 320 may be implemented in at least one hardware form of a Digital Signal Processor (DSP), a Field Programmable Gate Array (FPGA), a Programmable Logic Array (PLA), and the processor 320 may be one or a combination of a Central Processing Unit (CPU) or other Processing units with data Processing capability and/or instruction execution capability.
In the disclosed embodiment, the I/O interface 330 may be used to receive input instructions (e.g., numeric or character information, and generate key signal inputs related to user settings and function control of the electronic device 30, etc.), and may also output various information (e.g., images or sounds, etc.) to the outside. The I/O interface 330 in embodiments of the present disclosure may include one or more of a physical keyboard, function keys (e.g., volume control keys, switch keys, etc.), a mouse, a joystick, a trackball, a microphone, a speaker, a touch panel, and the like.
In some embodiments, the present disclosure provides a computer-readable storage medium having stored thereon computer-executable instructions that, when executed by a processor, perform any of the methods described above.
Although operations are depicted in the drawings in a particular order, this should not be understood as requiring that such operations be performed in the particular order shown or in serial order, or that all illustrated operations be performed, to achieve desirable results. In certain environments, multitasking and parallel processing may be advantageous.
The methods and apparatus of the present disclosure can be accomplished with standard programming techniques with rule-based logic or other logic to accomplish the various method steps. It should also be noted that the words "means" and "module," as used herein and in the claims, is intended to encompass implementations using one or more lines of software code, and/or hardware implementations, and/or equipment for receiving inputs.
Any of the steps, operations, or procedures described herein may be performed or implemented using one or more hardware or software modules, alone or in combination with other devices. In one embodiment, the software modules are implemented using a computer program product comprising a computer readable medium containing computer program code, which is executable by a computer processor for performing any or all of the described steps, operations, or procedures.
The foregoing description of the implementations of the disclosure has been presented for purposes of illustration and description. It is not intended to be exhaustive or to limit the disclosure to the precise form disclosed, and modifications and variations are possible in light of the above teachings or may be acquired from practice of the disclosure. The embodiments were chosen and described in order to explain the principles of the disclosure and its practical application to enable one skilled in the art to utilize the disclosure in various embodiments and with various modifications as are suited to the particular use contemplated.
Claims (12)
1. A method for video matting noise reduction, the method comprising:
acquiring noise reduction and image matting information of pixel points of a previous video frame image and pixel point image matting information of a current video frame image;
determining a pixel point damping coefficient of the current video frame image based on the pixel point keying information of the current video frame image;
determining the noise reduction and keying information of the pixel points of the current video frame image based on the noise reduction and keying information of the pixel points of the previous video frame image, the pixel point keying information of the current video frame image and the pixel point damping coefficient of the current video frame image;
And based on the noise reduction and image matting information of the pixel points of the current video frame image, carrying out noise reduction and image matting processing on the current video frame image to obtain a noise reduction and image matting image of the current video frame image.
2. The method of video matting noise reduction according to claim 1,
the pixel point keying information of the current video frame image represents the probability that the corresponding pixel point in the current video frame image is a foreground or a background;
if the probability that the keying information of the pixel points of the current video frame image is foreground is higher, the damping coefficient of the pixel points of the current video frame image is higher,
and if the probability that the keying information of the pixel points of the current video frame image is the background is higher, the larger the damping coefficient of the pixel points of the current video frame image is.
3. The method of video matting and noise reduction according to claim 2, wherein the determining of the pixel point damping coefficient of the current video frame image based on the pixel point matting information of the current video frame image is implemented by the following formula:
wherein k isijThe damping coefficient of the pixel point of the current video frame image is obtained; sijFor the pixel point keying information of the current video frame image, S is more than or equal to 0 ijLess than or equal to 1; γ is a first parameter.
4. The method for video matting noise reduction according to claim 3, characterized in that the first parameter γ is defined by:
2≤γ≤4。
5. the method of claim 1, wherein determining the noise reduction and matting information of the pixels of the current video frame image based on the noise reduction and matting information of the pixels of the previous video frame image, the noise reduction and matting information of the pixels of the current video frame image, and the damping coefficient of the pixels of the current video frame image comprises:
determining a first weight of noise reduction keying information of a pixel point of the previous video frame image and a second weight of the pixel point keying information of the current video frame image based on a pixel point damping coefficient of the current video frame image;
and based on the first weight and the second weight, carrying out weighted summation on the noise reduction keying information of the pixel point of the previous video frame image and the pixel point keying information of the current video frame image to obtain the noise reduction keying information of the pixel point of the current video frame image.
6. The method of claim 5, wherein determining a first weight of pixel point noise reduction matting information of the previous video frame image and a second weight of pixel point matting information of the current video frame image based on a pixel point damping coefficient of the current video frame image comprises:
Taking the pixel point damping coefficient of the current video frame image as a second weight of the pixel point keying information of the current video frame image;
determining the first weight based on the second weight, wherein the sum of the first weight and the second weight is 1.
7. The method of video matting and noise reduction according to claim 1, wherein the pixel matting information of the current video frame image is determined according to a neural network model.
8. The method of video matting noise reduction according to claim 1, characterized in that the method further comprises:
determining noise reduction and image matting information of pixel points of a preset video frame image;
and if the current video frame image is the first frame image, taking the pixel point noise reduction keying information of the preset video frame image as the pixel point noise reduction keying information of the previous video frame image.
9. The method of claim 1, wherein the performing noise reduction and matting processing on the current video frame image based on the noise reduction and matting information of the pixel point of the current video frame image to obtain a noise reduction and matting image of the current video frame image comprises:
based on the pixel point noise reduction and image matting information of the current video frame image, carrying out image matting and noise reduction processing on the current video frame image to obtain a first noise reduction image of the current video frame image;
And carrying out median filtering processing on the first noise reduction image to obtain a noise reduction image matting image of the current video frame image.
10. A video matting and noise reduction apparatus, the apparatus comprising:
the acquisition module is used for acquiring the noise reduction keying information of the pixel point of the previous video frame image and the pixel point keying information of the current video frame image;
the determining module is used for determining a pixel damping coefficient of the current video frame image based on the pixel keying information of the current video frame image; the method comprises the steps of obtaining pixel point noise reduction keying information of a current video frame image, and determining the pixel point noise reduction keying information of the current video frame image based on the pixel point noise reduction keying information of a previous video frame image, the pixel point keying information of the current video frame image and a pixel point damping coefficient of the current video frame image;
and the processing module is used for carrying out noise reduction and image matting processing on the current video frame image based on the noise reduction and image matting information of the pixel points of the current video frame image to obtain a noise reduction and image matting image of the current video frame image.
11. An electronic device, characterized in that the electronic device comprises:
a memory to store instructions; and
A processor for invoking the memory-stored instructions to perform the video matting noise reduction method of any one of claims 1-9.
12. A computer-readable storage medium, characterized in that,
the computer-readable storage medium stores computer-executable instructions that, when executed by a processor, perform the method of video matting noise reduction according to any one of claims 1 to 9.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010504351.1A CN111861900A (en) | 2020-06-05 | 2020-06-05 | Video image matting and noise reduction method and device |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010504351.1A CN111861900A (en) | 2020-06-05 | 2020-06-05 | Video image matting and noise reduction method and device |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN111861900A true CN111861900A (en) | 2020-10-30 |
Family
ID=72985027
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202010504351.1A Pending CN111861900A (en) | 2020-06-05 | 2020-06-05 | Video image matting and noise reduction method and device |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN111861900A (en) |
Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20100238262A1 (en) * | 2009-03-23 | 2010-09-23 | Kurtz Andrew F | Automated videography systems |
| US20130108167A1 (en) * | 2011-10-28 | 2013-05-02 | Raymond William Ptucha | Image Recomposition From Face Detection And Facial Features |
| CN108351685A (en) * | 2015-08-15 | 2018-07-31 | 谷歌有限责任公司 | System and method for the ocular signal based on biomethanics interacted with true and virtual objects |
| CN109919836A (en) * | 2019-03-20 | 2019-06-21 | 广州华多网络科技有限公司 | Video keying processing method, video keying processing client and readable storage medium storing program for executing |
| CN110717914A (en) * | 2019-09-11 | 2020-01-21 | 北京迈格威科技有限公司 | Method and device for human body image matting |
| CN111149141A (en) * | 2017-09-04 | 2020-05-12 | Nng软件开发和商业有限责任公司 | Method and apparatus for collecting and using sensor data from vehicles |
-
2020
- 2020-06-05 CN CN202010504351.1A patent/CN111861900A/en active Pending
Patent Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20100238262A1 (en) * | 2009-03-23 | 2010-09-23 | Kurtz Andrew F | Automated videography systems |
| US20130108167A1 (en) * | 2011-10-28 | 2013-05-02 | Raymond William Ptucha | Image Recomposition From Face Detection And Facial Features |
| CN108351685A (en) * | 2015-08-15 | 2018-07-31 | 谷歌有限责任公司 | System and method for the ocular signal based on biomethanics interacted with true and virtual objects |
| CN111149141A (en) * | 2017-09-04 | 2020-05-12 | Nng软件开发和商业有限责任公司 | Method and apparatus for collecting and using sensor data from vehicles |
| CN109919836A (en) * | 2019-03-20 | 2019-06-21 | 广州华多网络科技有限公司 | Video keying processing method, video keying processing client and readable storage medium storing program for executing |
| CN110717914A (en) * | 2019-09-11 | 2020-01-21 | 北京迈格威科技有限公司 | Method and device for human body image matting |
Non-Patent Citations (2)
| Title |
|---|
| 杨晖;张国华;王宇杰;孙其诚;: "密集颗粒体系的颗粒运动及结构测量技术", 力学进展, no. 00 * |
| 魏闪闪;谢巍;贺志强;: "数字视频稳像技术综述", 计算机研究与发展, no. 09 * |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN108876733B (en) | Image enhancement method, device, equipment and storage medium | |
| CN110827229A (en) | Infrared image enhancement method based on texture weighted histogram equalization | |
| CN110717522A (en) | Countermeasure defense method of image classification network and related device | |
| US11836898B2 (en) | Method and apparatus for generating image, and electronic device | |
| CN109035158B (en) | Image blurring processing method, image blurring processing device, storage medium and electronic equipment | |
| CN113837970B (en) | Desensitizing method and device for image data | |
| CN107979712B (en) | Video noise reduction method and device | |
| CN114418873B (en) | Dark light image noise reduction method and device | |
| CN110751593A (en) | Image blurring processing method and device | |
| CN112967207B (en) | Image processing method and device, electronic equipment and storage medium | |
| KR20230165191A (en) | Self-regularizing inverse filter for image deblurring | |
| CN108846814A (en) | Image processing method, image processing device, readable storage medium and computer equipment | |
| JP2015001981A (en) | Filtering method and filtering apparatus for recovering anti-aliasing edge | |
| CN110689496B (en) | Method and device for determining noise reduction model, electronic equipment and computer storage medium | |
| CN109934784A (en) | Image enhancement method, apparatus, device, and computer-readable storage medium | |
| CN110717914A (en) | Method and device for human body image matting | |
| CN116957962A (en) | Image correction method, device, equipment and storage medium | |
| CN115914765B (en) | Image quality adjusting method, device, equipment and medium | |
| CN114399639A (en) | Semantic segmentation model training method, electronic equipment and storage medium | |
| CN117788830A (en) | Adversarial sample generation method and system for image segmentation | |
| CN113570507A (en) | An image noise reduction method, device, equipment and storage medium | |
| CN110336943B (en) | A scene recognition method and device | |
| CN107404600B (en) | Image processing device and method | |
| CN111861900A (en) | Video image matting and noise reduction method and device | |
| WO2025082108A1 (en) | Image processing method and apparatus, storage medium, and electronic device |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| WD01 | Invention patent application deemed withdrawn after publication | ||
| WD01 | Invention patent application deemed withdrawn after publication |
Application publication date: 20201030 |