CN108174056A - A low-light video noise reduction method based on joint spatio-temporal domain - Google Patents

Abstract

The invention discloses a low-light video noise reduction method combining time and space domains. The method includes: performing boxfilter processing on the original low-light video; performing motion detection on adjacent frames of the processed video sequence, determining corresponding filter coefficients, and The noise reduction processing of the original low-light video is carried out through the three-dimensional coefficient time-domain recursive filtering algorithm; the image enhancement is carried out through the improved bilateral filtering processing method, and the low-light video after noise reduction is obtained. The invention enhances the effect of noise reduction as much as possible while avoiding smear; the time-space hybrid filter solves the defects of low-light video such as poor contrast and low signal-to-noise ratio caused by environmental and device factors, and improves the human eye. Visual effects.

Description

A low-light video noise reduction method based on joint spatio-temporal domain

technical field

The invention belongs to the technical field of video processing, in particular to a low-light video noise reduction method combining time and space domains.

Background technique

Under low-light conditions, due to the low illumination and the limitation of detector sensitivity, the signal-to-noise ratio of the video image obtained by the system is low, which affects the observation of the human eye, and even cannot effectively obtain the target scene image. The noise of low-light video is mainly composed of white noise that conforms to Gaussian distribution produced by CCD and quantum noise produced by image intensifier.

Existing low-light video noise reduction algorithms are mainly divided into two types: spatial filtering and temporal filtering. Temporal filtering uses the correlation between video frames to reduce video noise on the basis of motion detection or motion compensation. Spatial filtering uses the correlation between two-dimensional image pixel neighborhoods for processing, such as mean filtering, Wiener filtering, etc.

Using time-domain filtering alone, such as the time-domain recursive filtering algorithm, has no obvious noise reduction effect on moving target scenes, and errors are prone to occur in target matching, resulting in afterimages. Using spatial filtering alone, filtering median filtering, mean filtering, etc., due to the randomness of the noise at the same position between frames, it is easy to cause flickering between adjacent frames after filtering.

Contents of the invention

The object of the present invention is to provide a low-light video noise reduction method combining time and space.

The technical solution that realizes the object of the present invention is:

A low-light video noise reduction method combining time and space domains, comprising the following steps:

Step 1, perform boxfilter processing on the original low-light video;

Step 2: Perform motion detection on adjacent frames of the processed video sequence, determine corresponding filter coefficients, and perform noise reduction processing on the original low-light video through a three-dimensional coefficient time-domain recursive filtering algorithm;

Step 3, image enhancement is performed through an improved bilateral filtering method to obtain a noise-reduced low-light video.

Compared with prior art, remarkable advantage of the present invention is:

(1) When the present invention utilizes Boxfilter to calculate each ΣA (i, j), only two calculations are required, which simplifies the process of summing one by one, and after Boxfilter preprocessing, a certain value needs to be found in the filtering algorithm When the pixels in the window are summed, you can directly access the corresponding position in the B array, reducing the original complexity of O(9) to the complexity of O(1), and improving the running speed of the algorithm;

(2) On the basis of time domain filtering, the present invention uses an improved bilateral filtering processing method to carry out spatial filtering of video, which solves the defects of low-light video such as poor contrast and low signal-to-noise ratio caused by environmental and device factors, and improves the performance of video. Visual effects of the human eye.

Description of drawings:

FIG. 1 is a flow chart of a low-light video noise reduction algorithm based on the combination of three-dimensional coefficient time-domain recursive filtering and improved bilateral filtering in the present invention.

Figure 2 is a schematic diagram of the boxfilter algorithm.

Figure 3(a) and Figure 3(b) are the screenshots of the original video and the filtered screenshots of the still buildings, respectively.

Figure 4(a) and Figure 4(b) are the screenshots of the original video and the filtered screenshots of the rocking camera under the illumination of 0.11lux, respectively.

Figure 5(a) and Figure 5(b) are the original video screenshots and filtered screenshots of still portraits under 0.02lux illumination, respectively.

Figure 6(a) and Figure 6(b) are the original video screenshots and filtered screenshots of moving portraits under 0.02lux illumination, respectively.

Detailed ways

With reference to Fig. 1, a low-light video noise reduction method of joint spatio-temporal domain of the present invention comprises the following steps:

Step 1, perform boxfilter processing on the original low-light video;

Given the size of the sliding window, quickly add and sum the pixel values in each window, that is, create an array B with the same size as the original image A, so that the value of each element in the array B is the corresponding position of the original image A Sum of pixels in a pixel neighborhood:

B(i,j)=∑A(i,j),(i,j)∈M (1)

Where M is the set of all pixels in the window centered on (i, j).

Let the window size be 3×3, this step specifically includes:

Step 1-1, the size of the filtering window is 3×3, and the length of the middle array mid is equal to the number of columns of pixels in the original image;

Step 1-2, the filter window starts from the upper left corner, slides from left to right, from top to bottom pixel by pixel, each time it moves to a new position, set the center pixel of the window as (i, j), and set the i-1 , i, i+1 rows and each column are summed in turn, and the summation result is placed in the middle array mid, and mid[i, j-1], mid[i, j], mid[i, j+1] are summed , to get ΣA(i, j), A(i, j) is the pixel value whose pixel coordinates are (i, j) in the original image, and store the summation result in point (i, j) of array B;

Step 1-3. When the window is shifted to the right by one pixel, the center pixel of the window becomes (i, j+1), subtract mid[i, j-1] from the last summation result, and add mid [i, j+2], get the sum result of the new window:

ΣA(i, j+1) = ΣA(i, j) - mid[i, j-1]+mid[i, j+2] (2)

Step 1-4. When the window moves to the end of the line and jumps to the next line, update mid. For each mid[i+1, j], add A(i+2, j) and subtract A (i-1, j), and then start the calculation of a new row.

The above process is shown in Figure 2, each dot in the figure represents a pixel in the original image, the square frame represents the selected filter window, the size is 3×3, the square is the middle array, and the length is equal to the number of columns of pixels in the original image ;

Step 2: Perform motion detection on adjacent frames of the processed video sequence, determine the corresponding filter coefficients, and perform noise reduction processing on the original low-light video through the three-dimensional coefficient time-domain recursive filtering algorithm; the specific process is:

Step 2-1, set a difference d of the corresponding window pixel sum of the front and rear frames, and count d=|W _N -W _N-1 | while recursively reducing noise;

The filter coefficient K is set as a piecewise linear function of d:

Wherein, d ₁ and d ₂ are respectively the first difference threshold and the second difference threshold, K ₁ and K ₂ are respectively the first filter coefficient threshold and the second filter coefficient threshold, and K ₁ >K ₂ ;

Step 2-2, the improved time-domain recursive filtering filter formula is as follows:

W _N(i,j) ＝X _N(i,j) +K _N(i,j) (W _(N-1)(i,j) -X _N(i,j) ) (4)

In the formula, W _{N(i, j)} is the window centered on point (i, j) in the filtered output image of the current frame, and W _{(N-1)(i, j)} is the corresponding filter output image of the previous frame The window of the position; X _{N(i, j)} is the window corresponding to the position of the current input image; K _{N(i, j)} is the filter coefficient of the window centered on the point (i, j), K ∈ (0, 1).

Step 3, image enhancement is performed through an improved bilateral filtering method to obtain a noise-reduced low-light video. Based on the similarity principle of pixel grayscale in the window, a compensation function is added to the grayscale similarity factor. The specific process is:

Step 3-1, calculate the spatial proximity factor, the formula is as follows:

Where ω _s (p, q) is the spatial proximity factor, σ _s is the filter parameter, (x, y) is the pixel coordinate of the filter window center, (p, q) is the other pixel coordinates in the window;

Step 3-2. Calculate the improved gray similarity factor

Based on the similarity of pixel grayscale in the window, a compensation function is added to the grayscale similarity factor,

where ω _r (p, q) is the spatial proximity factor, σ _r is the filter parameter, (x, y) is the coordinate of the pixel in the center of the filter window, (p, q) is the coordinate of other pixels in the window, τ(x, y ) is the compensation function, and the setting rules for τ(x, y) are as follows:

1) Judging the similarity between the gray value of the pixel point and the central point in the filter window based on the similarity; if the absolute value of the pixel difference between the pixel point and the central point is less than σ _r /3, then judge I(p,q) and I( x, y) are similar, keep the original value of I(p,q), otherwise I(p,q) is 0;

2) Set the compensation function according to the number of similar points in the window, if the number of pixels set to 0 in the window is less than 1/3 of the number of pixels in the window, then set τ(x, y)=0, otherwise, follow rule 3 )set up.

3) Introduce the variables min, max, and mean, which represent the minimum, maximum, and average values of pixels in the filter window respectively; let a=I(p,q)-mean, if a>0, τ(x,y)= max-I(p,q); if a<0, τ(x,y)=min-I(p,q) if a=0, τ(x,y)=0;

Step 3-3, calculate the filtered image after improving bilateral filtering:

in, is the image obtained after filtering; M _{x, y} is a set of spatial neighborhood pixels centered on (x, y) and radius r; I(p, q) is M _{x, and the coordinates in y} are (p, q ) point pixel value; ω _s (p,q) and ω _r (p,q) are spatial proximity factor and gray similarity factor respectively.

The present invention will be described in detail below in conjunction with the accompanying drawings and specific embodiments.

Example

The present invention selects a plurality of experimental scenes to shoot low-light videos, and performs experimental verification on the video images shot in various situations.

Figure 3(a) and Figure 3(b) are the screenshots of the original video of the still building and the filtered screenshots respectively; Figure 4(a) and Figure 4(b) are the screenshots of the original video and the filtered screenshots of the swinging lens under the illumination of 0.11lux Figure 5(a) and Figure 5(b) are the original video screenshots and filtered schematic diagrams of still portraits under 0.02lux illumination; Figure 6(a) and Figure 6(b) are respectively under 0.02lux illumination The original video screenshot and filtered schematic diagram of a sports portrait.

It can be seen from Fig. 3 and Fig. 5 that for stationary targets, the space-time domain joint algorithm proposed by the present invention can well reduce noise and enhance image edge details and other information.

It can be seen from Fig. 4 and Fig. 6 that for moving objects, the algorithm proposed by the present invention can also complete the noise reduction process well without smear phenomenon.

To sum up, it can be seen that the low-light video noise reduction algorithm proposed by the present invention combines time-space domain to overcome the disadvantage that a single time-domain filter will make the edge features of the video blurred while filtering out noise. Time-domain or space-domain algorithms can achieve better noise reduction effects, provide good video quality, effectively suppress video image noise, and better preserve image details such as edges and textures.

Claims (5)

1. A low-light-level video noise reduction method based on time-space domain combination is characterized by comprising the following steps:

step 1, carrying out boxfilter processing on an original low-light-level video;

step 2, carrying out motion detection on adjacent frames of the processed video sequence, determining corresponding filter coefficients, and carrying out noise reduction processing on the original low-light-level video through a three-dimensional coefficient time domain recursive filter algorithm;

and 3, carrying out image enhancement through an improved bilateral filtering processing method to obtain the low-light-level video subjected to noise reduction.

2. The method for micro-optic video noise reduction in a temporal-spatial domain combination according to claim 1, wherein the step 1 is specifically:

and (3) giving the size of the sliding window, and performing quick addition summation on the pixel values in each window, namely creating an array B with the same size as the original image A, wherein the value of each element in the array B is the sum of pixels in the pixel neighborhood of the corresponding position of the original image A:

B(i，j)＝ΣA(i，j)，(i，j)∈M (1)

where M is the set of all pixel points within the window centered at (i, j).

3. The method for low-light level video noise reduction in combination of time-space domain according to claim 1 or 2, wherein the step 1 is specifically:

step 1-1, setting the size of a filtering window to be 3 multiplied by 3, wherein the length of a middle array mid is equal to the number of columns of pixels of an original image;

step 1-2, starting a filtering window from the upper left corner, sliding pixels one by one from the top to the bottom from the left to the right, setting a central pixel point of the window as (i, j) when the filtering window moves to a new position, summing each row of the (i-1) th row, the i +1 th row and each column of the i +1 th row in sequence, putting a summation result in a middle array mid, summing mid [ i, j-1], mid [ i, j ] and mid [ i, j +1] to obtain a sigma A (i, j), wherein A (i, j) is a pixel value with a pixel coordinate of (i, j) in an original image, and storing the summation result to a point (i, j) of an array B;

step 1-3, when the window is shifted to the right by one pixel point, the center pixel point of the window is changed into (i, j +1), mid [ i, j-1] is subtracted from the last summation result, and mid [ i, j +2] is added to obtain the summation result of a new window:

ΣA(i，j+1)＝ΣA(i，j)－mid[i，j-1]+mid[i，j+2](2)

and 1-4, when the window moves to the end of the row and jumps to the next row, updating mid, adding A (i +2, j) to each mid [ i +1, j ], subtracting A (i-1, j), and then starting the calculation of a new row.

4. The method for low-light-level video denoising in time-space domain combination according to claim 1, wherein in step 2, motion detection is performed on the adjacent frames of the video sequence after the boxfilter processing, and the corresponding filter coefficients are determined based on the motion detection, and the specific process is as follows:

step 2-1, setting a difference d of corresponding window pixel sums of a front frame and a rear frame, and calculating d ═ W while recursively reducing noise_N-W_N-1|；

The filter coefficient K is set as a piecewise linear function with respect to d:

wherein d is₁、d₂To be a first difference threshold and a second difference threshold, respectively, K₁、K₂Respectively a first filter coefficient threshold and a second filter coefficient threshold, K₁＞K₂。

Step 2-2, the improved time domain recursive filtering formula is as follows:

W_N(i,j)＝X_N(i,j)+K_N(i,j)(W_(N-1)(i,j)-X_N(i,j)) (4)

in the formula, W_N(i,j)For a window centered at point (i, j) in the filtered output image of the current frame, W_(N-1)(i,j)Outputting a window of a corresponding position of the image for the previous frame; x_N(i,j)A window corresponding to the current input image; k_N(i,j)The filter coefficients are windowed with point (i, j) as the center, and K ∈ (0, 1).

5. The method for reducing noise of a low-light level video in a time-space domain combination according to claim 1, wherein a compensation function is added to the gray level similarity factor in step 3 based on the similarity principle of pixel gray levels in a window, and the specific process is as follows:

step 3-1, calculating a spatial proximity factor, wherein the formula is as follows:

wherein ω is_s(p, q) is a spatial proximity factor, σ_sIs the filter parameter, (x, y) is the filter window center pixel coordinate, (p, q) is the other pixel coordinate in the window;

step 3-2, calculating the improved gray level similarity factor

Based on the similarity of the pixel gray levels in the window, a compensation function is added to the gray level similarity factor,

wherein ω is_r(p, q) is a spatial proximity factor, σ_rFor the filter parameters, (x, y) is the filter window center pixel coordinate, (p, q) is the other pixel coordinates in the window, τ (x, y) is the compensation function, and τ (x, y) is set as follows:

1) judging the similarity of the gray values of the pixel points and the central points in the filtering window based on the similarity; if the absolute value of the difference value between the pixel point and the central point pixel is less than sigma_rIf the I (p, q) is similar to the I (x, y), keeping the original value of the I (p, q), otherwise, keeping the I (p, q) as 0;

2) and setting a compensation function according to the number of the similar points in the window, and if the number of the window pixels which are set to be 0 is less than 1/3 of the number of the window pixels, setting tau (x, y) to be 0, otherwise, setting according to a rule 3).

3) Introducing variables min, max and mean which respectively represent the minimum value, the maximum value and the average value of pixels in the filtering window; let a ═ I (p, q) -mean, if a > 0, τ (x, y) ═ max-I (p, q); if a < 0, τ (x, y) ═ min-I (p, q) if a ═ 0, τ (x, y) ═ 0;

step 3-3, calculating a filtered image after improving bilateral filtering:

wherein,the image obtained after filtering; m_x，yA spatial neighborhood pixel set taking (x, y) as a center and r as a radius; i (p, q) is M_x，yA point pixel value with a middle coordinate of (p, q); omega_s(p, q) and ω_r(p, q) are the spatial proximity factor and the grayscale similarity factor, respectively.