CN1761286A

CN1761286A - Method for detecting movement detection by using edge detection, and for removing ripple noise through medium filtering

Info

Publication number: CN1761286A
Application number: CN 200510030971
Authority: CN
Inventors: 郑世宝; 董云朝; 王嘉
Original assignee: Shanghai Jiao Tong University
Current assignee: Shanghai Jiao Tong University
Priority date: 2005-11-03
Filing date: 2005-11-03
Publication date: 2006-04-19

Abstract

A method for removing mosquito noise by using edge detection, motion detection and median filtering in the field of image processing technology, that is, intra-frame processing is performed on an image first, and then inter-frame processing is performed on the three consecutive frames of images that have undergone intra-frame processing. Specifically: use the Laplace operator to perform edge detection on the image to obtain the edge information of each pixel. If a certain pixel is an edge pixel or there are more than two edge pixels among the four pixels around it, keep its value unchanged; Otherwise, use a median filter to filter the pixel; perform motion detection, and judge whether the pixel in the current frame is moving according to the difference between adjacent frames. If the pixel is a moving pixel, keep its value unchanged ; If it is a non-moving pixel, the result of performing median filtering on the corresponding points in the three frames is taken as the filtering result of the pixel. The invention protects the details and clarity of the image to the greatest extent, and improves the visual quality of the output image.

Description

Remove the method for mosquito noise with rim detection, motion detection and medium filtering

Technical field

What the present invention relates to is a kind of method of technical field of image processing, specifically is a kind of method of removing mosquito noise with rim detection, motion detection and medium filtering.

Background technology

TV signal has developed into the digital signal stage, has produced the standard of a series of encoding and decoding, for example MPEG2, MPEG4 etc.These standards are based on all that the dct transform of piecemeal quantizes to compress then, wherein certainly exist the loss of information.The loss meeting of this information produces the phenomenon of some distortions in the decoding and rebuilding process of image, blocking effect for example, the flicker of interframe, the glimpsing of image border, all since the dct transform compression cause.These distortions have been given different titles according to different phenomenons, are referred to as noise usually, for example block noise, mosquito noise etc.Have a lot of methods to be used to handle these noises, and because the unification of coded format so these noises are generally all handled, is referred to as the post-processed algorithm behind decoding and rebuilding.

Find through literature search prior art, the Chinese patent name is called: noise detector, noise detecting method, signal processor and signal processing method, application number is 00122465, this patent has proposed around a kind of large amplitude edge of the high frequency composition detected image signal by extracting picture signal the noise detecting circuit as the little amplitude edge of noise, and noise signal is carried out the signal processing circuit of horizontal direction and/or vertical direction smoothing processing, and remove mosquito noise and ring in this way according to the result of noise detecting circuit.This patent only adopts the image information in the frame to carry out the detection of noise and level and smooth, causes the fuzzy of details in the image easily, especially continuous motion details fuzzy.

Summary of the invention

The objective of the invention is to overcome deficiency of the prior art; a kind of method of removing mosquito noise with rim detection, motion detection and medium filtering is provided; make it adopt the frame inward flange to detect and the adaptive method detection of interframe movement mosquito noise; and it is carried out medium filtering; protect the details composition in the image effectively, thereby improved whole treatment effect.

The present invention is achieved by the following technical solutions, the present invention includes two big aspects, promptly earlier image is carried out handling in the frame, handle carrying out interframe then through the three continuous two field pictures of handling in the frame, described image is carried out handling in the frame, be specially: use the Laplace operator that image is carried out rim detection earlier, obtain the marginal information of each pixel, if a certain pixel is in edge pixel or four pixels around it plural edge pixel to be arranged, it is constant to keep its value; Otherwise, this pixel is carried out Filtering Processing with a median filter.Described to carry out the interframe processing through the three continuous two field pictures of handling in the frame, be specially: at first carry out motion detection, according to the difference between the consecutive frame, judge whether the pixel in the present frame moves, if this pixel is the motion pixel, it is constant then to keep its value; If be non-motor image element, then the corresponding points in three frames are carried out the filtering result of the result of medium filtering as this pixel.

Mosquito noise is the noise on a kind of time domain, and mosquito noise of the present invention is defined as follows: mosquito noise is the fluctuation flicker in the intersection brightness of moving object and background and colourity.From the image of single frames, mosquito noise shows as on the fuzzy or background at object edge place some near the speckle on object edges; From the image of motion, mosquito noise shows as near irregular fluctuation object edge.The generation of mosquito noise mainly is because during interframe encode, and frame has adopted different quantization steps with coded prediction error between the frame, and in a frame sequence, the coding of same object may be different like this, thus the fluctuation that has produced brightness or chromatic value.Generally we call mosquito noise to this fluctuation of edge, and the Processing Algorithm of various mosquito noises also is to carry out at this fluctuation of edge.

Below the inventive method is further described, particular content is as follows:

1, use the Laplace operator to carry out rim detection

Edge of image can be defined as the discontinuity of image local characteristic, for example: the sudden change of brightness, the sudden change of color, the sudden change of texture structure etc.In general the edge in the image is the border that has between two zones of different average gray grades.Thereby the first derivative amplitude of the gray scale at place, image border can be bigger.Most of edge detecting technology are used the gradient operator of certain form.Rim detection of the present invention is that the second differnce with pixel grey scale serves as to detect foundation.

What the present invention adopted is the Laplace operator, the Laplace operator

Be a second-order differential operator, in digital picture, can be similar to difference

Δ ²f _t(m，n)＝f _t(m+1，n)+f _t(m-1，n)+f _t(m，n+1)+f _t(m，n-1)-4f _t(m，n) (1)

F wherein _t(m n) is in the gray scale of the pixel of the capable n row of m, f in the expression t two field picture _t(m+1, n), f _t(m-1, n), f _t(m, n+1) and f _t(m n-1) represents respectively in the same frame and pixel (m, the n) gray scale of four pixels that cross is adjacent, Δ ²f _t(m, n) the above pixel of expression is through the result of Laplace operator computing.

For the better effects if of rim detection, can carry out some corrections to the Laplace operator, as shown in Equation (2).

(2)

Rim detection can be described below: (m n) loads the Laplace operator, obtains its second order gradient delta to the pixel of t frame in the former sequence to adopt formula (2) ²f _t(m, n); If (m, n) than rim detection thresholding height, then (m n) is edge pixel, otherwise is not edge pixel.The thresholding of rim detection is that self adaptation determines that the computational methods of rim detection thresholding as shown in Equation (3).

edg_threshold (m, n) = \frac{1}{{(2 N + 1)}^{2}} Σ_{i = - N}^{N} Σ_{j = - N}^{N} Δ^{2} f_{t} (m + i, n + j) \cdot \cdot \cdot (3)

Wherein (2N+1) is the size of rim detection thresholding calculation window, and (m n) is pixel (m, rim detection thresholding n) to edg_threshold.If Δ ²f _t(m, n)＞edg_threshold (m, n), then (m n) is edge pixel, its marginal information edge (m, n)=1, otherwise edge (m, n)=0.

2, filtering in the frame

Pattern with filtering in the marginal information decision frame.Filtering is divided into two patterns in the frame: medium filtering pattern and retained-mode.At first calculate pixel (m, the n) number of cross window inward flange pixel on every side, as shown in Equation (4).Again by formula (5) (m n) carries out filtering in the frame to pixel.If edg (m, n)=1 or edge_sum (m n)＞2, then keeps its value and does not process; Other situation then uses median filter that it is carried out filtering.

edge_sum(m，n)＝edge(m，n)+edge(m-1，n)+edge(m+1，n)+edge(m，n-1)+edge(m，n+1) (4)

Wherein median represents the several variablees in the bracket are got the computing of intermediate value,

(m, gray scale n) is carried out the result of filtering in the frame to pixel in expression.

3, motion detection

Motion detection is made up of two parts, is specially frame difference and calculates and motion determination.

The pixel grey scale that is used for the three continuous two field pictures of motion detection is used respectively

With

Expression, wherein

The gray scale of the pixel of the capable n row of m in the expression present frame, Represent to be ahead of in time (m, the n) gray scale of the pixel in a frame period, Expression lags behind in time

The gray scale of the pixel in a frame period, they all are the results of filtering in the frame.Pixel (m in the present frame, n) pixel grey scale in the neighborhood around gray scale and its compares with the pixel grey scale of front and back frame correspondence respectively, and with the summation of the absolute value of corresponding points difference, obtain difference prediff (m respectively, n) and postdiff (m, n), shown in formula (6) and (7).If these two differences have one or two all greater than thresholding, then judge (m n) is the motion pixel, movable information motion (m n)=1, otherwise is non-motion pixel, and motion (m, n)=0.Being calculated as follows of frame difference, (m, n), its gray scale is used for the pixel in the present frame

Expression uses following two formula to obtain two differences

prediff (m, n) = Σ_{i = - M}^{M} Σ_{j = - M}^{M} | {\hat{f}}_{t} (m + i, n + j) - {\hat{f}}_{t + 1} (m + i, n + j) | \cdot \cdot \cdot (6)

postdiff (m, n) = Σ_{i = - M}^{M} Σ_{j = - M}^{M} | {\hat{f}}_{t - 1} (m + i, n + j) - {\hat{f}}_{t - 1} (m + i, n + j) | \cdot \cdot \cdot (7)

Wherein the window size that calculates of frame difference is 2M+1, prediff (m, n) and postdiff (m n) is respectively the difference of present frame and former frame, and the present frame and the back difference of a frame.Two differences of formula (6) and (7) compare with thresholding respectively, and thresholding is according to the pixel in the present frame (m, gray scale n)

Determine, as shown in Equation (8).(m, calculating n) as shown in Equation (9) for movable information motion.

Wherein (m, n) expression is to current frame pixel (m, n) the motion detection thresholding of She Dinging for mov_threshold.

Frame difference is then judged pixel greater than the detection threshold of setting Move, promptly its movable information motion (m, n)=1, otherwise motion (m, n)=0.

4, frame filter

Result according to motion detection determines the mode that interframe is handled, and promptly determines the filtering mode to pixel in the present frame.If (m n) is the motion pixel, promptly motion (m, n)=1, the result who handles in this retention frame so; Otherwise do the medium filtering on the time domain, promptly replace the pixel value of current point with the result of three pixel value medium filterings of front and back frame and present frame same point.Promptly

f_{t}^{*} (m, n) = \{\begin{matrix} {\hat{f}}_{t} (m, n) & motion (m, n) = 1 \\ median [{\hat{f}}_{t + 1} (m, n), {\hat{f}}_{t} (m, n), {\hat{f}}_{t - 1} (m, n)] & motion (m, n) = 0 \end{matrix} \cdot \cdot \cdot (10)

F wherein _t ^*(m, n) expression is carried out the result of frame filter to the t frame,

Expression is to pixel (m, n) carry out the result of filtering in the frame, median represents the several variablees in the bracket are got the computing of intermediate value, motion (m, n) in t frame of expression through processing in the frame, the motion state of the capable n row of m pixel, motion (m, n)=1 the expression this pixel be the motion pixel, otherwise till pixel.

The mosquito noise noise-reduction method that the present invention relates to; the protection to the image border details has been considered in the processing section in its frame; the protection to motion parts has been considered in the interframe processing section; when effectively removing mosquito noise; the details and the definition of image have been protected to greatest extent; reduce the level that only adopts in the prior art in the frame and vertical filtering to the loss that the details and the motion composition of image causes, improved the visual quality of output image.

Description of drawings

Fig. 1 principle of the invention figure

Embodiment

Provide following examples in conjunction with content of the present invention:

As shown in the figure, Fig. 1 is the mosquito noise noise reduction algorithm schematic diagram that the present invention relates to.Noise reduction process is as follows:

(1) frame delay unit I and II: frame delay unit I is with video signal delay one frame of input, frame delay unit II is output as input with frame delay unit I, and it is postponed a frame, if promptly Shu Ru vision signal is (t+1) two field picture, the output of frame delay unit I is (t) two field picture, and the output of frame delay unit II is (t-1) two field picture.

(2) edge detection unit I, II, III: adopt the Laplace operator, respectively (t+1) two field picture, t two field picture and (t-1) two field picture of input are carried out rim detection according to formula (2), and by formula (3) calculate each rim detection thresholding, determine the marginal information of each pixel in (t+1) two field picture, t two field picture and (t-1) two field picture.

(3) filter unit I, II, III in the frame: according to the marginal information of each pixel in (t+1) two field picture, t two field picture and (t-1) two field picture, frame interior filter unit I, II, III by formula (4) and (5) carry out Filtering Processing in the frame to each pixel in (t+1) two field picture, t two field picture and (t-1) two field picture, obtain filtering result in the frame

With

(4) motion detection block: adopt formula (6) and (7) to calculate respectively

With

Difference prediff (m, n),

With Difference postdiff (m n), and adopts formula (8) to calculate the motion detection thresholding, and by formula (9) are determined In pixel movable information motion (m, n).(5) frame filter unit: right according to formula (10) Carry out frame filter, obtain final mosquito noise noise reduction result as video output signals.

Concrete implementation content is as follows:

At first according to the result through the Laplace operator of each pixel in formula (2) calculating input image, and calculate the thresholding that each pixel edge detects according to formula (3), in an embodiment, the thresholding calculation window of selected rim detection is 3 * 3 windows, i.e. N=1 in the formula (3).Extract the edge of input picture as mentioned above, determine the marginal information of each pixel.According to formula (5) input picture is carried out filtering in the frame, obtain the result of filtering in the frame.The calculation window of selected frame differences is 3 * 3 windows, i.e. M=1 in formula (6) and (7).Adopt formula (6) and (7) to calculate the difference between the filtering result in the frame of three continuous frame input pictures,, and finish frame filter, obtain output image according to formula (9) and (10) according to the thresholding of formula (8) calculating motion detection.

The present invention adopts above algorithm that 10 frames are handled through the Phase Alternation Line system image sequence of MPEG2 compression.Decoded video sequence is the 4:2:0 form, luminance signal and carrier chrominance signal separate processes, last combined color image.Before carrying out the mosquito noise noise reduction, the signal to noise ratio of MPEG2 decoded picture is 31.181, and behind the noise reduction, signal to noise ratio is 31.523.Near the noise reduction back edge scintillation has obtained effective inhibition, and the details composition in the image is not affected.

Claims

1, a method for removing mosquito noise with edge detection, motion detection and median filter, it is characterized in that, image is carried out intra-frame processing earlier, then carries out inter-frame processing to the continuous three frame images through intra-frame processing, The described intra-frame processing of the image is specifically: first use the Laplace operator to perform edge detection on the image to obtain the edge information of each pixel, if a certain pixel is an edge pixel or there are two pixels in the four pixels around it Above edge pixel, keep its value unchanged; Otherwise, use a median filter to carry out filter processing to this pixel; Described to carry out inter-frame processing to the continuous three-frame image through intra-frame processing, be specifically: at first carry out Motion detection, according to the difference between adjacent frames, judge whether the pixel in the current frame is moving, if the pixel is a moving pixel, keep its value unchanged; if it is a non-moving pixel, the corresponding point in the three frames will be processed The result of value filtering is used as the filtering result for this pixel.

2. The method for removing mosquito noise by edge detection, motion detection and median filter according to claim 1, characterized in that said edge detection is based on the second-order difference of pixel gray scale.

3. The method for removing mosquito noise with edge detection, motion detection and median filtering according to claim 1 or 2, characterized in that the threshold of edge detection is determined adaptively, and the threshold calculation formula of edge detection is

edg edg__threshold threshold ((m m,, n no)) = = \frac{11}{{((22 N N + + 11))}^{22}} {Σ Σ}_{i i = = - - N N}^{N N} {Σ Σ}_{i i = = - - N N}^{N N} {Δ Δ}^{22} {f f}_{t t} ((m m + + i i,, n no + + j j))

Among them, Δ ² f _t (m, n) is the second-order difference of the pixel (m, n) in the mth row and nth column in the tth frame image, (2N+1) is the size of the edge detection threshold calculation window, edg_threshold(m , n) is the edge detection threshold of pixel (m, n), if Δ ² f _t (m, n) > edg_threshold (m, n) then the edge information edg (m, n) of pixel (m, n) = 1 , otherwise edg(m,n)=0.

4. The method for removing mosquito noise with edge detection, motion detection and median filtering as claimed in claim 1, wherein the mode of intra-frame filtering is determined by edge information, as shown in the following formula:

edge_sum(m,n)=edge(m,n)+edge(m-1,n)+edge(m+1,n)+edge(m,n-1)+edge(m,n+1)

Among them, edg(m, n) represents the edge information of the pixel (m, n), edge_sum(m, n) represents the number of edge pixels in the pixel (m, n) and its neighborhood, median represents the selection of several variables in brackets median operation,

Indicates the result of intra-filtering on pixel (m, n).

5. The method for removing mosquito noise with edge detection, motion detection and median filtering according to claim 1, characterized in that said motion detection consists of two parts, specifically inter-frame difference calculation and motion judgment .

6. The method for removing mosquito noise with edge detection, motion detection and median filtering according to claim 5, characterized in that the calculation of the difference between frames is completed by the following two formulas,

prediff prediff ((m m,, n no)) = = {Σ Σ}_{i i = = - - M m}^{M m} {Σ Σ}_{j j = = - - M m}^{M m} | | {\overset{^^}{f f}}_{t t} ((m m + + i i,, n no + + j j)) - - {\overset{^^}{f f}}_{t t + + 11} ((m m + + i i,, n no + + j j))

postdiff postdiff ((m m,, n no)) = = {Σ Σ}_{i i = = - - M m}^{M m} {Σ Σ}_{j j = = - - M m}^{M m} | | {\overset{^^}{f f}}_{t t - - 11} ((m m + + i i,, n no + + j j)) - - {\overset{^^}{f f}}_{t t - - 11} ((m m + + i i,, n no + + j j))

The window size for inter-frame difference calculation is 2M+1, prediff(m, n) and postdiff(m, n) are the difference between the current frame and the previous frame, and the difference between the current frame and the next frame, respectively.

7. The method for removing mosquito noise with edge detection, motion detection and median filtering according to claim 5, wherein the threshold for motion judgment is calculated according to the gray scale of the pixel, as shown in the following formula:

Where mov_threshold(m, n) represents the motion detection threshold set for the current frame pixel (m, n), Indicates the result of intra-filtering on pixel (m, n).

8. If the method for removing mosquito noise by using edge detection, motion detection and median filter as claimed in claim 6 or 7, it is characterized in that, if the difference between frames is greater than the set detection threshold, then it is judged that the pixel

is moving, that is, its motion information motion(m,n)=1, otherwise motion(m,n)=0.

9. The algorithm for mosquito noise reduction using edge detection, motion detection and median filtering according to claim 1, characterized in that the inter-frame processing is determined according to the result of motion detection, and the inter-frame processing is according to the following formula conduct:

{f f}_{t t}^{* *} ((m m,, n no)) = = \{\begin{matrix} {\overset{^^}{f f}}_{t t} ((m m,, n no)) & motion motion ((m m,, n no)) = = 11 \\ median median [[{\overset{^^}{f f}}_{t t + + 11} ((m m,, n no)),, {\overset{^^}{f f}}_{t t} ((m m,, n no)),, {\overset{^^}{f f}}_{t t - - 11} ((m m,, n no))]] & motion motion ((m m,, n no)) = = 00 \end{matrix}

Where f _t ^* (m, n) represents the result of inter-frame filtering on the tth frame,

Indicates the result of intra-frame filtering on pixels (m, n), median indicates the operation of taking the median value of several variables in brackets, motion(m, n) indicates that in the t-th frame that has undergone intra-frame processing, the m-th The motion state of the pixel in the nth column of the row, motion(m, n)=1 indicates that the pixel is a motion pixel, otherwise it is a static pixel.