JP2000102020A - Digital image processor and filtering method for image signal - Google Patents

Abstract

PROBLEM TO BE SOLVED: To eliminate noise and distortion included inside images, the distortion at a block boundary accompanying a block compression encoding system and the distortion around a contour part inside the images for instance, without damaging the high band components of source images. SOLUTION: For respective pixels for forming digital images, a signal level difference from each pixel surrounding the respective pixels is detected by a difference detection part 12. The signal level difference is compared with a prescribed reference value by a threshold judgement part 13. Corresponding to the result of the comparison, a value for which a prescribed coefficient is multiplied with the signal level difference by an arithmetic part 14 is added to the respective pixel values. By selecting the reference value in the comparison and the coefficient in a multiplication part corresponding to the position of the pixel and the image, the distortion included inside the image is eliminated while preserving the high band components of the large change of a signal level.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

[0001] 1. Field of the Invention [0002] The present invention relates to an image processing apparatus and an image signal filtering method. In particular, the present invention relates to a filter for removing distortion included in an image, for example, distortion generated during compression encoding or decoding of an image, and an image processing apparatus using the same.

[0002]

2. Description of the Related Art Digital images can contain distortion and noise by various methods. For example, in the case of an encoding method in which image compression is performed in units of blocks using discrete cosine transform (DCT) or the like, distortion may occur where the boundaries of blocks are discontinuous, or distortion may occur around the outline of the image.

[0003] FIG. 10 is a diagram for explaining image distortion due to the block compression coding method. FIG. 10A shows an example of a signal in a case where the luminance value of the original image changes gradually, and FIG. 10B shows a signal in the case where a contour component is included in a block. This is an example. In these figures, four signals are arranged in a direction perpendicular to the boundaries of the blocks in the image, and the dotted lines represent the boundaries of the blocks.

[0004] As shown in FIG. 10A, a step occurs in a luminance value at a block boundary due to encoding, and distortion occurs in which a block boundary becomes discontinuous. Such a distortion causes the image to look like a mosaic image, resulting in a visually unnatural image. If the original image contains a rapidly changing high-frequency component such as an outline as shown in FIG. 10B, irregular distortion is generated around the outline in the encoded image.

[0005] The distortion accompanying such block coding is an unnatural deterioration in image quality that is visually noticeable, and it is necessary to reduce or eliminate such distortion in order to improve the image quality of a coded image.

[0006] Conventionally, for images containing distortion and noise,
The distortion was removed using a low-pass filter. For a block coded image, there is a technique that uses a low-pass filter to smooth a step only in a block boundary portion or that changes a signal value irregularly using a random number to reduce distortion. .

[0007]

However, when distortion is removed using a low-pass filter, there is a problem that the image quality is degraded. For example, when a process of removing distortion accompanying block encoding is performed, high-frequency components other than the distortion of the original image are also removed, and there is a problem that the image is blurred. Further, in the method using a random number as in the above example, the amount of change in the signal value may be different for each frame in a moving image, and thus there is a problem that flicker occurs.

[0008] Further, when only the block boundary is processed in the removal of the distortion associated with the block encoding, even if the distortion at the block boundary is removed in an image having a gradual change in luminance, the distortion generated inside the block remains. Often it was unsightly because it was not removed. Furthermore, inside a block having a contour component, distortion occurs at the periphery of the contour, and the conventional method cannot suppress such image quality deterioration.

Accordingly, an object of the present invention is to provide an image processing apparatus and method for removing distortion and noise contained in a digital image without damaging the contour components of the original image. It is another object of the present invention to provide an image processing apparatus and a method for removing distortion generated not only in a block boundary portion but also in an entire image in a block-coded image.

[0010]

According to the present invention, there is provided a distortion removing filter for detecting a signal level difference between each pixel forming a digital image and each of at least one or more pixels surrounding the pixel. Means, (b) comparing means for comparing the signal level difference with a reference value, and (c) based on the result of the comparison, multiplying each of the signal level differences by a predetermined coefficient to the signal level of the pixel. And a calculating means for adding.

[0011] Another feature of the present invention is that, when image data in which one image is formed by a plurality of pixel blocks is used as an input image, the comparison means based on the position of each pixel in the block. It is characterized by comprising a selection means for selecting a value.

[0012] Here, the comparing means may be characterized in that the absolute value of the signal level difference is compared with the reference value. Further, an adjusting means for adjusting the reference value in the comparing means may be provided. Further, the calculating means adds (a) the result of the multiplication when the absolute value of the signal level difference is smaller than the reference value, and (b) when the absolute value of the signal level difference is larger than the reference value. The coefficient is set to 0, and (c) when the absolute value of the signal level difference is larger than the reference value, the coefficient is gradually approached to 0 as the signal level difference increases. May be.

[0013] The distortion removal filtering method of the present invention comprises the steps of: (a) detecting a signal level difference between each pixel forming a digital image and each of at least one or more pixels surrounding the pixel; Comparing the level difference with a reference value, and (c) multiplying each of the signal level differences by a predetermined coefficient based on a result of the comparison and adding the result to the signal level of the pixel.

[0014] Another feature of the present invention is that, when image data in which one image is formed by a plurality of pixel blocks is used as an input image, the comparison means based on the position of each pixel in the block. The method further includes the step of selecting a value.

Here, the comparing step (b) may be characterized by comparing the absolute value of the signal level difference with the reference value. Further, the calculation step (c) includes: (a) when the absolute value of the signal level difference is smaller than the reference value,
Adding the result of the multiplication, (b) setting the coefficient to 0 when the absolute value of the signal level difference is larger than the reference value, and (c) when the absolute value of the signal level difference is larger than the reference value. Alternatively, the coefficient may gradually approach 0 as the signal level difference increases.

[0016] The image processing apparatus according to the present invention comprises: (a) decoding means for decoding an encoded digital image; and (b) for each pixel of the decoded image, Detecting means for detecting a signal level difference between each of one or more pixels, (c) comparing means for comparing the signal level difference with a reference value, and (d) based on a result of the comparison, Calculating means for multiplying the signal level difference by a predetermined coefficient and adding the result to the signal level of the pixel.

Another feature of the present invention is that (a) encoding means for encoding a prediction error signal of a digital image;
Decoding means for decoding the encoded signal, (c)
Detecting means for detecting a signal level difference between each pixel of the decoded image and each of at least one or more pixels surrounding the pixel; and (d) comparing the signal level difference with a reference value. Comparing means, (e) based on the result of the comparison, calculating means for multiplying each of the signal level differences by a predetermined coefficient and adding to the signal level of the pixel, (f) a prediction signal from the output signal of the calculating means And (g) difference means for calculating the difference between the digital image and the prediction signal and generating the prediction error signal.

Another feature of the present invention is that (a) decoding means for decoding an encoded digital image signal, and (b) adding the decoded image signal and a prediction signal. Addition means, and (c) detection means for detecting a signal level difference between each pixel of the output image of the addition means and each of at least one or more pixels surrounding the pixel,
(d) comparing means for comparing the signal level difference with a reference value, and (e) calculating means for multiplying each of the signal level differences by a predetermined coefficient and adding the result to the signal level of the pixel based on the result of the comparison And (f) prediction means for generating the prediction signal from the output signal of the calculation means.

[0019]

DESCRIPTION OF THE PREFERRED EMBODIMENTS One embodiment of the present invention will be described below in detail with reference to FIG. FIG. 1 is a configuration diagram of a distortion removal filter according to the embodiment, which includes a memory unit 11, a signal difference detection unit 12, a threshold value determination unit 13, and a calculation unit 14.

The memory unit 11 stores input image data (for example, a luminance signal and a color difference signal). The difference detection unit 12 detects a difference value (for example, a luminance signal) of a signal between one pixel of image data and surrounding pixels. Further, the threshold value judging unit 13 compares the detected absolute value of each difference value with a predetermined reference value, and according to the comparison result, calculates each difference value to the next arithmetic unit 14.
It is determined whether or not to use. The calculation unit 14 multiplies each difference value given according to the result of the threshold value determination unit 13 by a predetermined coefficient corresponding to the distance between the pixels that have obtained the difference, and multiplies the original input pixel value by the respective integration values. Add values.

Hereinafter, the operation of the distortion removal filter shown in FIG. 1 will be described with reference to the schematic diagram of FIG. 2 and the flowchart of FIG. As shown in FIG. 2, the coordinates (m, n) of the digital image
The pixel value located at is represented by _{Xm, n} , and this pixel and its surrounding
When the pixel signals X _{m + i, n + j} located at (m + i, n + j) are input, the difference values Δ _{m + i, n + j} are Δ _{m + i, n + j} = _{Xm + i, n + j-} Xm _{, n} , which are obtained by the difference detection unit 12 in FIG. 1 in step S2 in FIG. Here, i and j represent deviations in the horizontal and vertical directions from (m, n), respectively. In the example of FIG.
It is an integer value in the range of i ≦ 1, −1 ≦ j ≦ 1.

[0022] Next the difference value delta _{m + i} in step _S3, the absolute value of _{n + j} | compares the magnitude of the predetermined threshold value Th at the threshold determination unit _{13 | Δ m + i, n} + j. Here, if | Δm _{+ i, n + j} | is smaller than the threshold Th, the difference is considered to be due to distortion, and the process proceeds to step S4. (In the example of FIG. 2, Δ _{m−1, n−1} , Δ _{m−1, n} ,
It is assumed that five difference values of Δm _{+ 1, n} , Δm _{, n + 1} , Δm _{+ 1, n + 1} satisfy the above conditions. ) Also, | Δm _{+ i, n + j}
Is larger than the threshold value Th, it is considered that the difference between the pixel values is due to the contour component existing in the original image, and step S
Proceed to 6.

In step S4, the calculation unit 14 calculates a value A _{i, j} × Δ _{m + i} obtained by multiplying a predetermined coefficient A _{i , j} based on the value of _{i , j} by the difference value Δm _{+ i, n + j. , N + j} are added to the pixel values X _{m, n} located at (m, n) and output.

[0024] The distortion due to compression encoding or the like is noise generated at a position where it should not exist, and the absolute value of the distortion difference value is often smaller than the difference value of the contour component existing in the original image. . For example, in the case of an encoding method for quantizing DCT coefficients, the generated distortion is often about twice or less the quantization step T for the DC component. Therefore, a pixel whose difference absolute value | Δm _{+ i, n + j} | is smaller than a threshold value Th (for example, about 2T in the case of the above-described encoding method) is selected from surrounding pixels.
If these difference values are weighted and added to the input pixel value, the difference between those pixels becomes small, so that distortion can be sufficiently removed.

Rather, if a pixel having a large absolute difference value | Δm _{+ i, n + j} | is also processed, a contour existing in the original image is consequently obtained as in the case of using the conventional low-pass filter. There is a risk that the components may be made gentle and the image quality may be degraded. Therefore, by performing the threshold determination of the difference value between the pixels and selecting the pixel signal to be used for the processing as in the present embodiment, it is possible to prevent the inconvenience that the image quality is deteriorated by the filtering process.

FIG. 4 shows an example in which the above-described processing is performed on each pixel forming a digital image. Here, FIG.
The black circles in indicate the luminance values of the signals, respectively, and represent four scan lines. As shown in FIG.
In the case of a signal having a difference of h or more, the output signal holds this difference component. On the other hand, as shown in FIG. 4B, in a signal including a difference equal to or smaller than the threshold Th, an output from which a step is removed is obtained, and distortion is effectively removed.

[0027] Various modifications are possible in practicing the present invention. For example, in the embodiment of FIG. 2, the window used for the filtering process is a square of 3 pixels in the horizontal direction and 3 pixels in the vertical direction. However, not limited to this size and shape, various windows can be applied. Or a cross shape. The values of the coefficients A _{i, j} in the arithmetic unit are values based on only the values of i and j in the embodiment of FIG. 2, but may be adaptively selected according to the value of the pixel. For example, as in the embodiment shown in FIG. 5, the pattern of pixel values in the window of the filter process is classified by a method such as vector quantization, and the pattern p of the classification result is optimized so as to reduce distortion. The selected coefficients A _{i, j, p} may be selected. Here, it is assumed that p is a variable representing the type of the pattern.

Although the reference value in the comparison means is constant in the embodiment of FIG. 2, the control unit 61 controls the reading from the memory unit 11 and the control of the threshold value judgment unit 13 as in the embodiment shown in FIG. , May be changed according to the position of the pixel in the image. For example, in an image using a block coding method, distortion at a block boundary is easily noticeable. Therefore, when processing the pixels at the block boundary so that the distortion is sufficiently removed, the control unit 61 may issue an instruction to set the threshold to be larger than the threshold when processing the pixels inside the block. .

[0029] A manual adjuster may be provided so that the viewer of the image can adjust the reference value in the comparing means as desired.

In FIG. 3, when it is determined in step S3 that the difference absolute value | Δm _{+ i, n + j} | is _{equal to} or larger than the threshold value, the process proceeds to step S6. Is also good. (A) If the difference absolute value | Δm _{+ i, n + j} | is _{equal to} or larger than the threshold value in the determination in step S3, the process proceeds to step S4. However, at this time, the coefficient A _{i, j} in step S4 is selected as 0. (B) If the absolute value of the difference | Δm _{+ i, n + j} | is _{equal to} or larger than the threshold value in the determination in step S3, the process proceeds to step S4. However, at this time, the values of Δm _{+ i and n + j} given in step S4 are set to 0. Also,
If it is necessary to avoid discontinuous characteristics when the difference absolute value | Δm _{+ i, n + j} | takes a value near the threshold, the following two methods may be used. (C) If the absolute value of the difference | Δm _{+ i, n + j} | is _{equal to} or greater than the threshold value in the determination in step S3, the process proceeds to step S4. However, at this time, the coefficients _{Ai, j} in step S4 are changed so as to approach 0 as | Δm _{+ i, n + j} | (D) If the absolute value of the difference | Δm _{+ i, n + j} | is _{equal to} or larger than the threshold value in the determination in step S3, the process proceeds to step S4. However, in this case given to step S4 Δ _{m + i,} the value of _{n + j | Δ m + i} , n + j | varied to approach zero in accordance with increases.

An application example of the distortion removal filter of the embodiment described above will be described. FIG. 7 shows an example of the configuration of a decoding device for encoded image data. As shown in FIG. 7, in the decoding apparatus, the distortion removal filter according to the present embodiment is connected to the subsequent stage of the decoder 71. When the image signal encoded by the encoding device is input to the present decoding device, the image signal is decoded by the
In step 2, distortion generated in the image is removed.

FIG. 8 shows an example of the configuration of a coding apparatus corresponding to a coding method for a moving picture predicted and coded in the time direction. As shown in FIG. 8, when the distortion removal filter according to the present embodiment is used after being connected to the subsequent stage of the local decoder 81 of the encoder, the prediction error data encoded by the encoder 82 is used by the local decoder 81. And the distortion removal filter 8
After the distortion is removed in step 3, the signal is supplied to the predictor 85. The difference between the input image and the predicted signal from the image from which the distortion has been removed by the differentiator 86 is supplied to the encoder 82. When a prediction signal is generated using an image from which distortion has been removed in this way, a prediction error from newly input image data is smaller than a prediction signal from image data having distortion, and the same coding rate is obtained. , The image quality is improved.

FIG. 9 shows a configuration example of a decoding device corresponding to the encoding device shown in FIG. The image data encoded by the encoder 82 of FIG. 8 is decoded into a prediction error signal by the decoder 91 of FIG. This signal and the prediction signal from the predictor 92 are added by the adder 93 and input to the distortion removal filter 94 of the present embodiment. Here, the image data from which the distortion has been removed is the output image, and this signal is also supplied to the predictor 92.

As shown in FIGS. 8 and 9, by using the distortion removal filter of this embodiment in both the encoding device and the decoding device, the image quality can be further improved as compared with the application example shown in FIG. Can be obtained.

[0035]

As described above, the distortion removal filter of the present invention first detects the difference between the signal level of one pixel and each of the surrounding pixels, and compares this difference with the reference value. Next, according to the result of the comparison, the difference value is multiplied by a predetermined coefficient in accordance with the positional relationship between one pixel and surrounding pixels, and is added to the input pixel value. If the difference between one pixel and the surrounding pixels is larger than the reference value, the values of the surrounding pixels are excluded from the processing, so that the contour portion including the high-frequency component of the original image is not damaged. On the other hand, only the pixels around one pixel whose difference from one pixel is smaller than the reference value are selected, and the difference value between these pixels is smoothed so that the difference value is further reduced. Distortion can be effectively removed.

[Brief description of the drawings]

FIG. 1 is a block diagram showing one embodiment of a distortion removal filter of the present invention.

FIG. 2 is a diagram for explaining an operation of the distortion removal filter of the embodiment.

FIG. 3 is a flowchart illustrating an operation of the distortion removal filter according to the embodiment.

FIG. 4 is a diagram for explaining an operation example of the distortion removal filter of the embodiment.

FIG. 5 is a block diagram showing one embodiment of a distortion removal filter of the present invention.

FIG. 6 is a block diagram showing an embodiment of a distortion removal filter of the present invention.

FIG. 7 is a block diagram illustrating an application example of the distortion removal filter of the embodiment.

FIG. 8 is a block diagram illustrating an application example of the distortion removal filter according to the present embodiment.

FIG. 9 is a block diagram showing an application example of the distortion removal filter of the embodiment.

FIG. 10 is a diagram for explaining distortion caused by the block compression encoding method.

[Explanation of symbols]

 11 memory unit 12, 21 difference detection unit 13, 22 threshold value judgment unit 14 operation unit 23 multiplication unit 24 addition unit 51 pattern classification unit 52 coefficient selection unit 61 control unit 71 decoder 72 distortion removal filter 81 local decoder 82 code Transformer 83 Distortion removal filter 84 Adder 85 Predictor 86 Difference unit 91 Decoder 92 Predictor 93 Adder 94 Distortion removal filter

 ────────────────────────────────────────────────── ─── Continuing on the front page (72) Inventor Hiroaki Watanabe 7-22-1, Roppongi, Minato-ku, Tokyo Part 3 F-term in the Institute of Industrial Science, University of Tokyo 5B057 CA16 CB16 CG02 CG03 CH09 5C059 KK03 MA23 MC15 MD04 PP16 PP25 SS26 TA48 TB08 TC02 TD05 TD12 UA02 UA05 UA11 UA34

Claims (12)

[Claims] 1. The method according to claim 1, wherein each pixel forming a digital image is:
(1-a) detecting means for detecting a signal level difference between each of at least one or more pixels around the pixel, (1-b)
Comparing means for comparing the signal level difference with a reference value; (1
-c) a distortion removing filter, comprising: calculating means for multiplying each signal level difference by a predetermined coefficient based on the result of the comparison and adding the result to the signal level of the pixel. 2. The image processing apparatus according to claim 1, further comprising: selecting means for selecting a reference value in said comparing means based on a position of each pixel in the block when an image is an image formed by a plurality of pixel blocks. 2. The method according to claim 1, wherein
3. The distortion removal filter according to 1. 3. The apparatus according to claim 1, wherein the comparing means compares the absolute value of the signal level difference with the reference value.
3. The distortion removal filter according to 1. 4. The distortion removing filter according to claim 1, further comprising adjusting means for adjusting a reference value in said comparing means. 5. The arithmetic means includes: (5-a) adding the result of the multiplication when the absolute value of the signal level difference is smaller than the reference value; and (5-b) calculating the absolute value of the signal level difference. When the value is larger than the reference value, the coefficient is set to 0, (5-
2. The distortion according to claim 1, wherein, when the absolute value of the signal level difference is larger than the reference value, the coefficient gradually approaches 0 as the signal level difference increases. Removal filter. 6. For each pixel forming a digital image,
(6-a) detecting a signal level difference between each of at least one or more pixels surrounding the pixel, (6-b) comparing the signal level difference with a reference value, and (6-c) comparing the signal level difference with a reference value. A method of multiplying each of the signal level differences by a predetermined coefficient based on a result and adding the result to the signal level of the pixel. 7. The method according to claim 1, further comprising the step of selecting a reference value in said comparing means based on a position of each pixel in the block, when image data in which one image is formed by a plurality of pixel blocks is used as an input image. The filtering method according to claim 5, wherein: 8. The method according to claim 5, wherein the comparing step (6-b) compares the absolute value of the signal level difference with the reference value. 9. The computing step (6-c) comprises: (9-a) adding the multiplication result when the absolute value of the signal level difference is smaller than the reference value; When the absolute value of the level difference is greater than the reference value, the coefficient is set to 0. (9-c) When the absolute value of the signal level difference is greater than the reference value, the signal level difference is increased. 6. The distortion removal filtering method according to claim 5, wherein the coefficient is gradually approached to 0 according to the following. (10-a) decoding means for decoding an encoded digital image; and (10-b) at least one pixel surrounding the pixel for each pixel of the decoded image. Detecting means for detecting a signal level difference between each of the pixels described above, (10-c) comparing means for comparing the signal level difference with a reference value, and (10-d) based on a result of the comparison, An image processing apparatus comprising: an arithmetic unit that multiplies each signal level difference by a predetermined coefficient and adds the result to the signal level of the pixel. (11-a) encoding means for encoding a prediction error signal of a digital image, (11-b) decoding means for decoding the encoded signal, (11-c) Detecting means for detecting a signal level difference between each pixel of the decoded image and each of at least one or more pixels surrounding the pixel; (11-d) determining the signal level difference and a reference value; Comparing means for comparing, (11-e) arithmetic means for multiplying each of the signal level differences by a predetermined coefficient based on the result of the comparison and adding the result to the signal level of the pixel, and (11-f) the arithmetic means Prediction means for generating a prediction signal from the output signal of the image, (11-g) an image characterized by comprising a difference means for taking the difference between the digital image and the prediction signal, and generating the prediction error signal Processing equipment. (12-a) decoding means for decoding the encoded digital image signal, (12-b) addition means for adding the decoded image signal and the prediction signal, 12-c) detecting means for detecting a signal level difference between each pixel of the output image of the adding means and each of at least one or more pixels around the pixel, and (12-d) detecting the signal level difference. Comparison means for comparing with a reference value; (12-e) arithmetic means for multiplying each of the signal level differences by a predetermined coefficient based on the result of the comparison and adding the result to the signal level of the pixel; (12-f) An image processing apparatus comprising: a prediction unit that generates the prediction signal from an output signal of the calculation unit.