JP2008153812A - Noise removal device - Google Patents

Abstract

There is provided a noise removal device that removes noise from a compressed / decompressed reproduced image without adversely affecting the reproduced image.
The noise removing device performs orthogonal transformation on image data for each transformation unit block of orthogonal transformation, extracts an orthogonal transformation result for a central region of a pixel to be filtered from the orthogonal transformation result, and performs filtering processing. An orthogonal transformation result corresponding to all the pixels in the peripheral region for the pixel is extracted. From all the orthogonal transformation results in the surrounding area, determine the attribute of the surrounding area, output the determination result, the maximum value and the distance, and determine the attribute of the central area using the orthogonal transformation result of the central area. The determination result and the maximum value are output. Then, based on the attribute determination result of the central region and the attribute determination result of the peripheral region, the filter of the pixel to be filtered is selected, and the selected filter processing is performed on the pixel to be filtered to remove noise. Generate a pixel.
[Selection] Figure 1

Description

  The present invention relates to a noise removing apparatus, which removes noise generated in a compressed / decompressed reproduced image and improves image quality.

  When an image is recorded or transmitted as a digital signal, the amount of information is large, so compression encoding is generally performed, and a DCT (Discrete Cosine Transform) encoding method is known as the most popular compression encoding method. . In this DCT encoding method, an image is divided into a plurality of blocks, frequency-converted by DCT for each block, and the resulting DC component coefficient data and a plurality of AC component coefficient data are used as appearance probabilities. Accordingly, it is converted into an entropy code having a different bit length, for example, a Huffman code, and the amount of visually redundant frequency components is greatly reduced to reduce the amount of information.

  When the compression rate for recording or transmission is in an appropriate range with respect to the information amount of the image, even if the information amount is reduced by compression, the image is not deteriorated. However, when the compression rate for recording or transmission is low or a complex image, characteristic degradation called mosquito noise occurs near the contour or edge where the density changes greatly.

  As a method for removing the noise as described above, an edge-preserving smoothing filter is effective, and an ε filter represented by the following equation (1) is known (for example, see Non-Patent Document 1).

Here, N is an integer that defines the number of pixels used for smoothing, and c (p, q) is a scale factor. The center pixel value is x (i, j), and the adjacent pixel value x (i + p, j + q) is used for smoothing to remove noise, but the absolute value of the difference between the center pixel value and the adjacent pixel value is the threshold ε Is larger than the threshold value ε, the adjacent pixel value is not used for smoothing. If the absolute value of the difference between the central pixel value and the adjacent pixel value is smaller than the threshold ε, smoothing is performed.
Therefore, since this filter exceeds the threshold value ε in a large shade change portion corresponding to a contour or an edge, the filter is not applied and is stored, and in the shade change portion below the threshold value ε due to noise, the filter is smoothed by the noise. Is removed.

  However, while the ε filter is effective in removing noise, if it is applied to all blocks in the image with a certain threshold, the noise removal effect is high if the threshold is large, but the original texture texture is blurred. Cause image quality to deteriorate. In addition, when the threshold value is small, blurring of a fine texture portion can be suppressed, but the noise removal effect is reduced.

  Therefore, as disclosed in Patent Document 1, the reproduction image data obtained by decoding the image data encoded by the block unit encoding is generated around the edge without adversely affecting the edge portion of the reproduction image. A method for accurately reducing easy mosquito noise has been proposed.

FIG. 14 is a block diagram showing the functional configuration of this method. In FIG. 14, MPEG bit stream data input from the input terminal 110 is decoded by the MPEG decoder 120 and decoded by the edge detection unit 130. The presence / absence of an edge and the intensity are detected from the pixel difference value of two pixels (edge detection target pixel group) and its peripheral pixels from the data.
The noise reduction unit 140 determines whether or not the filtering target pixel is a pixel constituting an edge based on the edge presence / absence detection information supplied from the edge detection unit 130, and when the pixel is determined to be a pixel constituting the edge Does not perform the filtering process on the filtering target pixel.

  On the other hand, if the noise reduction unit 140 determines that the pixel does not constitute an edge, the noise reduction unit 140 determines the distance between the pixel to be filtered and the nearest edge closest to the pixel to be filtered, and the intensity of the nearest edge, Obtained based on edge presence / absence detection information and edge strength information, and according to the obtained distance and strength, select an edge-preserving smoothing filter characteristic to be used for filter processing, and perform filter processing using the selected filter, Output from the output terminal 150.

With this configuration, mosquito noise that is likely to occur around the edge of reproduced image data obtained by decoding image data encoded by block-unit coding does not adversely affect the reproduced image (especially in the edge portion). (Without adverse effects) can be accurately reduced.
Science theory (A), vol. J65-A, pp. 297-304, 1982, "ε-separated nonlinear digital filter and its application" JP 2005-117449 A

  However, in the above-mentioned Patent Document 1, since edge detection is detected from the pixel difference values of two pixels and surrounding pixels, a relatively large edge portion and flat portion can be distinguished, but the edge portion and detail portion are accurately detected. Difficult to distinguish well. For example, the description will be given with reference to the example shown in FIG. Assuming that the actual pixel value is represented by a solid line and the absolute value of the difference between adjacent pixels is represented by a broken line, the difference between the absolute value of the adjacent difference at the edge portion and the absolute value of the adjacent difference at the detail portion is small. It is difficult to distinguish the edge portion from the detail portion.

In Patent Document 1, since the strength of the noise removal filter is determined only by the distance to the adjacent edge and the size of the edge, for example, in the case of an image as shown in FIG. Although the mosquito noise in the vicinity (area B indicated by the broken line in the figure) can be removed, the mosquito noise that appears due to the influence of the edge (the pixel indicated by the broken line in C in the figure) existing at a distance of up to 7 pixels (in the figure) It is considered that it is difficult to remove the mosquito noise generated in the area A indicated by the broken line.
On the other hand, if noise removal priority is set up to about 7 pixels away, there is a risk that the detail portion will be crushed due to the edge of another block.

  The present invention has been made in consideration of the above situation, and provides a noise removing device that removes noise from a compressed / decompressed reproduced image without adversely affecting the reproduced image. For the purpose.

  In order to solve the above-described problems, the noise removal device of the present invention has the following configuration.

  The present noise removal apparatus is a noise removal apparatus that performs filtering on input image data to remove noise, and an orthogonal transform processing unit that orthogonally transforms the image data for each transform unit block of orthogonal transform; and the orthogonal transform A center region extraction unit that extracts an orthogonal transformation result for the center region of the filtering target pixel from the orthogonal transformation result generated by the processing unit, and the filtering target pixel from the orthogonal transformation result generated by the orthogonal transformation processing unit A peripheral region extraction unit that extracts orthogonal transformation results corresponding to all pixels in the peripheral region with respect to, and attributes of the peripheral region are determined from all orthogonal transformation results in the peripheral region obtained by the peripheral region extraction unit Then, the peripheral region attribute determination unit that outputs the determination result, the maximum value and the distance, and the orthogonal transformation result of the central region, A center region attribute determination unit that outputs a determination result and a maximum value, and filter selection that selects a filter of a pixel to be filtered based on output information from the center region attribute determination unit and the peripheral region attribute determination unit And a filter processing unit that performs a filter process selected by the filter selection unit on the filter processing target pixel to generate a pixel from which noise has been removed.

  The orthogonal transform is a 4 × 4 Hadamard transform, or a discrete cosine transform (DCT), a Walsh-Hadamard transform (WHT), a discrete Fourier transform (DFT), or a discrete sine. Transformation (DST: Discrete Sine Transform), Haar transformation (HT: Haar Transform), slant transformation (SLT: SLant Transform), Karhunen-Loeve transformation (KLT), etc. can be applied.

  The attribute determination of the surrounding area is determined to include an edge if the maximum absolute value of the frequency components of all orthogonal transformation results in the surrounding area is equal to or greater than an edge threshold value, otherwise the edge is included. Is determined not to be included.

The attribute determination of the central area is performed as follows.
(1) When the absolute values of all frequency components, which are the results of orthogonal transformation of the center region, are smaller than the flat threshold value, it is determined that the region is a flat region. At this time, when it is determined that an edge is included in the peripheral area, the flat threshold value is changed according to the maximum value and the distance obtained by the peripheral area attribute determination unit.
(2) When the maximum value of the absolute value of the frequency component of the orthogonal transformation result with respect to the center region is equal to or greater than the edge threshold value, it is determined that the center region is an edge region.
(3) When any one of the absolute values of the frequency components as a result of the orthogonal transformation of the central region is larger than the flat threshold value and the maximum absolute value of the frequency components of the orthogonal transformation result for the central region is equal to or smaller than the edge threshold value. In addition, it is determined that the center area is the detail area.

  The attribute determination of the center region may be performed using an average value of frequency components of orthogonal transform results of all center region candidates including the pixel to be filtered, or the frequency of the orthogonal transform result with respect to the center region. You may determine using the ratio of components.

The filter selection unit selects a filter as follows.
(1) When there is an edge in the central area or the peripheral area, an edge preserving filter is selected.
Further, the next filter is selected according to the attributes of the central region and the peripheral region.
(1-1) When the center region is an edge region and there are no edges in the peripheral region, an ε filter that determines the threshold ε with the maximum value obtained by the center region attribute determination unit is selected.
(1-2) When the central region is a flat region and the peripheral region includes an edge, an ε filter that determines the threshold ε with the maximum value obtained by the peripheral region attribute determination unit is selected.
(1-3) When the center region is a detail region and the peripheral region includes an edge, an ε filter that determines the threshold ε based on the maximum value and the distance obtained by the peripheral region attribute determination unit is selected.
(1-4) When the central region is an edge region and the peripheral region includes an edge, the largest maximum value among the maximum value obtained by the central region attribute determination unit and the maximum value obtained by the peripheral region attribute determination unit To select the ε filter that determines the threshold ε.

(2) A simple blur filter is selected for a flat region.
(3) When the center region is a detail region and the peripheral region does not include an edge, the filter is not applied.

  In addition, a program for causing the computer to function as each unit of the noise removal apparatus having the above-described configuration is created, or the program is recorded on a computer-readable recording medium, and the program is executed by the computer. The above problem can also be solved by this.

  According to the present invention, the mosquito noise that stands out around the edge of the compressed / decompressed reproduced image and the imaging noise of the flat portion can be removed without adversely affecting the reproduced image. Improvements can be made.

DESCRIPTION OF EXEMPLARY EMBODIMENTS Hereinafter, a preferred embodiment according to a noise removing device of the invention will be described in detail with reference to the drawings.
FIG. 1 is a block diagram illustrating a functional configuration of an embodiment according to a noise removal apparatus of the present invention. In the figure, the noise removal apparatus includes an orthogonal transformation processing unit 12, a center region extraction unit 13, and a peripheral region extraction unit 14. , A peripheral region attribute determination unit 15, a central region attribute determination unit 16, a filter selection unit 17, and a filter processing unit 18.

Image data to be subjected to noise removal is input from the input terminal 11 and supplied to the orthogonal transform processing unit 12 and the filter processing unit 18.
The input image data is a signal obtained by digitizing YPbPr (YCbCr) or RGB analog signals using an AD converter, a signal obtained by decoding a digital broadcast, a signal from a digital video output from a DVD player, etc. Assuming that m lines are input with n pixels per line, the pixel array X is input as a (m, n) matrix.

  The orthogonal transform processing unit 12 performs orthogonal transform on the image data supplied from the input terminal 11. In the present embodiment, the 4 × 4 Hadamard transform of Expression (2) will be described as the orthogonal transform method.

The matrix X _{0 on} the right side of the above equation (2) is a (4, 4) partial matrix of the pixel array X of the input image data, and the matrix F on the left side indicates the result of 4 × 4 Hadamard transform. F00 is a direct current component, F01, F02 and F03 are horizontal frequency components, F10, F20 and F30 are vertical frequency components, and the other F11 to F33 are horizontal and vertical frequency components (see FIG. 2).

As shown in FIG. 3, the orthogonal transform processing unit 12 acquires the partial matrix X ₀ of (4, 4) from the pixel array X while shifting it in units of one pixel, and formula (2) for all the partial matrices X ₀ . ) To output the conversion result.
This conversion result is, for example, 15 elements per pixel, which is a shift unit (F01, F02, F03, F10, F11, F12, F13, F20, F21, F22, F23, F30, F31, F32, F33). And Hadamard transform results for the pixel blocks of the transform unit obtained by shifting are sequentially output.

In FIG. 3, the shift direction is shifted by one pixel in the horizontal direction (A⇒B⇒C⇒ ...). When one line is completed, the shift is made by one pixel in the vertical direction, and the second line is similarly horizontal. Shift (D⇒E⇒F⇒…). This is repeated until there are n lines.
As an example of how to retrieve from the image data input partial sequence X ₀ of 4 × 4 as described above is performed by the circuit shown in FIG.

In this embodiment, the unit to be shifted is one pixel, but is not limited to one pixel and may be 2 to 4. In general, it is possible to use a unit for shifting within a size for performing orthogonal transform (size in the case of 4 × 4 Hadamard transform is 4). In this case, for example, when the partial matrix cannot be extracted as shown in FIG. 5, the pixel value at the end is repeatedly used.
As described above, the processing amount can be reduced by increasing the unit to be shifted, but the accuracy deteriorates. Therefore, the unit to be shifted is determined in advance in consideration of the processing amount and accuracy.

  In the noise removal apparatus of the present invention, other orthogonal transform methods such as discrete cosine transform (DCT), Walsh-Hadamard transform (WHT), discrete Fourier transform (DFT) are used. Fourier Transform), Discrete Sine Transform (DST), Haar Transform (HT), Slant Transform (SLT), Karhunen-Loeve Transform (KLT), etc. The same can be applied.

  The center region extraction unit 13 extracts the Hadamard transform result for the center region of the pixel to be filtered from the Hadamard transform result transformed by the orthogonal transform processing unit 12 and supplies the extracted Hadamard transform result to the center region attribute determination unit 16. . In this central region, it is determined in advance by user adjustment or the like whether the image evaluation is improved when the filtering target pixel is located in any of the 4 × 4 elements.

  For example, as illustrated in FIG. 6A, there are 16 Hadamard transform results for the pixel to be filtered, and at which element position of the 4 × 4 elements the pixel to be filtered is located. If there is, the position is determined by user adjustment or the like to determine whether the video evaluation is improved. In this example, the area when the pixel to be filtered is in the position shown in (B) is set as the central area.

The peripheral region extraction unit 14 sets a peripheral region for the pixel to be filtered, extracts the Hadamard transform result for all the pixels in the peripheral region from the Hadamard transform result converted by the orthogonal transform processing unit 12, and sets the peripheral region attribute It supplies to the determination part 15. Noise is caused by the influence of edges in the unit block, but the block boundary is unknown, so the unit block size of the compression format of the video source so that the target pixel can be located anywhere in the unit block. The peripheral area is set according to the size of.
For example, in the case of MPEG2, since the unit block size is 8 × 8, the range of the peripheral area is determined in consideration of the influence of edges existing at a distance of 7 pixels at the maximum.

  In this embodiment, as illustrated in FIG. 7, a range that is 7 pixels away from the pixel to be filtered is set as a peripheral region, and all Hadamard transform results are obtained by shifting the pixel block of the transform unit as illustrated in FIG. 8. Extract.

When this noise removal device is installed inside a player such as a BD, HD DVD, or DVD or inside a television equipped with a digital tuner, the compression format information can be obtained from the internal decoder, so the block size information is Get and set.
Further, in the case of a video signal obtained by digitizing an analog video signal with an AD converter, the compression format is not known, so that it is set by user adjustment.

Since the mosquito noise is generated in the peripheral portion of the edge, the peripheral region attribute determining unit 15 includes an edge in the peripheral region from all the Hadamard transform results in the peripheral region with respect to the filtering target pixel obtained by the peripheral region extracting unit 14. And the presence / absence of an edge in the peripheral region, the maximum value (size of the edge), and the distance are supplied to the central region attribute determination unit 16 and the filter selection unit 17 (the maximum value and the distance will be described later). .
This determination is performed as follows.

(1) Edges are included in the peripheral area.
First, frequency components other than F00 of one Hadamard transform result in the peripheral region obtained by the peripheral region extraction unit 14 are set in the horizontal direction (F01, F02, F03), the vertical direction (F10, F20, F30), and the horizontal + vertical direction. (F11, F12, F13, F21, F22, F23, F31, F32, F33) are divided into three types, and the maximum value after the absolute value of the frequency component in each classification and the frequency component having the maximum value Find the position.
This is performed for all Hadamard transform results in the surrounding area, the maximum value in the surrounding area and the position of the frequency component that had the maximum value, the element position of the pixel to be filtered and the frequency that had the maximum value Find the distance to the element position of the component.

This distance is √ {(the horizontal element position of the filtering target pixel−the horizontal position of the element having the maximum value) ² + (the vertical element position of the filtering target pixel−the vertical element position of the frequency component having the maximum value) ² } or || the horizontal element position of the pixel to be filtered−the horizontal position of the element having the maximum value | or the vertical element position of the pixel to be filtered−the vertical element of the frequency component having the maximum value The larger of position | is adopted. At the time of realization, select either one according to the available resources.

Next, when the element having the maximum value obtained as described above is one of F02, F20, and F22, the surrounding area attribute determination unit 15 determines a threshold value for determining that the maximum value is an edge. If it is above, it determines with the edge which may generate | occur | produce mosquito noise being included.
This threshold value is determined by user adjustment by considering various video sources, and is about several tens to several hundreds when the video data is quantized with 8 bits.
In this determination, in the case of the 4 × 4 Hadamard transform, when the horizontal frequency component is F02, the vertical frequency component is F20, and the horizontal + vertical frequency component is F22, the frequency component that well represents the edge component. Take advantage of something.

(2) No edge is included in the peripheral area.
When the element having the maximum value obtained in the above (1) is an element other than F02, F20, and F22, or when the maximum value is smaller than a threshold value for determining an edge, an edge is not included in this peripheral area. Is determined not to be included.

FIG. 9 is a flowchart illustrating a processing procedure of the peripheral area attribute determination unit 15 that determines whether or not there is an edge in the peripheral area for one filter processing target image.
If the processing has been completed for all orthogonal transformation (Hadamard transformation) results in the peripheral region for the filter processing target pixel (step S1 / YES), control is transferred to step S6.

  On the other hand, if the processing has not been completed for all orthogonal transformation (Hadamard transformation) results in the peripheral region for the pixel to be filtered (step S1 / NO), all of the untransformed orthogonal transformation (Hadamard transformation) results The frequency components other than F00 are converted into absolute values (step S2), and the frequency components other than F00 are divided into three types: horizontal direction (F01, F02, F03), vertical direction (F10, F20, F30), and horizontal + vertical direction (all remaining). The maximum value in each classification is obtained (step S3).

The maximum value obtained last time is compared with the maximum value obtained this time for each classification, and when the maximum value obtained this time is larger (step S4 / YES), the frequency with the current maximum value and the maximum value for each classification is determined. The element position of the component, the element position of the pixel to be filtered and the distance to the element position having the maximum value are obtained (step S5), and the control is returned to step S1.
When the maximum value obtained this time is smaller (step S4 / NO), the control is returned to step S1.

  When all the orthogonal transformation (Hadamard transformation) results in the elephant peripheral area for the filter processing pixel are finished (step S1 / YES), the elements having the maximum values obtained so far are F02, F20, and F22. If it is any (step S6 / YES), the maximum value among the frequency components F02, F20, and F22 is obtained, and the position and distance of the element having the maximum value are obtained (step S7). If the value is equal to or greater than a threshold value for determining an edge (step S8 / “maximum value> threshold value”), it is determined that an edge that may generate mosquito noise is included (step S9).

On the other hand, when the element having the maximum value obtained so far is an element other than F02, F20, and F22 (step S6 / NO), or because the maximum value obtained in step S7 is determined to be an edge. (Step S8 / “maximum value <threshold”), it is determined that no edge is included (step S10).
Finally, the presence / absence of an edge, the maximum value (edge size), and the distance in the peripheral region with respect to the filter processing target pixel are output (step S11).

The center region attribute determination unit 16 determines whether the filter target pixel is flat, detail, or edge using the Hadamard transform result of the central region with respect to the filter target pixel, and the center region is flat. Which area of the detail or edge is supplied, and the maximum value after the absolute value of the frequency component when it is determined as an edge is supplied to the filter selection unit 17.
The attribute determination of the center area is performed as follows.

(1) The central region is flat.
When the absolute values of all 15 frequency components excluding F00 that is a DC component among the 16 frequency components F00 to F33, which are the Hadamard transform results of the central region, are smaller than a threshold for determining that they are flat Determined to be flat. The threshold value at this time is determined by considering various video sources, and is about several tens when the video data is quantized with 8 bits.

Further, when it is determined that the edge includes an edge in the surrounding area, for example, the threshold value is proportional to the maximum value obtained by the surrounding area attribute determination unit 15 and inversely proportional to the distance. Vary with distance.
New threshold = Old threshold + (Maximum value of frequency components in the surrounding area) x (Standardization coefficient of maximum value) /
(Distance to the area where the maximum value of the frequency components in the surrounding area) × (distance normalization factor)
The reason for setting such a threshold is to prevent a portion that was originally flat from being determined as detail by mosquito noise.

(2) The center region is an edge.
If it is not determined to be flat in (1) above, the frequency components other than F00 are set in the horizontal direction (F01, F02, F03), the vertical direction (F10, F20, F30), and the horizontal + vertical direction as in the peripheral area attribute determination. (F11, F12, F13, F21, F22, F23, F31, F32, F33) are classified into three types, and the maximum value after the absolute value of the frequency component and the frequency component are obtained in each of the three categories. , F02, F20, and F22 have a maximum value, and when the maximum value is equal to or greater than a threshold value for determining an edge, the edge is determined.
This threshold is determined by considering various video sources, and is about several tens to several hundreds when video data is quantized with 8 bits.

(3) The center area is detail.
When it is determined that none of the above (1) and (2) is present, it is determined that the center region is the detail.

The above determination is based on the Hadamard transform of all the center region candidates including the pixel to be processed (for example, when the 4 × 4 Hadamard transform is shifted pixel by pixel, the Hadamard transform including the pixel to be processed includes 16 sets of center regions. You may perform using the average value of the conversion value of a candidate exists. For example, when the average value is equal to or greater than the threshold for edge determination, the edge is determined. When the average value is equal to or less than the threshold for flat determination, it is determined to be flat.
Or you may carry out using the ratio of frequency components. For example, the ratios of F01 and F02 and F02 and F03 in the horizontal frequency direction are obtained, and when both are greater than or equal to a predetermined threshold, the edge is determined to be detail, and when both are equal to or less than the predetermined threshold, it is determined to be flat.

FIG. 10 is a flowchart illustrating a processing procedure of the central region attribute determination unit 16 that determines the region attribute of the central region for the filter processing target image.
Of the 16 frequency components F00 to F33, which are the Hadamard transform results of the central region, the absolute values of all 15 frequency components excluding the DC component F00 are obtained (step S21).
When the absolute values of all frequency components are smaller than the threshold value for determining that the frequency is flat (step S22 / “component <threshold”), the frequency is determined to be flat (step S23), and control is transferred to step S29. When it is determined that an edge is included in the peripheral region of the filter processing target pixel, this threshold value is changed according to the maximum value and distance obtained by the peripheral region attribute determination unit 15.

On the other hand, if any of the absolute values of the frequency components is larger than the threshold value for determining that it is flat (step S22 / “component> threshold”), frequency components other than F00 are set in the horizontal direction (F01, F01, as in the peripheral region attribute determination). F02, F03), vertical direction (F10, F20, F30), horizontal + vertical direction (F11, F12, F13, F21, F22, F23, F31, F32, F33), and each of the three classifications The maximum value after the absolute value of the frequency component and its frequency component are obtained (step S24).
When the maximum value exists in any one of F02, F20, and F22 (step S25 / “Yes”) and the maximum value is equal to or greater than a threshold value for determining an edge (step S26 / YES), the central region is set as an edge region. Is determined (step S27).

  On the other hand, when the position of the element of the maximum value is not any of F02, F20, and F22 in step S5 (step S25 / “No”), or when the maximum value is smaller than the threshold value for determining the edge in step S26 (step S26 / NO), it is determined that the center area is the detail (step S28).

  Finally, the maximum value after the absolute value of the frequency component when it is determined as an edge whether the central region is a flat region, a detail region, or an edge region is output (step S29).

When there is an edge in the central region or the peripheral region based on information from the central region attribute determination unit 16 and the peripheral region attribute determination unit 15, the filter selection unit 17 uses an edge-preserving filter (for example, the above formula (1)). (Epsilon filter indicated by) is selected. The strength of the filter at this time is determined according to what kind of region the central region is.
In addition, the filter selection unit 17 has noise in the imaging system even in a seemingly uniform place such as the sky, and noise may be generated in the process of compressing / decompressing the noise. For this, a simple blur filter (for example, a 5-tap LPF (Low Pass Filter) expressed by the following equation (3)) is selected.

  Here, the selection of the filter is performed as follows.

(1) When the central region is a flat region and there are no edges in the peripheral region, it is considered that the entire region is flat. Therefore, by performing LPF processing, noise at a seemingly uniform place such as the sky is removed.

(2) If the center region is a detail region and there are no edges in the peripheral region, the filter is turned off and the detail is saved.

(3) When the center region is an edge region and there are no edges in the peripheral region, an ε filter (referred to as ε2 filter) that determines the threshold ε with the maximum value of the center region attribute determination unit 16 is selected, and noise around the edge is removed. To do. This threshold is determined based on the maximum value or the standardized maximum value. The standardization of the maximum value is obtained by examining various video sources, setting a normalization coefficient in advance, and multiplying by this normalization coefficient.
FIG. 11 is an example of characteristics when the threshold value of the ε filter is determined by the maximum value of the frequency component, and the threshold value ε increases in proportion to the maximum value.

(4) When the central region is a flat region and the peripheral region includes an edge, an ε filter (referred to as ε0 filter) that determines the threshold ε with the maximum value of the peripheral region attribute determination unit 15 is selected, and noise around the edge is selected. Remove. This threshold is determined based on the maximum value or the standardized maximum value (FIG. 11). The standardization of the maximum value is obtained by examining various video sources, setting a normalization coefficient in advance, and multiplying by this normalization coefficient.

(5) When the center region is a detail region and the peripheral region includes an edge, an ε filter (referred to as ε1 filter) that determines the threshold ε based on the maximum value and distance of the peripheral region attribute determination unit 15 is selected, and Noise is removed, but it is saved as detail away from the edge. The threshold ε at this time is determined so as to be proportional to the maximum value and inversely proportional to the distance. Alternatively, the maximum value or distance may be multiplied by each normalization coefficient set by considering various video sources in advance.
FIG. 12 is an example of characteristics when the threshold ε of the ε filter is determined by the maximum value of the frequency component and the distance from the filter target pixel to the region where the maximum value exists, and is proportional to the maximum value. The threshold ε is determined so as to be inversely proportional to.

(6) When the central region is an edge region and the peripheral region includes an edge, the threshold ε is set to the larger maximum value of the maximum value of the central region attribute determination unit 16 and the maximum value of the peripheral region attribute determination unit 15. The ε filter to be determined (referred to as ε3 filter) is selected, and noise around the edge is removed. This threshold is determined based on the maximum value or the standardized maximum value (FIG. 11). The standardization of the maximum value is obtained by examining various video sources, setting a normalization coefficient in advance, and multiplying by this normalization coefficient.

  The filter processing unit 18 performs the filter processing selected by the filter selection unit 17 on the filter processing target pixel, generates a pixel from which noise has been removed, and outputs the pixel from the output terminal 19.

FIG. 13 is a flowchart illustrating a processing procedure of the noise removal apparatus according to the present embodiment that performs the filtering process on all the filtering target pixels input from the input terminal 11.
Image data to be subjected to noise removal is input from the input terminal 11, and the input image data is orthogonally transformed by the orthogonal transformation processing unit 12 (step S31). This orthogonal transformation is repeatedly performed by obtaining a pixel block of a transform unit while shifting the transform unit by one pixel unit.

When the filtering process has been completed for all the pixels of the input image data (step S32 / No), the noise removal process is terminated.
On the other hand, if the filtering process has not been completed for all the pixels of the input image data (step S32 / "Yes"), a pixel for which the process has not been completed (hereinafter referred to as a filtering target pixel) is selected and the center is selected. The region extraction unit 13 extracts the Hadamard transformation result for the central region of the pixel to be filtered from the Hadamard transformation result transformed by the orthogonal transformation processing unit 12 (step S33).

  A peripheral region for the pixel to be filtered is set by the peripheral region extraction unit 14, and Hadamard transform results for all the pixels in the peripheral region are extracted from the Hadamard transform result converted by the orthogonal transform processing unit 12 (step S34). The peripheral region attribute determination unit 15 determines whether or not there is a region including an edge in the peripheral region from all the Hadamard transformation results in the peripheral region, and the presence or absence of the edge in the peripheral region, the maximum value ( Edge size) and distance are output (step S35).

  The central region attribute determination unit 16 determines whether the pixel to be filtered is a flat, detail, or edge region using the Hadamard transform result of the central region, and determines whether the central region is flat, detail, or edge. The maximum value after the absolute value of the frequency component when it is determined that the region is an edge is output (step S36).

When there is an edge in the central region or the peripheral region based on the information output in steps S35 and S36 by the filter selection unit 17, an edge-preserving filter is selected, and a simple blur filter is applied to a flat region. Select (step S37).
The filter processing unit 18 applies the filter processing selected in Step 37 to the pixel to be filtered, and outputs it from the output terminal 19. The filter processing target pixel is processed, and the process returns to Step S32 (Step S38).

With the configuration as described above, it is possible to remove the mosquito noise that stands out around the edge while preserving the detail portion by orthogonally transforming the input image data and analyzing it in the frequency domain.
In addition, since the flat portion is also subjected to LPF processing, there is imaging system noise at a seemingly uniform place such as the sky, and noise generated in the process of compressing / expanding it can be removed.

  Note that the present invention is not limited to the above-described embodiment, and various modifications and corrections can be made without departing from the scope of the present invention. For example, the function of each part of the noise removing device is realized as a computer program, and this computer program is executed. Further, the computer program can be recorded on a detachable recording medium, or downloaded via a network or broadcast wave, so that the transfer can be performed easily and easily.

It is a block diagram which shows the function structure which concerns on the noise removal apparatus of this invention. It is a figure explaining a Hadamard transform frequency distribution. It is a figure explaining the pixel block made into object by one orthogonal transformation. It is an example of a circuit that extracts a 4 × 4 pixel block from input image data. It is a figure explaining a complementation process when there is no number of pixels in performing orthogonal transformation. It is a figure explaining the center area | region with respect to a filter process target pixel. It is a figure explaining the peripheral area | region with respect to the filter process target pixel. It is a figure explaining extraction of the Hadamard transformation result with respect to a peripheral region. It is a flowchart which shows the process sequence of a surrounding area attribute determination part. It is a flowchart which shows the process sequence of a center area | region attribute determination part. It is an example of the threshold characteristic of the epsilon filter determined by the maximum value. It is an example of the threshold characteristic of the epsilon filter determined by the maximum value and distance. It is a flowchart which shows the process sequence of the noise removal apparatus of this invention. It is a block diagram which shows the function structure of the conventional noise removal apparatus. It is a figure explaining that the distinction of an edge part and a detail part is difficult in the conventional noise removal apparatus. It is a figure explaining that the removal of the mosquito noise which appears under the influence of the edge which exists in the distant place is difficult in the conventional noise removal apparatus.

Explanation of symbols

DESCRIPTION OF SYMBOLS 11 ... Input terminal, 12 ... Orthogonal transformation process part, 13 ... Center area extraction part, 14 ... Peripheral area extraction part, 15 ... Peripheral area attribute determination part, 16 ... Center area attribute determination part, 17 ... Filter selection part, 18 ... Filter processing unit, 19 ... output terminal, 110 ... input terminal, 120 ... MPEG decoder, 130 ... edge detection, 140 ... noise reduction unit, 150 ... output terminal.

Claims (20)

In a noise removal device that removes noise by filtering input image data,
For each transform unit block for orthogonal transform, an orthogonal transform processing unit that orthogonally transforms the image data;
A central region extraction unit that extracts an orthogonal transformation result for the central region of the pixel to be filtered from the orthogonal transformation result generated by the orthogonal transformation processing unit;
A peripheral region extraction unit that extracts an orthogonal transformation result corresponding to all pixels in the peripheral region with respect to the filtering target pixel from the orthogonal transformation result generated by the orthogonal transformation processing unit;
From all the orthogonal transformation results in the peripheral region obtained by the peripheral region extraction unit, determine the attribute of the peripheral region, a determination result, a peripheral region attribute determination unit that outputs a maximum value and a distance;
Using the orthogonal transformation result of the central area, determine the attribute of the central area, and output a determination result and a maximum value, a central area attribute determination unit;
A filter selection unit that selects a filter of a pixel to be filtered based on output information from the central region attribute determination unit and the peripheral region attribute determination unit;
A noise removal apparatus comprising: a filter processing unit that performs a filter process selected by the filter selection unit on a filter processing target pixel to generate a pixel from which noise has been removed.   The noise removal apparatus according to claim 1, wherein the orthogonal transform is a 4 × 4 Hadamard transform.   3. The noise removing device according to claim 1, wherein the orthogonal transform processing unit performs orthogonal transform by shifting the transform unit block by the number of pixels within the size of the orthogonal transform. 4.   4. The noise removing apparatus according to claim 1, wherein the peripheral area is determined based on a unit block size of a compression format of input image data.   5. The noise removal device according to claim 1, wherein the attribute determination of the peripheral region is performed when a maximum absolute value of frequency components of all orthogonal transformation results in the peripheral region is equal to or greater than an edge threshold value. , A noise removing device that determines that an edge is included, and otherwise determines that an edge is not included.   6. The noise removal apparatus according to claim 1, wherein the attribute determination of the center region is performed when the absolute value of all frequency components that are the result of orthogonal transformation of the center region is smaller than a flat threshold value. It is determined that the noise removal device.   The noise removal apparatus according to claim 6, wherein the flat threshold value is changed according to a maximum value and a distance obtained by the peripheral area attribute determination unit when it is determined that an edge is included in the peripheral area. A noise removal device.   The noise removal apparatus according to any one of claims 1 to 5, wherein the attribute determination of the center region is performed when the maximum absolute value of the frequency component of the orthogonal transformation result with respect to the center region is equal to or greater than an edge threshold value. Is determined to be an edge region.   The noise removal apparatus according to any one of claims 1 to 5, wherein in the attribute determination of the center region, any one of absolute values of frequency components as a result of orthogonal transform of the center region is larger than a flat threshold value, and A noise removing apparatus, characterized in that, when a maximum absolute value of frequency components of an orthogonal transformation result with respect to a center region is equal to or less than an edge threshold value, the center region is determined to be a detail region.   10. The noise removal device according to claim 6, wherein the attribute determination of the center region is performed using an average value of frequency components of orthogonal transform results of all center region candidates including the filtering target pixel. A noise removing device characterized by:   10. The noise removal device according to claim 6, wherein the attribute determination of the center region is performed using a ratio between frequency components of an orthogonal transformation result with respect to the center region. .   12. The noise removing device according to claim 1, wherein the filter selecting unit selects an edge preserving type filter when there is an edge in a central region or a peripheral region. .   13. The noise removal apparatus according to claim 12, wherein the filter selection unit determines the threshold value ε with the maximum value obtained by the center region attribute determination unit when the center region is an edge region and there is no edge in the peripheral region. A noise removing device characterized by selecting   13. The noise removal device according to claim 12, wherein when the central region is a flat region and the peripheral region includes an edge, an ε filter that determines the threshold ε with the maximum value obtained by the peripheral region attribute determination unit is selected. A featured noise removal device.   The noise removal apparatus according to claim 12, wherein when the center region is a detail region and the peripheral region includes an edge, an ε filter that determines a threshold ε based on the maximum value and the distance obtained by the peripheral region attribute determination unit is provided. A noise removing device characterized by being selected.   13. The noise removal device according to claim 12, wherein when the center region is an edge region and the peripheral region includes an edge, the maximum value obtained by the center region attribute determination unit and the maximum value obtained by the periphery region attribute determination unit. A noise removing apparatus that selects an ε filter that determines a threshold value ε with a large maximum value.   12. The noise removing device according to claim 1, wherein the filter selecting unit selects a simple blur filter in a flat region. In the noise removal method that removes noise by filtering input image data,
Orthogonal transformation processing step for orthogonal transformation of the image data for each transformation unit block of orthogonal transformation;
A central region extraction step for extracting an orthogonal transformation result for the central region of the pixel to be filtered from the orthogonal transformation result generated in the orthogonal transformation processing step;
From the orthogonal transformation result generated in the orthogonal transformation processing step, a peripheral region extraction step for extracting an orthogonal transformation result corresponding to all pixels in the peripheral region with respect to the filter processing target pixel;
From all orthogonal transformation results in the peripheral region obtained by the peripheral region extraction step, determine the attribute of the peripheral region, and determine the peripheral region attribute determination step for outputting the maximum value and distance;
Using the orthogonal transformation result of the central region, determine the attribute of the central region, and output a determination result and a maximum value, a central region attribute determining step;
A filter selection step of selecting a filter of a pixel to be filtered by output information from the central region attribute determination step and the peripheral region attribute determination step;
A noise removal method comprising: a filter processing step of applying a filter process selected by a filter selection unit to a filter processing target pixel to generate a pixel from which noise has been removed.   The program for functioning a computer as each part of the noise removal apparatus in any one of Claims 1 thru | or 17.   A computer-readable recording medium on which the program according to claim 19 is recorded.

Images