WO2012102191A1 - Image data processing apparatus, image data processing method, image data processing program, and computer-readable recording medium - Google Patents

Abstract

An image data processing apparatus (1) is provided with: a data separation unit (10) for separating input image data into first image data that does not include edge areas, and data other than the first image data; a standard deviation calculation unit (22) for calculating standard deviation of brightness data of each of the pixels contained within the first image data, and areas surrounding the pixels; and a mosquito-noise calculation unit (23) for calculating the degree that mosquito-noise is produced for each of the pixels of the input image data, using the standard deviation.

Description

Image data processing apparatus, image data processing method, image data processing program, and computer-readable recording medium

The present invention relates to an image data processing device, an image data processing method, an image data processing program, and a computer-readable recording medium that reduce mosquito noise included in an image signal.

When digital image data and digital video data are stored or transferred, the video data (hereinafter, digital image data and digital video data are collectively referred to as video data) is encoded to reduce the data capacity. Is common. In MPEG (Moving Picture Experts Group), which is a standard method of video data encoding, video data is blocked, each block is subjected to two-dimensional DCT (Discrete Cosine Transform) processing, and the high-frequency component conversion coefficient is regarded as 0. Encode by doing.

On the other hand, it is necessary to decode in order to display or play back the encoded video data. At the time of decoding, mosquito noise is generated due to a high-frequency component conversion coefficient regarded as 0 at the time of encoding.

When there is a sharp brightness change such as an edge in the video data, a lot of high-frequency components are generated, resulting in an increase in conversion coefficients that are missing during encoding. For this reason, it is known that mosquito noise is likely to occur around the edge where the luminance changes rapidly.

Conventionally, it is known that mosquito noise is reduced by processing video data with, for example, a low-pass filter (LPF). However, this method has the problem that the entire video data is smoothed equally by LPF, and at the same time, mosquito noise is reduced, and at the same time, high-frequency components other than noise such as fine patterns in video data are lost. It was.

In order to solve this problem, the following technology is disclosed. The video data is divided into blocks, and it is determined whether each block is an edge region, a texture region, or a flat region, and noise removal processing is performed on the video data using a nonlinear smoothing filter (see Patent Document 1). ).

In addition, focusing on the fact that mosquito noise generally occurs near the edges, edge detection, dispersion value detection, and difference calculation are performed on video data, and a pixel region including mosquito noise is determined based on the results. A technique is disclosed (Patent Document 2).

Japanese Patent Publication “Japanese Patent Laid-Open No. 2008-278185 (Publication Date: November 13, 2008)” Japanese Patent Publication “Japanese Unexamined Patent Application Publication No. 2009-225299 (Publication Date: October 1, 2009)”

In any of the above techniques, mosquito noise is determined by detecting an edge region from video data and calculating a variance value for data around the pixel. However, since the periphery of the edge is an area where the luminance change is large, the data dispersion value in the area inevitably increases. Therefore, when there is a significant high-frequency component that does not have a large luminance change around the edge, it is difficult to distinguish it from mosquito noise. In addition, the above-mentioned brightness | luminance change is not large and a significant high frequency component is set as a detail part below.

The present invention has been made in view of the above problems, and performs mosquito noise determination with high accuracy in the entire area of video data, effectively removing mosquito noise while retaining the detail portion included in the image data, The purpose is to obtain output image data with improved image quality.

In order to solve the above problems, the image data processing apparatus of the present invention separates input image data into first image data that does not include an edge region and second image data other than the first image data. A standard deviation value calculating means for calculating a standard deviation value of luminance data in each pixel included in the first image data and a peripheral area of the pixel as a first standard deviation value, and using the first standard deviation value. Mosquito noise calculating means for calculating the degree of occurrence of mosquito noise for each pixel of the input image data.

Further, in order to solve the above problems, the image data processing method of the present invention separates input image data into first image data not including an edge region and second image data other than the first image data. A separation step; a standard deviation value calculating step for calculating a standard deviation value of luminance data in each pixel included in the first image data and a peripheral region of the pixel as a first standard deviation value; and the first standard deviation value And a mosquito noise calculating step for calculating the degree of occurrence of mosquito noise for each pixel of the input image data.

In the above configuration, the input image data (O data) is divided into first image data (T data) that does not include information related to the edge region by the data separation means (edge separation step), and second data other than the first image data. Separated into image data (C data). Then, the standard deviation value of the first image data is calculated as the first standard deviation value in the standard deviation value calculating means (standard deviation value calculating step), and the image data processing is performed using the first standard deviation value, thereby obtaining the mosquito. In the noise calculation means (mosquito noise calculation step), the degree of occurrence of mosquito noise is calculated.

In the prior art, an edge portion is detected from input image data, and numerical processing such as dispersion value detection is performed on the peripheral region of the edge portion to determine a pixel region including mosquito noise. In the above prior art, mosquito noise is discriminated by paying attention to the fact that mosquito noise is likely to occur at the edge portion of the input image data due to the principle of encoding.

However, the input image data in the peripheral area of the edge part has a large difference value of luminance data caused by the edge. Therefore, it is difficult to discriminate between mosquito noise and the above-mentioned details when the peripheral area of the edge portion includes significant image data (hereinafter referred to as detail) having a small luminance difference value other than mosquito noise. there were.

In the present invention, with respect to the first image data not including the edge region, the standard deviation value calculating means calculates the standard deviation value of the luminance data in each pixel and the peripheral region of the pixel as the first standard deviation value. Further, the mosquito noise calculation means calculates the degree of occurrence of mosquito noise using the first standard deviation value.
As a result, it is possible to determine the degree of occurrence of mosquito noise regardless of a large difference value caused by the edge portion of the input image data. Therefore, it is possible to discriminate between fine image data existing in the peripheral area of the edge portion and mosquito noise, which is difficult with the prior art.

According to the present invention, the data from which the influence of the edge area is removed is separated from the video data, and the mosquito noise determination is performed using the data from which the influence of the edge area has been removed. Is possible. As a result, it is possible to effectively remove mosquito noise while retaining the detail portion included in the image data, and obtain output image data with an improved image quality.

1 is a block diagram illustrating a configuration of an image data processing apparatus according to an embodiment of the present invention. It is a block diagram which shows the structure of the digital television apparatus using the image data processing apparatus which is one Embodiment of this invention. 3A is a diagram showing an example of image data (alphabet a) that has not been subjected to image data processing, and FIG. 3B encodes the image data of FIG. 3A. It is a figure which shows the image data decoded after having performed. FIG. 4A is a diagram showing decoded image data of the letter a. FIG. 4B is a diagram illustrating image data when edge-considered video smoothing processing is performed on the image data of FIG. FIG. 4C is a diagram showing image data obtained by decoding a minute pattern (detail) with a small amount of change in luminance. FIG. 4D is a diagram illustrating image data when edge-considered video smoothing processing is performed on the image data in FIG. It is a block diagram which shows the structure of the data separation part with which the image data processing apparatus which is one Embodiment of this invention is provided. It is a block diagram which shows the structure of the mosquito noise detection part with which the image data processing apparatus which is one Embodiment of this invention is provided. It is a figure which shows the attention pixel (P _{i, j} ) and attention pixel surrounding area in the case of calculating the standard deviation value in the standard deviation value calculation unit provided in the image data processing apparatus according to the embodiment of the present invention. (A) of FIG. 8 is a figure which shows the general form of the 1st function which calculates the 1st mosquito noise generation degree from a 1st standard deviation value. FIG. 8B is a diagram showing an outline of the second function for calculating the second mosquito noise occurrence degree from the second standard deviation value. 9 (a), 9 (b), and 9 (c), the standard deviation value for each pixel obtained in the standard deviation value calculation unit is stored for all pixels for one frame. It is a figure which shows the histogram obtained every time. It is a block diagram which shows the structure of the mosquito noise reduction part with which the image data processing apparatus which is one Embodiment of this invention is provided. FIG. 11A shows an example of a decoded image. FIG. 11B, FIG. 11C, and FIG. 11D are views showing images obtained by enlarging the α part of FIG. 11A. In particular, FIG. 11B shows an image before image processing, FIG. 11C shows an image after conventional edge-considered video smoothing processing, and FIG. 11D shows image data processing of the present invention. It is a figure which shows each subsequent image.

(E) in FIG. 11, (f) in FIG. 11 and (g) in FIG. 11 are images showing an enlarged view of the β portion in (a) in FIG. 11 (e) shows an image before image processing, FIG. 11 (f) shows an image after conventional edge-considered video smoothing processing, and FIG. 11 (g) shows an image after image data processing of the present invention. It is a figure which shows an image, respectively.
FIG. 12A shows an example of a decoded image. FIG. 12B is an image after the mosquito noise is reduced by applying the first mosquito noise detection method to the image of FIG. (C) of FIG. 12 is an image after the mosquito noise is reduced by applying the third mosquito noise detection method to the image of (a) of FIG. FIG. 12D is an image after the mosquito noise is reduced by applying the conventional bilateral filter to the image of FIG. 1 is a block diagram illustrating a configuration of a PC that performs image data processing of image data distributed via a network, and an image display system according to an embodiment of the present invention.

Hereinafter, embodiments of an image data processing device and a display device according to an embodiment of the present invention will be described with reference to FIGS. The embodiment described below is merely a specific example of the present invention, and the present invention is not limited to these in any way.

[1. About Digital TV Device 100]
FIG. 2 is a block diagram showing a configuration of the digital television apparatus 100 using the image data processing apparatus 1. The digital television device 100 includes a receiving device 101, an image data processing device 1, and a display device 110. The receiving device 101 includes an antenna 102, a tuner 103, and a decoder 104, and the display device 110 includes an image post-processing unit 111 and a display unit 112.

The image data processing apparatus 1 includes a data separation unit (data separation unit) 10, a mosquito noise detection unit 20, and a mosquito noise reduction unit (mosquito noise reduction unit) 30.

The digital TV apparatus 100 receives digital image data (hereinafter referred to as image data) of digital TV broadcast by the receiving apparatus 101 and displays it on a display unit 112 typified by, for example, a liquid crystal display panel or a plasma display panel. Device.

Here, when transferring or storing image data, the image data is encoded in order to reduce the data capacity. Encoding is to compress image data capacity by making image data into blocks, performing two-dimensional DCT (Discrete Cosine Transform) processing on each block, and assuming that the high-frequency component conversion coefficient is zero.

The receiving device 101 of the digital television device 100 that has received the encoded image data decodes the encoded image data with the decoder 104, thereby making the image data displayable on the display unit 112.

Note that the decoded image data can be displayed on the display unit 112 as it is. However, in order to realize a better appearance, a plurality of image processing such as noise reduction, color management, and enhancement are performed in the image post-processing unit 111.

As described above, by receiving and decoding the encoded image data, it becomes possible to view a digital television broadcast, but there are noises that are difficult to avoid due to the encoding principle. As described above, encoding is to perform DCT processing on blocked image data. During the DCT processing, many high-frequency components are generated in a region where there is a rapid luminance change (that is, an edge region), and as a result, many conversion coefficients are regarded as zero.

Then, when image data in which many conversion coefficients are regarded as 0 is decoded and reproduced, wavy noise appears because many high-frequency components of the conversion coefficients are missing in the edge region. Such a noise caused by missing high-frequency components in the conversion coefficient of image data in accordance with the encoding DCT processing is known as so-called mosquito noise.

In order to reduce the occurrence of the mosquito noise, it is necessary to devise such as finely quantizing the transform coefficient during the DCT processing. However, in this case, there is a problem that the image data capacity is not sufficiently compressed. For this reason, in general encoding, mosquito noise that occurs in an edge region where the luminance changes rapidly is unavoidable.

Here, the mosquito noise will be described more specifically with reference to FIGS. 3A and 3B. FIG. 3A shows image data before encoding, and FIG. 3B shows image data after encoding and decoding the image data. In the encoded and decoded image data shown in FIG. 3B, wavy noise that is not present in the pre-encoded image data shown in FIG. 3A can be confirmed around the alphabet a. This wavy noise is so-called mosquito noise.

In order to reduce mosquito noise, it is conventionally known to perform noise processing represented by a low-pass filter (LPF) (hereinafter referred to as edge-considered video smoothing processing) on image data.

Note that the edge-considered video smoothing process here refers to an edge area where the luminance value changes abruptly and an area where the luminance value hardly changes (referred to as a flat area) in a more clearly visible state. It means that image data including high frequency components is removed by applying smoothing. The LPF is exemplified as the edge-considered video smoothing process, but among them, for example, an LPF that considers not only the smoothing according to the distance between pixels but also the luminance difference like the bilateral filter is desirable. Any method other than the LPF can be used as long as the above-described image processing can be realized.

In order to explain the conventional mosquito noise reduction processing, the decoded image data is shown in FIG. 4A, and the result of performing edge-considered video smoothing processing on the image data is shown in FIG. 4B. . In FIG. 4B, it can be confirmed that the mosquito noise is reduced and the letter a is easy to see.

However, in this conventional method, the mosquito noise is reduced by smoothing the image data including the high-frequency component as described above and blurring the entire image. Therefore, although it is not mosquito noise, the luminance change is not so large and a portion including a high-frequency component (hereinafter referred to as a detail portion) is blurred, and the detail portion is lost. There was a problem. In order to explain the loss of the detail portion, the image data after decoding the detail portion is shown in FIG. 4C, and the result of performing the edge-considered video smoothing process on the image data is shown in FIG. Show. By performing the edge-considered video smoothing process, it can be confirmed that the lattice-like detail portion existing in FIG. 4C is lost in FIG. 4D.

[Outline of Image Data Processing Apparatus 1]
In the present invention, in order to effectively reduce mosquito noise without losing detail, the image data processing apparatus 1 is installed between the receiving apparatus 101 and the display apparatus 110 of the digital television apparatus 100 shown in FIG. I have. Hereinafter, the configuration of the image data processing apparatus 1 will be described more specifically.

As shown in FIG. 1, the image data processing apparatus 1 includes a data separation unit 10, a mosquito noise detection unit 20, and a mosquito noise reduction unit 30. Then, the image data decoded by the decoder 104 (FIG. 2) of the receiving apparatus 101 (FIG. 2) is sent to the data separation unit 10. This decoded image data is hereinafter referred to as input image data.

[2. Data separation unit 10]
Next, a more specific configuration of the data separation unit 10 will be described with reference to FIG. The input image data sent to the data separation unit 10 is duplicated into three identical data, and one of the input image data is subjected to edge-considered video smoothing processing. Here, the edge-considered video smoothing process means that input image data is processed using a low-pass filter or a bilateral filter. Thus, by performing edge-considered video smoothing processing on the input image data, it is possible to obtain image data from which high-frequency components including mosquito noise and details are removed from the input image data. Note that the image data from which the high-frequency components have been removed is simply referred to as C data (Carton data, second image data) below.

The C data is image data obtained by removing high frequency components from the input image data, and includes information on the edge region. Then, by taking the difference between the C data and one of the duplicated input image data, image data including high-frequency components such as mosquito noise and detail is generated without including edge region information. The image data created in this way is hereinafter simply referred to as T data (Texture data, first image data). Further, the C data is used to create T data, and at the same time, the C data itself is output from the data separation unit 10.

Further, one of the three duplicated input image data is output from the data separation unit 10 as it is. Hereinafter, this data is simply referred to as O data (original data).

As described above, the data separation unit 10 outputs T data that does not include information related to the edge region, C data that includes information related to the edge region, and O data that is input image data.

[3. Mosquito noise detector 20]
Next, the mosquito noise detection unit 20 will be described. As shown in FIG. 6, the mosquito noise detector 20 includes a luminance value calculator 21, a standard deviation value calculator (standard deviation value calculator) 22, a mosquito noise calculator (mosquito noise calculator) 23, and a histogram. Is stored in the storage unit 24.

In performing mosquito noise detection in the mosquito noise detection unit 20,
(1) When using only T data,
(2) When T data and C data are used, and (3) When T data, C data, and O data are used.

In the present embodiment, (1) the case where mosquito noise detection is performed using only T data, which is the basic of the above three methods, will be described, and other mosquito noise detection methods will be described later. In the following,
(1) A detection method using only T data is a first mosquito noise detection method,
(2) The detection method using T data and C data is the second mosquito noise detection method, and
(3) A detection method using T data, C data, and O data is defined as a third mosquito noise detection method.

In any of the first to third mosquito noise detection methods, T data is input from the data separation unit 10 to the luminance value calculation unit 21 included in the mosquito noise detection unit 20. In the mosquito noise detection unit 20 of FIG. 6, as an example of the embodiment, a case where three data of T data, C data, and O data are input to the luminance value calculation unit 21 is illustrated. However, in the first mosquito noise detection method, only T data is input to the luminance value calculation unit 21.

Here, in order to understand the details of the luminance value calculation unit 21, it is necessary to describe the configuration of the liquid crystal display 113, which is an example of the display unit 112, and will be described with reference to FIG. As shown in FIG. 7, the liquid crystal display panel 113 includes pixels including, for example, red (R), green (G), and blue (B) sub-pixels.

Then, the luminance value calculation unit 21 calculates luminance data for each pixel in the frame of the liquid crystal display panel 113 from the T data of the sub-pixels as described below. In the following description, the pixel at the coordinates (i, j) shown in FIG. 7 is P _{i, j,} and the luminance data obtained from the T data of P _{i, j} is T_Y _{i, j} . Here, if the frame size of the liquid crystal display panel 113 is 1920 horizontal pixels and 1080 vertical pixels, 1 ≦ i ≦ 1920 and 1 ≦ j ≦ 1080.

Also, the T data of R, G, and B of the sub-pixels of P _{i, j} are T_R _{i, j} , T_G _{i, j} , and T_B _{i, j} , respectively. In the case of the digital television device 100, T_Y _{i, j} can be calculated by the following equation, for example.
T_Y _{i, j} = 0.213 × T_R _{i, j} + 0.715 × T_G _{i, j} + 0.072 × T_B _{i, j}
When the luminance value calculation unit 21 calculates T_Y _{i, j} for all pixels in the frame as described above, the T_Y _{i, j} is sent to the standard deviation value calculation unit 22 which is a standard deviation value calculation unit. It is done.

Further, in FIG. 6, as the luminance data that is input to the standard deviation calculation unit 22, the luminance data _{(O_Y i, j)} of O data and luminance data _{(c_y i, j)} of the C data and _{T_y i,} and the _j Are listed. In this regard, in the first mosquito noise detection method, since the mosquito noise detected by using only the T data, T_y i, only _j are input to the standard deviation calculation unit 22.

Then, the standard deviation value calculation unit 22 calculates the standard deviation value of the luminance data in each pixel (for example, P _{i, j} ) in the frame and the peripheral region of P _{i, j} as the first standard deviation value. Hereinafter, the first standard deviation value is defined as T_STD _{i, j} .

In addition, the size of the peripheral area of P _{i, j} when calculating T_STD _{i, j} is preferably about the compressed block size at the time of encoding. Specifically, 8 pixels × 8 pixels is appropriate, but there is no problem if the size is about 3 pixels × 3 pixels to 11 pixels × 11 pixels. In this embodiment, for the sake of simplicity, the peripheral area of each pixel is described as 3 pixels × 3 pixels (see an enlarged view in FIG. 7).

T_STD _{i, j} is calculated from the luminance data in each pixel and the surrounding area of each pixel according to the following formula.

Here, T_MEAN _{i, j} represents an average value of luminance data in a peripheral region including P _{i, j} , and T_MEAN2 _{i, j} represents a mean square value. In addition, since the surrounding area is 3 pixels × 3 pixels, i−1 ≦ k ≦ i + 1 and j−1 ≦ l ≦ j + 1.

Here, T_Vari _{i, j} represents the dispersion value of the luminance data in the peripheral area including P _{i, j} .

Then, the standard deviation value calculation unit 22 calculates T_STD _{i, j} of all pixels in the frame as described above _, and outputs it to the mosquito noise calculation unit 23. In addition, in order to save a histogram (details will be described later) obtained by counting T_STD _{i, j} for each frame, the standard deviation value calculation unit 22 also outputs T_STD _{i, j} to the storage unit 24 (see FIG. 6). ).

The mosquito calculation unit 23, which is a mosquito noise calculation means, uses T_STD _{i, j} to calculate the mosquito noise occurrence degree (T_W _{i, j} ) for each pixel in the frame from the following equation and outputs it (FIG. 6). reference).
_{T_W i, j = F (|} T_STD i, j -Pt' |)
Here, the form of the function F (x) corresponding to the first function described in the claims is shown in FIG. As shown in FIG. 8A, the function F (x) shows a local maximum value as an output value when the absolute value of the input value x is minimum, and the output value as the input value x increases. Is a function that monotonously decreases and the output value becomes 0 when the input value x is the first predetermined value. Here, the first predetermined value can also be expressed as an x-intercept of the function F (x), and its value is Tm. When Tm is a fixed value, the value of Tm is preferably 0.5 ≦ Tm ≦ 2.

Further, Pt and Pt ′ are defined as follows. The storage unit 24 for storing the histogram shown in FIG. 6 aggregates the first standard deviation values of the previous frame (assumed to be n−1 frames) of the frame in which mosquito noise detection is performed (this is assumed to be n frames). Stored as a histogram. FIG. 9A shows a T_STD histogram of n−1 frames stored in the storage unit 24 that stores the histogram. Here, Pt is the value of T_STD that maximizes the number of pixels in the T_STD histogram of the (n−1) th frame, and Pt ′ is defined as the following equation as a value that is larger than Pt by δt.
Pt ′ = Pt + δt
In this case, it is empirically known that the value of δt is desirably 0.5 ≦ δt ≦ 1.

Incidentally, as seen from the form of the function F (x), calculates the mosquito calculator 23 _{T_W i,} 0 ≦ _{T_W i} as _j, a value in the range of _j ≦ 1. Then, _{T_W i,} as _j increases, the pixel _{P i, j} can be determined to contain a lot of mosquito noise.

Here, in order to explain the basic concept in one embodiment of the present invention, T_Wi _{, j} is a parameter at the time of calculation, Tm and δt are fixed values. However, as will be described later, O data By calculating the standard deviation value (O_STD _{i, j} ) of the luminance data in each pixel and the peripheral area of each pixel, it is possible to determine Tm and δt optimized for each frame. However, even when Tm and δt are fixed values and the degree of occurrence of mosquito noise is simply estimated, it is possible to improve the image quality (to reduce mosquito noise while leaving details).

[4. Mosquito noise reduction unit 30]
Mosquito noise occurrence rate T_W i for each pixel calculated by the mosquito noise calculating means 23 as described above, _j is output to the mosquito noise reduction unit 30 (see FIG. 10). The mosquito noise reduction unit 30 includes a mosquito noise determination unit (mosquito noise determination unit) 31 and an output processing unit 34 which are configured by an occurrence degree total unit 32 and a mosquito function determination unit 33.

In the mosquito noise determination according to one aspect of the present invention, the average value of the mosquito noise occurrence degree of the entire frame is calculated from the mosquito noise occurrence degree for each pixel calculated by the mosquito noise calculation unit 23, and the average mosquito noise occurrence degree is calculated. From the value, it is judged how much the frame contains mosquito noise. Below, the mosquito noise reduction part 30 is demonstrated.

The mosquito noise generation degree T_W _{i, j for} each pixel input to the mosquito noise reduction unit 30 is duplicated into two, and one of them is input to the generation degree totaling unit 32. The generation degree summation unit 32 calculates an integrated value T_Sum of T_Wi _{, j} for all pixels of the frame according to the following equation.

Here, for example, if the frame size is 1920 horizontal pixels and 1080 vertical pixels, the integration range is 1 ≦ p ≦ 1920 and 1 ≦ q ≦ 1080.

The calculated T_Sum is output to the mosquito function determination unit 33, and is divided by the total number of pixels of the frame to calculate the mosquito noise occurrence average value (T_Ave).

The magnitude relationship between T_Ave and a predetermined threshold value is compared. If T_Ave is greater than the threshold value, it is determined that the ratio of mosquito noise in the frame is high, and MJ = 1 is output as the mosquito noise determination value (MJ). The data is output to the processing unit 34. On the other hand, if T_Ave is smaller than the threshold value, it is determined that the ratio of mosquito noise in the frame is low, and MJ = 0 is output to the output processing unit 34.

判断 By changing the threshold value used at this time, it is possible to arbitrarily change the judgment standard of the mosquito noise judging means. In the experience so far, it has been obtained that the threshold is preferably set to about 0.7, but it may be set to a value other than 0.7. Increasing the threshold means reducing the mosquito noise criterion, and conversely decreasing the threshold means stricter mosquito noise criterion.

This threshold value may be determined in advance as a value considered optimal by the manufacturer, or may be designed so that the user can change it within a certain range according to his / her preference. By changing the threshold, the mosquito noise criterion can be changed arbitrarily, so that flexible processing is possible depending on the processing capability of the image data processing device and the level of compression during encoding. It becomes.

MJ (= 0 or 1), T_W _{i, j} , C data, and O data are input to the output processing unit 34 (see FIG. 10), and the following calculation is performed according to the value of MJ. .

P _{outi, j} = O _{i, j} (when MJ = 0)
P _{outi, j} = (1−T_W _{i, j} ) × O _{i, j} + T_W _{i, j} × C _{i, j} (when MJ = 1)
Here, P _{outi, j} is output image data for each pixel in the frame. When MJ = 0, since the ratio of mosquito noise included in the frame is low, the input image data for each pixel is used as output image data for each pixel as it is.

On the other hand, when MJ = 1, since the ratio of mosquito noise included in the frame is high, the mosquito noise reduction processing is performed for each pixel by calculating P _{outi, j} according to the above equation. According to the above formula, mosquito noise generated from T data is generated for each pixel of O data as input image data and C data subjected to edge-considered video smoothing processing (for example, LPF processing). Weighting is performed using the degree T_Wi _{, j} , and the O data and the C data are combined.

As a result, the balance between maintaining the details included in the input image data in the output image data and the effect of reducing the mosquito noise is balanced for each pixel based on the mosquito noise occurrence degree T_Wi _{, j.} It is possible to obtain output image data that can be set appropriately and whose image quality is improved more appropriately.

FIG. 11 shows an example of improvement in image quality when image data processing according to one embodiment of the present invention is performed. FIG. 11A shows an example of a decoded image. This decoded image is composed of a portion having a large luminance difference represented by the α portion and a detail portion in the β portion. FIG. 11B shows an image obtained by enlarging the α portion in FIG. 11A, and FIG. 11E shows an image obtained by enlarging the β portion. In FIG. 11B, mosquito noise can be confirmed around the letter a.

The image data obtained as a result of performing the edge-considered video smoothing process conventionally known for the decoded image data is shown in FIG. 11 (c) and FIG. 11 (f). It can be confirmed from FIG. 11C that the mosquito noise existing around the alphabet a is reduced and the image quality is improved. However, it can be confirmed from FIG. 11F that the detail portion is lost simultaneously with the reduction of the mosquito noise.

On the other hand, image data obtained as a result of performing image data processing according to one embodiment of the present invention is shown in FIG. 11 (d) and FIG. 11 (g). From these two figures, it can be confirmed that the reduction of the mosquito noise and the retention of the detail portion are realized at the same time.

As described above, by performing the image data processing according to one aspect of the present invention, it is possible to effectively reduce mosquito noise while retaining the detail portion included in the input image data, Output image data with improved image quality can be obtained more appropriately.

[5. Mosquito noise detection using T data and C data)
In the description of the mosquito noise detection unit 20, the method using only T data has been described as the most basic mosquito noise detection method of the present invention. Here, however, mosquito noise detection is performed using T data and C data. The method to perform, ie, the 2nd mosquito noise detection method, is demonstrated. This method is an extension of the method using only the T data, and realizes more precise mosquito noise detection.

The first mosquito is that the data separation unit 10 (data separation means) included in the image data processing apparatus 1 receives the decoded input image data and outputs it as three data of T data, C data, and O data. This is the same as the noise detection method.

In the second mosquito noise detection method, as shown in FIG. 6, T data and C data are sent to the luminance value calculation unit 21 included in the mosquito noise detection unit 20. In the brightness value calculation unit 21, the luminance value of T data _{(T_Y i, j)} and likewise, to calculate the luminance value of the C data _{(C_Y i, j).} That is, the C data of R, G, and B of the sub-pixels of P _{i, j} is set to C_R _{i, j} , C_G _{i, j} , and C_B _{i, j} , respectively. _{i, j} are calculated.
C_Y _{i, j} = 0.213 × C_R _{i, j} + 0.715 × C_G _{i, j} + 0.072 × C_B _{i, j}
Thus, T_Y _{i, j} and C_Y _{i, j} calculated by the luminance calculating unit 21 are output to the standard deviation value calculating unit 22 which is a standard deviation value calculating unit. Then, _{T_y i,} calculates _j, and _{c_y i,} the standard deviation calculating section 22 receives the _j, the standard deviation value for each pixel T_std _{i, j,} and C_STD _i, a _j. Here, T_std _i, the method for calculating the _j are, T_std _i in the first mosquito noise detection method _is the same as the method of calculating the _{j. Further,} C_STD _i, the method for calculating the _{j are, C_Y i,} T_STD _i, may be calculated as with _j with _j. T_STD _{i, j} and C_STD _{i, j} obtained by the above method are output to the mosquito noise calculation unit 23, which is a mosquito calculation means, as a first standard deviation value and a second standard deviation value, respectively.

Next, mosquito noise calculation unit 23 Mosquito by T_W i, and _j as the first mosquito noise occurrence rate, C_W i, is calculated from the following equation and _j as the second mosquito occurrence rate, taking each product The degree of noise generation (W _{i, j} ) is obtained. Here, T_W _{i, j} is the same as in the first mosquito noise detection method.

T_W _{i, j} = F (| T_STD _{i, j} −Pt ′ |)
C_W _{i, j} = G (| C_STD _{i, j} −Pc |)
W _{i, j} = T_W _{i, j} × C_W _{i, j}
Here, the form of the function F (x) corresponding to the first function described in the claims is shown in FIG. 8A, and the form of the function G (x) corresponding to the second function is shown in FIG. As shown in (b) of FIG. The function F (x) is the same as that used in the first mosquito detection method.

The other second function, function G (x), is also a function having a form similar to function F (x). That is, the function G (x) shows a maximum value as an output value when the absolute value of the input value x is minimum, and the output value monotonously decreases as the input value x increases. This is a function whose output value is 0 when x is a second predetermined value. In other words, the second predetermined value is the x intercept of the function G (x), and this value is Cm. When Cm is a fixed value, the value of Cm is preferably 0.5 ≦ Cm ≦ 2.

Also, Pc is the value of C_STD that maximizes the number of pixels in the C_STD histogram (see FIG. 9B), and is considered to be the standard deviation value of the pixels in the edge region. Further, the second standard deviation value in the vicinity of Pc is considered as the standard deviation value of the peripheral pixels in the edge region.

Here, in order to reduce the amount of calculation, Tm and δt of the first function and Cm of the second function are fixed values. However, as described in the section of the third mosquito noise detection method to be described later, by calculating the standard deviation value (O_STD _{i, j} ) of the luminance data in each pixel of O data and the peripheral region of each pixel, It is possible to determine Tm, δt, and Cm optimized for. Even when these values are fixed values and the degree of occurrence of mosquito noise is simply estimated, the image quality can be improved (the mosquito noise can be reduced while leaving the detail portion).

As described above, in the second mosquito noise detection method, the final mosquito noise is obtained by taking the product of the first mosquito noise occurrence degree (T_W _{i, j} ) and the second mosquito noise occurrence degree (C_W _{i, j} ). The degree of noise generation (W _{i, j} ) is obtained. The significance of this can be explained as follows.

That is, the T data that is the first image data is image data that does not include information related to the edge region, but may include high-frequency components such as mosquito noise and details other than mosquito noise. Therefore, as described in the section of the first mosquito noise detection method, the occurrence of the first mosquito noise is calculated using only the first standard deviation value of the first image data, so that it is not affected by the edge. It is possible to discriminate mosquito noise. On the other hand, since the T data is data that can include a detail portion in addition to the mosquito noise as described above, the possibility that the detail portion affects the determination of the mosquito noise cannot be denied.

On the other hand, the C data as the second image data includes information related to the edge region. By calculating the second standard deviation value of the second image data in the standard deviation calculation unit 22, it is possible to determine whether the pixel is near or away from the edge portion. Therefore, the mosquito noise calculation unit 23 can obtain a more appropriate degree of mosquito noise occurrence.

More specifically, when C_STD _{i, j} that is the second standard deviation value of a certain pixel P _{i, j} is approximately the same as Pc, it is determined that P _{i, j} is a pixel in the vicinity of the edge region. Can do. In this case, since the value | C_STD _{i, j} −Pc | assigned to the second function G (x) is a small value, from the shape of the function G (x), C_W _{i, j} . It can be seen that a value of 1 is obtained. As a result, as mosquito noise occurrence rate, _{T_W i, j} and _{C_W i,} take the product of _j, so _{that T_W i,} the value comparable to the _j obtained.

That is, if it is determined from C_STD _{i, j} that P _{i, j} is close to the edge portion, it is taken into the mosquito noise calculation unit 23 without affecting the first standard deviation value, and does not include the edge region. This is the same value as when the degree of occurrence of mosquito noise is calculated using only T_W _{i, j} obtained from

On the other hand, when C_STD _{i, j} is a value significantly different from Pc, it can be determined that P _{i, j} is a pixel far from the edge region. In this case, since the value | C_STD _{i, j} −Pc | substituted for G (x) is a large value, from the shape of G (x) _, the value of C_W _{i, j} is a small value almost close to 0. I understand that Therefore, C_W _{i, j} acts in the mosquito noise calculation unit 23 as a correction term for further reducing T_W _{i, j} obtained from T_STD _{i, j of the} T data that is not affected by the edge. It is known that mosquito noise is likely to occur in the vicinity of the edge region due to encoding characteristics. In other words, it is considered that mosquito noise is unlikely to occur at a location away from the edge region, and it can be seen that C_W _{i, j} functions reasonably as the correction term.

As described above, the second mosquito noise detection method allows more appropriate discrimination of mosquito noise by calculating the degree of occurrence of mosquito noise using C_STD _{i, j} in addition to T_STD _{i, j.} .

The mosquito noise calculation unit 23 outputs the calculated degree of mosquito generation to the mosquito noise reduction unit 30. Subsequent image data processing in the mosquito noise reduction unit 30 is performed in the same manner as in the first mosquito noise detection method.

[6. How to set various parameters using O data]
Here, a method of performing mosquito noise detection using T data, C data, and O data, that is, a third mosquito noise detection method will be described. The purpose of using the O data for mosquito noise detection is to optimize the first for each pixel by calculating the standard deviation value (O_STD _{i, j} ) of the luminance data in each pixel of the O data and the peripheral area of each pixel. And determining each parameter (δt, Tm, and Cm) in the function and the second function.

In the third mosquito noise detection method, the data separation unit 10 and the mosquito noise reduction unit 30 have the same configurations as those in the first and second mosquito noise detection methods. Therefore, here, the function of the mosquito detection unit 20 in the third mosquito noise detection method will be described.

First, the luminance calculation unit 21 receives three data of T data, C data, and O data from the data separation unit 10 (see FIG. 6). In the brightness value calculation unit 21, the luminance value of T data _{(T_Y i, j)} and, like the luminance value of the C data _{(C_Y i, j),} calculates the luminance value of O data _{(O_Y i, j).} In other words, the O data of R, G, and B of the sub-pixels of P _{i, j} are set to O_R _{i, j} , O_G _{i, j} , and O_B _{i, j} , respectively. _{i, j} are calculated.
O_Y _{i, j} = 0.213 × O_R _{i, j} + 0.715 × O_G _{i, j} + 0.072 × O_B _{i, j}
The luminance calculation unit 21 outputs the three luminance values T_Y _{i, j} , C_Y _{i, j} and O_Y _{i, j} calculated in this way to the standard deviation value calculation unit 22 which is a standard deviation value calculation unit.

The standard deviation value calculation unit 22 that has received T_Y _{i, j} , C_Y _{i, j} , and O_Y _{i, j} receives T_STD _{i, j} , C_STD _{i, j} , and O_STD _{i, j} that are standard deviation values for each pixel. calculate. Here, T_std _i, the method for calculating the _j are, T_std _i in the first mosquito noise detection method _is the same as the method of calculating the _{j. Further,} C_STD _{i, j} and O_STD _i, the method for calculating the _j are, _{c_y i, j} and _{O_Y i,} T_STD _i with _j, may be calculated as with _j. Mosquito, which is a mosquito calculating means, using T_STD _{i, j} , C_STD _{i, j} and O_STD _{i, j} obtained by the above method as a first standard deviation value, a second standard deviation value, and a third standard deviation value, respectively. Output to the noise calculation unit 23.

_{Next, T_STD i, j, C_STD i} , j, and O_STD _i, mosquito calculator 23 having received the _j are, T_std _{i, j, and,} C_STD _i, from _j, the first mosquito noise occurrence rate (T_W _{i, j} ) and the second mosquito noise occurrence degree (C_W _{i, j} ) are calculated, and W _{i, j} is calculated by taking the product of the respective values. This series of image data processing is the same as the second mosquito noise detection method.

Then, the difference from the second mosquito noise detection method is that δt, Tm, which are parameters used when determining the shape of F (x) as the first function and G (x) as the second function, and This is a method for determining Cm. That is, in the second mosquito noise detection method, δt, Tm, and Cm are treated as fixed values. This is because even when the image data processing of the present invention is performed with each parameter as a fixed value, the detail portion included in the input image data is retained, mosquito noise is effectively reduced, and the image quality is appropriately improved. This is because the effect of obtaining image data can be sufficiently obtained.

However, each parameter can be optimized for each frame by determining δt, Tm, and Cm using the third standard deviation value O_STD _{i, j} in the mosquito calculating unit 23 that is a mosquito calculating means. This makes it possible to obtain a more precise degree of mosquito generation. Hereinafter, a method of dynamically determining δt, Tm, and Cm using T_STD _{i, j} , C_STD _{i, j} , and O_STD _{i, j} will be described.

First, δt can be calculated from the following equation.
δt = | Pt−Pf | × A + B
Here, the definition of Pf is as follows. In the O_STD histogram shown in FIG. 9C, the peak having the smallest O_STD among a plurality of peaks that can be confirmed is defined as Pf.

A and B are empirically obtained constants, and are preferably 0.5 ≦ A ≦ 2 and 0 ≦ B ≦ 1. Further, A to 1 and B to 0.5 are desirable. This value is obtained from past experimental results and may change depending on the target video.

Also, Tm and Cm can be calculated from the following equations.

Tm = g (| T_AVE_STD−Pt |)
g (x) = 2x + 1.0
Here, T_AVE_STD indicates an average value of T_STDi, j in the previous frame.

Cm = f (| Pc−Pn |)
f (x) = x / γ−0.016 (when x / γ> 0.516)
f (x) = 0.5 (when x / γ ≦ 0.516)
Here, among the plurality of peaks that can be confirmed in the O_STD histogram, the peak closest to Pc that is the peak of C_STD (see FIG. 9B) is defined as Pn. Further, γ is a constant, and γ˜0.5 is desirable based on experience so far.

The parameters δt, Tm, and Cm described above change the shapes of the first function and the second function that determine the first and second mosquito noise generation degrees. Each parameter O_STD i, by determining for each frame the results of _j, it is possible to calculate more appropriate mosquito noise occurrence rate than the second mosquito noise detection method.

Here, the method of determining δt, Tm, and Cm based on O data for each frame based on the second mosquito noise detection method using T data and C data has been described. However, the second method uses only T data. A method of determining δt and Tm based on O data for each frame based on the one mosquito noise detection method is also possible.

Here, the effect of the mosquito noise detection method according to an aspect of the present invention will be described with reference to FIGS. 12 (a) to 12 (d).

(A) of FIG. 12 is a figure which shows an example of the decoded image. Then, the image shown in FIG. 12A is output as input image data (O data) to the image data processing apparatus 1 (FIG. 2, etc.), and the first mosquito noise is detected by the mosquito noise detector 20 (FIG. 6). Assume that the detection method is executed. FIG. 12B shows output image data obtained in this case.

In addition, the image shown in FIG. 12A is output as input image data (O data) to the image data processing apparatus 1 (FIG. 2, etc.), and the third mosquito noise is detected by the mosquito noise detector 20 (FIG. 6). Assume that the detection method is executed. FIG. 12C shows output image data obtained in this case.

Furthermore, an output image when the conventional mosquito noise detection process is performed on the image shown in FIG. 12A using a bilateral filter is shown in FIG.

As can be seen by comparing FIG. 12B and FIG. 12C with FIG. 12D, the images of FIG. 12B and FIG. In the image of (a), the details of the grassland image surrounded by the dashed ellipse are held more clearly than the image of (d) in FIG. Further, in the images of FIG. 12B and FIG. 12C, the mosquito noise related to the image near the back of the zebra surrounded by the solid ellipse in the image of FIG. It is reduced more than the image of (d).

Further, as can be seen by comparing the image of FIG. 12B and the image of FIG. 12C, the image of FIG. 12C is more than the image of FIG. 12B. The details relating to the grassland image are clearly preserved, and the mosquito noise relating to the image near the back of the zebra is also reduced.

As described above, in one aspect of the present invention, the detail of the mosquito noise detection method is higher than that of the conventional method regardless of whether the first mosquito noise detection method or the third mosquito noise detection method is used. Good results can be obtained both in holding and in reducing mosquito noise. Furthermore, the use of the third mosquito noise detection method can maintain clear details and can effectively reduce the mosquito noise, compared to the case of using the first mosquito noise detection method.

However, in the third mosquito noise detection method, T data, C data, and O data are used for mosquito noise detection, while in the first mosquito noise detection method, only T data is used for mosquito noise detection. That is, the first mosquito noise detection method has an advantage that the amount of calculation is smaller than that of the third mosquito noise detection method. In this regard, in one aspect of the present invention, which of the first to third mosquito noise detection methods is used as the mosquito noise detection method may be appropriately selected according to the allowable calculation amount and image accuracy.

[7. (Summary)
An image data processing apparatus for reducing mosquito noise generated during reproduction of decoded input image data has been described using the digital television apparatus 100 as an example. An object of the image data processing apparatus of the present invention is to realize output image data in which mosquito noise is effectively reduced while retaining detail portions included in input image data.

Therefore, in one aspect of the present invention, first, the input image data is separated into the first image data, the second image data, and the third image data by the data separation means. Here, the first image data is image data not including an edge region, the second image data is image data including an edge region, and the third image data is the same image data as the input image data.

Next, the standard deviation value for each pixel is calculated by the standard deviation value calculating means. An important approach of the present invention is to perform image data processing based on the first image data that does not include the edge region. is there. By performing image data processing using the first image data, it is possible to calculate the degree of occurrence of mosquito noise for each pixel by the mosquito noise calculation means without being affected by the edge region having a large luminance difference. .

Using the degree of occurrence of mosquito noise obtained in this way, the second image data appropriately weighted for each pixel and the third image data are synthesized to obtain output image data. As a result, it is possible to appropriately set the balance between maintaining the detail portion included in the input image data and the effect of reducing the mosquito noise for each pixel based on the degree of occurrence of the mosquito noise. Improved output image data can be obtained.

The mosquito noise calculation means calculates the degree of mosquito noise occurrence in consideration of the second standard deviation value obtained from the second image data in addition to the first standard deviation value obtained from the first image data. It is possible to further improve the image quality.

Furthermore, in the mosquito calculating means, each parameter of the first function and the second function can be determined for each frame using the third standard deviation value obtained from the third image data. Thus, the degree of mosquito noise occurrence can be calculated using the first function and the second function optimized for each frame, and it is possible to further improve the image quality.

Furthermore, in the above description, the image data processing apparatus according to one embodiment of the present invention has been mainly described. However, the present invention can also be expressed as an image data processing method including steps for executing the same processing as each block in the image data processing apparatus.

In addition, although one Embodiment of this invention has demonstrated taking the digital television apparatus 100 as an example, it can utilize also in the set top box (STB) in a cable television etc. By providing the image data processing apparatus 1 of the present invention behind the decoder in the STB, it is possible to obtain output image data in which the image quality is appropriately improved.

It can also be used in a personal computer (PC) shown in FIG. Image data streamed over the Internet, image data downloaded from the Internet and stored in a recording medium 121 such as a PC memory or HDD, etc. are generally encoded and need to be decoded during playback. is there. Therefore, there is a high possibility that mosquito noise is included in the output image data. Therefore, by providing the image data processing device 1 of the present invention between the decoder 122 and the display application 123, it is possible to obtain output image data in which image quality has been improved appropriately.

In recent years, videophone devices using PCs and smartphones have become common. These videophone devices also transmit and receive encoded image data to reduce the data capacity, and display the decoded image data at each terminal. Therefore, also in the videophone device, by providing the image data processing device 1 of the present invention between the decoder and the display device, it is possible to obtain output image data in which the image quality is appropriately improved.

[8. Supplement)
Finally, each block (data separation unit 10, mosquito noise detection unit 20, mosquito noise reduction unit 30) of the image data processing apparatus 1 may be configured by hardware logic, or a CPU (central processing) as follows. unit), and may be realized by software.

That is, the image data processing apparatus 1 includes a CPU that executes instructions of a control program that realizes each function, a ROM (read only memory) that stores the program, a RAM (random access memory) that develops the program, the program, A storage device (recording medium) such as a memory for storing various data is provided. An object of the present invention is to supply a computer with a recording medium in which a program code (execution format program, intermediate code program, source program) of an image data processing program, which is software for realizing the functions described above, is recorded so as to be readable by the computer. However, this can also be achieved by reading and executing the program code recorded on the recording medium by the computer (or CPU or MPU).

Examples of the recording medium include a tape system such as a magnetic tape and a cassette tape, a magnetic disk such as a floppy (registered trademark) disk / hard disk, and a compact disk-ROM / MO / MD / digital video disk / compact disk-R. A disk system including an optical disk, a card system such as an IC card (including a memory card) / optical card, or a semiconductor memory system such as a mask ROM / EPROM / EEPROM / flash ROM can be used.

Further, the image data processing apparatus 1 may be configured to be connectable to a communication network, and the program code may be supplied via the communication network. The communication network is not particularly limited. For example, the Internet, intranet, extranet, LAN, ISDN, VAN, CATV communication network, virtual private network, telephone line network, mobile communication network, satellite communication. A net or the like is available. Also, the transmission medium constituting the communication network is not particularly limited. For example, even in the case of wired such as IEEE 1394, USB, power line carrier, cable TV line, telephone line, ADSL line, etc., infrared rays such as IrDA and remote control, Bluetooth ( (Registered trademark), 802.11 wireless, HDR, mobile phone network, satellite line, terrestrial digital network, and the like can also be used. Note that one aspect of the present invention can also be realized in the form of a computer data signal embedded in a carrier wave in which the program code is embodied by electronic transmission.

That is, the technical scope of the present invention includes an image data processing program that causes a computer to operate as each block of the image data processing apparatus 1 of the present embodiment, and a computer-readable recording medium that records the image data processing program. .

According to the above control program, the image data processing device 1 can be realized on a computer by realizing the above-described units with a computer. Further, according to the recording medium, the image data processing program read from the recording medium can be realized on a general-purpose computer.

Furthermore, in the image data processing apparatus according to an embodiment of the present invention, the standard deviation value calculating unit calculates the second standard deviation value of the luminance data in each pixel included in the second image data and the peripheral area of the pixel. It is calculated as a standard deviation value, and it is preferable that the mosquito noise calculating means calculates the degree of occurrence of mosquito noise using at least the first standard deviation value and the second standard deviation value.

That is, the first image data does not include the edge region of the input image data, but includes mosquito noise and details other than mosquito noise. As described above, it is possible to determine mosquito noise that is not affected by an edge by calculating the degree of occurrence of mosquito noise using only the first standard deviation value of the first image data. However, the first image data includes details in addition to mosquito noise, and it cannot be denied that the details may affect the determination of mosquito noise.

On the other hand, the second image data includes an edge region. In the standard deviation calculation means, by calculating the second standard deviation value of the second image data, it is possible to determine whether the pixel is close to or away from the edge portion.

When it is determined from the second standard deviation value that the pixel is close to the edge portion, the pixel is taken into the mosquito noise calculation means without affecting the first standard deviation value. That is, the value is the same as when the degree of occurrence of mosquito noise is calculated using only the first standard deviation value of the first image data not including the edge region.

On the other hand, if it is determined from the second standard deviation value that the pixel is far from the edge portion, it is determined that the possibility of including mosquito noise is low. Therefore, the second standard deviation value acts in the mosquito noise calculation means as a correction term for further reducing the value obtained by calculating the degree of occurrence of mosquito noise using only the first standard deviation value not affected by the edge.

As described above, by calculating the degree of occurrence of mosquito noise using the second standard deviation value in addition to the first standard deviation value, it is possible to determine mosquito noise strictly.

Further, in the image data processing device according to one embodiment of the present invention, the standard deviation value calculating means calculates the third standard value of the luminance data in each pixel included in the input image data and the peripheral area of the pixel. It is preferably calculated as a deviation value, and the mosquito noise calculation means preferably calculates the degree of occurrence of mosquito noise using at least the first standard deviation value and the third standard deviation value.

According to the above configuration, in the standard deviation value calculation means, in addition to the first standard deviation value and the second standard deviation value, the standard deviation value of the luminance data in each pixel of the input image data and the peripheral area of the pixel is added. Is calculated as the third standard deviation value. In the mosquito noise calculating means, the third standard deviation value is taken into account in both cases of using the first standard deviation value and using the first standard deviation value and the second standard deviation value. By calculating the degree of occurrence of mosquito noise, it becomes possible to determine the mosquito noise more strictly.

Furthermore, in the image data processing apparatus according to an embodiment of the present invention, the mosquito noise calculating means calculates the first mosquito noise occurrence degree by substituting the first standard deviation value into a predetermined first function, A second mosquito noise occurrence degree is calculated by substituting the second standard deviation value into a predetermined second function, wherein the first function is an absolute value of the first standard deviation value as an input value. This is a function that shows a maximum value as an output value in the case of the minimum, the output value monotonously decreases as the input value increases, and further the output value becomes 0 when the input value is the first predetermined value. When the absolute value of the second standard deviation value as the input value is minimum, the second function shows a maximum value as the output value, and the calculated value decreases monotonously as the input value increases. ,input Is a function whose output value is 0 when the second predetermined value is, the first predetermined value is determined using the first standard deviation value and the third standard deviation value, It is preferable that the second predetermined value is determined using the second standard deviation value and the third standard deviation value.

According to the above configuration, when calculating the degree of occurrence of mosquito noise in the mosquito noise calculation means, only the first function, or the first function and the second function are used. The first function and the second function include the first predetermined value and the second predetermined value as parameters. The first standard deviation value and the third standard deviation value are used to determine the first predetermined value, and the second standard deviation value and the third standard deviation value are used to determine the second predetermined value. By using this, it is possible to determine a more exacting degree of mosquito noise.

Furthermore, the image data processing apparatus according to an embodiment of the present invention provides, for each frame, the average value of the degree of occurrence of mosquito noise calculated by the mosquito noise calculation unit in the input image data as the average value of mosquito noise occurrence degree. In addition to the calculation, the magnitude relation between the mosquito noise occurrence average value and a predetermined threshold value is compared, and the frame in which the mosquito noise occurrence degree average value is determined to be equal to or greater than the threshold value includes mosquito noise in the frame. While determining that the ratio is high, the frame for which the average mosquito noise occurrence degree is determined to be less than the threshold value includes mosquito noise determination means for determining that the ratio of mosquito noise to the frame is small. preferable.

According to the above configuration, the mosquito noise calculation means calculates an average value of the degree of occurrence of the mosquito noise calculated for each pixel for all pixels of one frame of the input image data. If the average value is equal to or greater than a predetermined threshold, the mosquito noise determination means determines that the frame contains a high proportion of mosquito noise, and needs to perform mosquito noise reduction processing described later on the input image data. Judge that there is.

On the other hand, when the average value is less than the predetermined threshold, it is determined that the ratio of the mosquito noise is small in the frame, and it is determined that it is not necessary to perform the mosquito noise reduction process on the input image data.

It is possible to arbitrarily change the judgment standard of the mosquito noise judging means by changing the predetermined threshold value. This makes it possible to flexibly cope with the level of processing capability of the image data processing apparatus and the level of compression rate during encoding.

Furthermore, the image data processing apparatus according to an embodiment of the present invention uses the degree of mosquito noise generation for the frame that is determined by the mosquito noise determination means to have a high ratio of mosquito noise. It is preferable to provide a mosquito noise reduction unit that outputs the output image data by combining the input image data and the second image data after weighting the image data and the second image data. .

According to the above configuration, when the mosquito noise determination unit determines that the mosquito noise reduction process needs to be performed, the mosquito noise reduction unit determines that the input image data and the first image for each pixel of the input image data. Mosquito noise reduction processing is performed using the two image data and output as output image data.

The second image data is image data obtained by processing input image data with a low-pass filter (LPF) or the like. The mosquito noise is reduced, and at the same time, fine image data other than mosquito noise (detail portion). Is also lost image data.

Based on the degree of mosquito noise occurrence calculated by the mosquito calculating means, the input image data and the second image data are weighted, and the input image data and the second image data are combined to produce an output image. Output data.

If the mosquito noise determination means determines that there is no need to perform mosquito noise reduction processing, the input image data is set as output image data.

As a result, it is possible to effectively remove mosquito noise while retaining the detail portion included in the image data, and to obtain output image data with improved image quality appropriately.

According to the present invention, it is possible to perform mosquito noise removal processing that is not affected by edges. Therefore, for example, it can be applied when decoding and playing back encoded digital video data such as terrestrial digital broadcasting, cable TV, video distributed over a network, and videophones using a personal computer or smartphone as a terminal. .

1 Image Data Processing Device 10 Data Separation Unit (Data Separation Unit)
22 Standard deviation value calculation unit (standard deviation value calculation means)
23 Mosquito noise calculation unit (mosquito noise calculation means)
30 Mosquito noise reduction unit (Mosquito noise reduction means)
31 Mosquito noise judging section (mosquito noise judging means)

Claims (9)

Data separation means for separating the input image data into first image data not including an edge region and second image data other than the first image data;
A standard deviation value calculating means for calculating a standard deviation value of luminance data in each pixel included in the first image data and a peripheral area of the pixel as a first standard deviation value;
Mosquito noise calculating means for calculating the degree of occurrence of mosquito noise for each pixel of the input image data using the first standard deviation value;
An image data processing apparatus comprising: The standard deviation value calculating means further calculates a standard deviation value of luminance data in each pixel included in the second image data and a peripheral area of the pixel as a second standard deviation value.
2. The image data processing apparatus according to claim 1, wherein the mosquito noise calculation means calculates a degree of occurrence of mosquito noise using at least the first standard deviation value and the second standard deviation value. . The standard deviation value calculating means further calculates a standard deviation value of luminance data in each pixel included in the input image data and a peripheral area of the pixel as a third standard deviation value,
2. The image data processing apparatus according to claim 1, wherein the mosquito noise calculating means calculates a degree of occurrence of mosquito noise using at least the first standard deviation value and the third standard deviation value. . The mosquito noise calculation means further calculates the first mosquito noise occurrence degree by substituting the first standard deviation value into a predetermined first function, while each pixel included in the second image data and the Substituting a second standard deviation value, which is a standard deviation value of luminance data in the peripheral area of the pixel, into a predetermined second function, to calculate the second mosquito noise occurrence degree,
The first function shows a maximum value as the output value when the absolute value of the first standard deviation value as the input value is minimum, and the output value monotonously decreases as the input value increases. When the input value is the first predetermined value, the output value is 0.
The second function shows a maximum value as an output value when the absolute value of the second standard deviation value as an input value is minimum, and the calculated value monotonously decreases as the input value increases. When the input value is the second predetermined value, the output value is 0.
The first predetermined value is determined using the first standard deviation value and the third standard deviation value,
4. The image data processing apparatus according to claim 3, wherein the second predetermined value is determined by using the second standard deviation value and the third standard deviation value. In the input image data, an average value of the mosquito noise occurrence degree calculated by the mosquito noise calculation means is calculated for each frame as a mosquito noise occurrence degree average value, and the mosquito noise occurrence degree average value and a predetermined threshold value are calculated. Compare the magnitude relationship with
While determining that the mosquito noise occurrence average value is equal to or higher than the threshold value, the frame is determined that the ratio of mosquito noise included in the frame is high.
5. The mosquito noise determining means for determining that a frame in which the average mosquito noise occurrence value is determined to be less than the threshold includes a low ratio of mosquito noise in the frame. The image data processing device according to any one of the above. For frames determined to have a high ratio of mosquito noise by the mosquito noise determination means, the input image data and the second image data are weighted using the degree of mosquito noise occurrence. 6. The image data processing apparatus according to claim 5, further comprising mosquito noise reduction means for outputting output image data by combining the input image data and the second image data. A data separation step of separating the input image data into first image data not including an edge region and second image data other than the first image data;
A standard deviation value calculating step of calculating, as a first standard deviation value, a standard deviation value of luminance data in each pixel included in the first image data and a peripheral region of the pixel;
A mosquito noise calculating step for calculating a degree of occurrence of mosquito noise for each pixel of the input image data using the first standard deviation value;
An image data processing method comprising: An image data processing program for operating a computer as each means in the image data processing apparatus according to claim 1. A computer-readable recording medium on which the image data processing program according to claim 8 is recorded.

Images