JP2011029704A - Image processing unit, method of processing image, and imaging apparatus - Google Patents

Abstract

An image processing apparatus, an image processing method, and an imaging apparatus capable of achieving both noise removal performance and storage of an original image with high accuracy are provided.
An image processing apparatus according to the present invention includes a sorting unit that sorts pixel values of a plurality of pixels included in a predetermined area with respect to image data acquired from pixels arranged on an imaging surface, and a sorting unit. A noise determination unit that determines whether or not the pixel value of the specific pixel includes noise based on the sorting order of the pixel value of the specific pixel included in the predetermined region among the plurality of pixel values sorted by 106 108.
[Selection] Figure 1

Description

  The present invention relates to an image processing device, an image processing method, and an imaging device.

  Recently, with the widespread use of imaging devices such as digital cameras, photographing in various scenes has been performed, and there has been a need to restore digital images photographed under more adverse conditions. Under such circumstances, as described in Patent Documents 1 to 4 below, a technique for reducing noise by performing filtering processing on image data is known.

Japanese Patent Laid-Open No. 5-22637 JP 7-236074 A JP-A-10-43140 JP 2004-318356 A

  Image data captured by an imaging device such as a digital camera has a lot of noise because, for example, the amount of light input to the camera is small at low illumination such as at night. In particular, at low illuminance, the noise amplitude is large, and many impulsive noises that appear in isolation are generated. For example, assuming a night-time monitoring camera, there arises a problem that it becomes impossible to determine the face of a person to be monitored or the number of a car due to the influence of noise.

  Therefore, a means for removing noise from an image having noise is required. This is a technique generally called noise reduction. While the purpose of noise reduction is to remove noise, there is a desire to preserve the detailed (fine) portion of the original image (image without noise) as much as possible. For example, when a license plate of a car is photographed with a surveillance camera, even if the noise of the license plate image can be removed by noise reduction, it is not preferable that characters are crushed and cannot be read. Therefore, when performing noise reduction, it is to remove more noise from a deteriorated image having noise in a state in which the original image is kept as fine as possible.

  As the simplest method of noise reduction, there is a method using a smoothing filter (low-pass filter). In the processing by the smoothing filter, information in a high frequency band containing a lot of noise components is cut and noise is removed. However, for impulsive noise, it is difficult to achieve noise removal with a linear filter such as a smoothing filter. This is because, when viewed in the frequency domain, impulsive noise has equal components in the entire frequency band.

  On the other hand, as a noise reduction technique for impulsive noise, there is a method using a median filter as described in the above patent document.

  Since the median filter is a non-linear filter and is not a frequency selective filter process like a linear filter, impulsive noise can be removed under certain conditions. However, when a non-stationary part such as an edge of an image is included in the region, there is a possibility that the detailed part of the image cannot be stored by the median filter. In addition, in order to save a detailed portion in an image, when the central pixel is not impulsive noise, it is desirable that the pixel value of the central pixel before and after the filtering process does not change. Since the filter causes a change in the pixel value of the center pixel, it is not possible to save the detailed portion in the image.

    Therefore, the present invention has been made in view of the above problems, and an object of the present invention is a novel and improved technique capable of achieving both noise removal performance and original image storage with high accuracy. Another object of the present invention is to provide an image processing apparatus, an image processing method, and an imaging apparatus.

  In order to solve the above-described problem, according to an aspect of the present invention, a sorting process for sorting pixel values of a plurality of pixels included in a predetermined region with respect to image data acquired from pixels arranged on an imaging surface A pixel value of the specific pixel based on a sorting order of pixel values of the specific pixel included in the predetermined area in the plurality of pixel values sorted by the sorting unit and the sorting processing unit There is provided an image processing apparatus including a noise determination unit that determines whether or not.

  The noise determination unit compares the sorting order of the pixel values of the specific pixel with a preset threshold value, and determines whether noise is included based on the comparison result. May be.

  In addition, the noise determination unit, when the sorting order of the pixel values of the specific pixel is equal to or less than a first threshold value counted from the smaller pixel value, an impulse in the negative direction is added to the pixel value of the specific pixel. It may be determined that sexual noise is included.

  In addition, the noise determination unit, when the sorting order of the pixel values of the specific pixel is greater than or equal to a second threshold value counted from the smaller pixel value, adds a positive impulse to the pixel value of the specific pixel. It may be determined that sexual noise is included.

  In addition, when the noise determination unit determines that the pixel value of the specific pixel includes noise, the noise determination unit further includes a noise removal unit that removes noise from the pixel value of the specific pixel, The difference between the pixel value of the specific pixel and the provisional pixel value that is one of the plurality of pixel values, and the difference between the provisional pixel value and the pixel value adjacent to the provisional pixel value in the sort order And noise may be removed from the pixel value of the specific pixel based on the result of the comparison.

  In addition, when it is determined that the pixel value of the specific pixel includes a negative-direction impulsive noise, the noise removing unit sequentially sets the provisional pixel value from the smaller side of the plurality of pixel values. The comparison is performed every time the provisional pixel value is set, and the difference between the pixel value of the specific pixel and the provisional pixel value is adjacent to the provisional pixel value and the provisional pixel value in the sort order. When the difference from the value becomes larger, noise may be removed using the pixel value of the specific pixel as the provisional pixel value.

  In addition, when it is determined that the pixel value of the specific pixel includes an impulsive noise in the positive direction, the noise removing unit sequentially sets the provisional pixel value from the side with the larger plurality of pixel values. The comparison is performed every time the provisional pixel value is set, and the difference between the pixel value of the specific pixel and the provisional pixel value is adjacent to the provisional pixel value and the provisional pixel value in the sort order. When the difference from the value becomes larger, noise may be removed using the pixel value of the specific pixel as the provisional pixel value.

  In addition, the noise removing unit does not perform a process of removing noise when the provisional pixel value is counted from a smaller pixel value and becomes a pixel value larger than the first threshold value. The pixel value of the specific pixel may be output as it is.

  Further, the noise removal unit does not perform a process of removing noise when the provisional pixel value is counted from a smaller pixel value and becomes a pixel value smaller than a second threshold value. The original pixel value of the specific pixel may be output as it is.

  Further, in the comparison, the comparison may be performed using a predetermined value serving as a buffer.

  In order to solve the above problems, according to another aspect of the present invention, pixel values of a plurality of pixels included in a predetermined region are sorted for image data acquired from pixels arranged on an imaging surface. And whether or not noise is included in the pixel value of the specific pixel based on a sorting order of the pixel values of the specific pixel included in the predetermined area in the plurality of sorted pixel values. An image processing method comprising: determining.

  In the step of determining whether or not the noise is included, the sorting order of the pixel values of the specific pixel is compared with a preset threshold value, and the noise is included based on the comparison result. It may be determined whether or not there is.

  A step of removing noise from the pixel value of the specific pixel when it is determined that the pixel value of the specific pixel includes noise; and in the step of removing the noise, the pixel of the specific pixel A result of comparing a difference between a value and a provisional pixel value that is one of the plurality of pixel values, and a difference between the provisional pixel value and a pixel value adjacent to the provisional pixel value in a sort order Based on the above, noise may be removed from the pixel value of the specific pixel.

  In order to solve the above problem, according to another aspect of the present invention, a lens optical system that forms a subject image, and a plurality of pixels on which the subject image is formed by the lens optical system, An image pickup device that acquires image data of a formed subject image, a sort processing unit that sorts pixel values of a plurality of pixels included in a predetermined area, and the image processing data sorted by the sort processing unit A noise determination unit configured to determine whether or not the pixel value of the specific pixel includes noise based on a sorting order of pixel values of the specific pixel included in the predetermined region in a plurality of pixel values; An imaging device is provided.

  In addition, when the noise determination unit determines that the pixel value of the specific pixel includes noise, the noise determination unit further includes a noise removal unit that removes noise from the pixel value of the specific pixel, The difference between the pixel value of the specific pixel and the provisional pixel value that is one of the plurality of pixel values, and the difference between the provisional pixel value and the pixel value adjacent to the provisional pixel value in the sort order And noise may be removed from the pixel value of the specific pixel based on the result of the comparison.

  According to the present invention, it is possible to provide an image processing device, an image processing method, and an imaging device capable of achieving both noise removal performance and original image storage with high accuracy.

1 is a schematic diagram illustrating a configuration of an image processing apparatus according to each embodiment of the present invention. It is a schematic diagram which shows an example of a structure of the image processing apparatus which processes a color image. It is a flowchart which shows the whole flow of the process in an image processing apparatus. It is a schematic diagram which shows an example of the result of having sorted. It is a flowchart which shows the process of FIG.3 S16 in detail. FIG. 6 is a schematic diagram illustrating a state in which a temporary output pixel value is changed by decreasing the value of k by 1 when a positive impulsive noise is generated. It is a schematic diagram which shows the evaluation method of the value of Jp using temporary output pixel value x_ (k) in step S36 of FIG. FIG. 6 is a schematic diagram illustrating a case where an impulsive noise in the + direction is added only to the center pixel when pixel values are uniform within a region. It is a schematic diagram showing a case where an impulsive noise in the plus (+) direction is added to two pixels including a central pixel in a region where pixel values are uniform. It is a schematic diagram which shows the case where an edge exists in an area | region and impulsive noise is not added to the pixel value in an area | region. It is a schematic diagram which shows the operation example of a median filter. FIG. 10 is a schematic diagram illustrating a case where a plurality of impulsive noises are generated side by side in a region when noise is removed by a filter using a maximum / minimum replacement method.

  Exemplary embodiments of the present invention will be described below in detail with reference to the accompanying drawings. In addition, in this specification and drawing, about the component which has the substantially same function structure, duplication description is abbreviate | omitted by attaching | subjecting the same code | symbol.

The description will be made in the following order.
1. First Embodiment (1) Configuration of Image Processing Device (2) Flow of Processing in Image Processing Device (3) Noise Removal Processing (4) Specific Example of Noise Removal and Original Image Storage (5) Other Filter Processing Comparison with 1. Second embodiment

<1. First Embodiment>
(1) Configuration of Image Processing Device FIG. 1 is a schematic diagram showing a configuration of an image processing device 100 according to each embodiment of the present invention. As illustrated in FIG. 1, the image processing apparatus 100 includes an imaging element 102 in which a plurality of pixels are arranged in a matrix, and pixel values (luminances) of each pixel included in a predetermined area among the pixels of the imaging element 102. Value), a sorting unit 106 that sorts the extracted pixel values of the predetermined region in order of luminance, a noise determination unit 108, and an adaptive rank order filter 110.

  As illustrated in FIG. 1, a pixel value of a pixel in a predetermined region is extracted by an extraction unit 104 among pixel values of each pixel acquired by the image sensor 102. More specifically, with an arbitrary pixel as a central pixel, pixel values of nine pixels including the central pixel and eight pixels around the central pixel are extracted as one unit and sent to the sorting unit 106. As a result, pixel values of 3 × 3 = 9 pixels are extracted and input to the sorting unit 106.

  Although the details will be described later, the sorting unit 106 sorts the pixel values in order of luminance based on the luminance magnitudes of the nine input pixel values. The noise determination unit 108 determines whether or not impulsive noise is included in the nine pixel values based on the result of sorting by the sorting unit 106. The adaptive rank order filter 110 performs a filtering process to remove the impulsive noise when the impulsive noise is included in the nine pixel values.

  FIG. 2 is a schematic diagram illustrating an example of the configuration of the image processing apparatus 100 that processes a color image. As shown in FIG. 2, the image processing apparatus 100 includes an image sensor 102 in which pixels of each color of RGB are arranged in a Bayer array, and the luminance value of each pixel of the image sensor 102 is input for each RGB and is sorted in order of luminance. A sorting unit 104, a noise determination unit 106, and an adaptive rank order filter 108.

  As shown in FIG. 2, the signal of each pixel acquired by the image sensor 102 is input to each sort unit 104R, 104G, 104B for each color of RGB. At this time, as shown in FIG. 2, a signal is sent to each sorting unit 104R, 104G, and 104B with 9 signals consisting of a signal of an arbitrary center pixel and signals of 8 pixels around the center pixel as one unit. Will be sent.

  Although the details will be described later, the sorting units 104R, 104G, and 104B sort the signals based on the input nine pixel signals. The noise determination units 106R, 106G, and 106B determine whether impulse noise is included in the nine signals based on the result of sorting by the sorting units 104R, 104G, and 104B. The adaptive rank order filters 108R, 108G, and 108B correct this when impulse noise is included.

  In addition to the configuration shown in FIG. 2, the image processing device that processes a color image may have the same configuration as the above-described image processing device in FIG. 1 for each color image. In this case, the pixel value of each pixel is a three-dimensional vector representing the luminance value of each color of RGB.

  In addition, the imaging apparatus according to the present embodiment includes the image processing apparatus 100 illustrated in FIG. 1 or FIG. 2 and a lens optical system, and forms a subject image on the imaging surface of the imaging element 102 by the lens optical system. .

(2) Process Flow in Image Processing Device FIG. 3 is a flowchart showing the overall flow of processing in the image processing device 100 shown in FIG. In the image processing apparatus 100 shown in FIG. 2, the processing of each color after sorting by the sorting units 104R, 104G, and 104B is performed in the same manner as the image processing apparatus 100 in FIG.

  First, in step S <b> 10, image data including pixel values of a plurality of pixels is input from the image sensor 102 to the sorting unit 106. Here, the image value has a structure having data of an arbitrary dimension per pixel. For example, a monochrome multi-valued image has a structure having one-dimensional data per pixel, and a color image has a structure having three-dimensional data per pixel. Further, in the case of a color image, the configuration in which pixel signals are input to the sorting units 104R, 104G, and 104B for each color of RGB as shown in FIG. 2 has a structure having one-dimensional data per pixel.

  Next, in step S12, an arbitrary area including the center pixel is set and sorted in ascending or descending order of the norm value of the vector of image values (hereinafter simply referred to as pixel value). As described above, in this embodiment, the norm values of the image vectors are sorted from the area of nine pixels including the center pixel. In the following description, it is assumed that ascending sort is performed.

  FIG. 4 is a schematic diagram illustrating an example of the result of sorting. As shown in FIG. 4, the nine pixel values including the center pixel are arranged in the order of luminance. In FIG. 4, the pixel marked with ◆ represents the center pixel. In step S14, the rank of the central pixel (how much smaller the pixel value of the central pixel is in the region) is calculated from this result. The lowest rank is 0, and the rank is a value from 0 to 8. In the example of FIG. 4, the center pixel rank is “8”.

  Next, in step S16, noise removal processing is performed. In the noise removal process, first, it is determined whether or not impulsive noise in the plus (+) direction or minus (−) direction is added to the pixel value of the central pixel, and it is determined whether or not impulsive noise is added. If it does, remove it. Here, the impulsive noise in the plus (+) direction refers to, for example, a case where the pixel value of the central pixel is significantly larger than other pixel values. Moreover, the impulsive noise in the minus (−) direction refers to, for example, a case where the pixel value of the central pixel is significantly smaller than other pixel values.

  In determining whether or not impulsive noise is added, it is determined that impulsive noise in the minus (−) direction is not added to the central pixel when the order of the central pixel is higher than a preset threshold value m_peak_thre. . Further, when the order of the center pixel is smaller than a preset threshold value p_peak_thre, it is determined that the plus (+) direction impulsive noise is not added to the center pixel. Here, both m_peak_thre and p_peak_thre are positive integers that do not exceed the number of pixels in the region. m_peak_thre and p_peak_thre are parameters that specify the number of impulsive noises to be removed, and can be set to appropriate values according to the amount of noise generated.

  The example of FIG. 4 shows a case where m_peak_thre = 2 and p_peak_thre = 6. In this case, since the rank (= c) of the central pixel is 8 and c> p_peak_thre, it is determined that positive impulsive noise is added.

  If it is determined in step S16 that no negative (−) direction impulsive noise or positive (+) direction impulsive noise is added to the central pixel, noise removal is not performed. In this case, the pixel value of the original center pixel is output as it is in step S20. On the other hand, if it is determined that the negative (−) direction impulsive noise or the positive (+) direction impulsive noise is added to the central pixel, the noise is removed in step S16 to remove the noise. The pixel value of the center pixel is output in step S20.

  After the processing in step S20 is completed, the next nine pixel values are input to the sorting unit 106, and the same processing is performed. In this embodiment, the noise of the center pixel is removed, but the noise of pixel values other than the center pixel among the nine pixel values may be removed. The specific pixel can be any one of the pixels.

(3) Noise Removal Process FIG. 5 is a flowchart showing in detail the process of step S16. In the following description, the number of pixels in the region is k−1, and the pixel values sorted in the region are x_ (j) (x_ (0) ≦ x_ (1) ≦... X_ (k−1)), The order of the center pixel is c, and the output pixel value of the center pixel is y. The sorting is performed in ascending order, and the smaller the value of j, the smaller the pixel value.

  First, in step S20 of FIG. 5, it is determined whether c <m_peak_thre. When c <m_peak_thre, since the order c of the center pixel is smaller than the threshold value m_peak_thre, it is determined that there is a possibility that impulsive noise in the minus (−) direction is added to the center pixel. Proceed to the process of (1) to remove impulsive noise. When c ≧ m_peak_thre, the order c of the central pixel is equal to or higher than the threshold value m_peak_thre, so the process proceeds to step S30 to determine whether c> p_peak_thre. When c> p_peak_thre, since the order c of the central pixel is larger than the threshold value p_peak_thre, it is determined that there is a possibility that impulsive noise in the plus (+) direction is added to the central pixel. Proceed to the process of (1) to remove impulsive noise.

  First, the removal of the impulsive noise in the plus (+) direction will be described. In step S30, when c> p_peak_thre, k = c + 1 is set as the initial value of the provisional output pixel value order. Next, in step S32, k = k-1. Then, as described below, in the loop of steps S32 to S36, the provisional output pixel value x_ (k) is set while decreasing the value of k by 1, and the first k value that satisfies the evaluation function Jp <0 is set. Is calculated.

  In step S34, it is determined whether k <p_peak_thre. When k <p_peak_thre, since the rank of the provisional output pixel value is smaller than the threshold value p_peak_thre, it is determined that the plus (+) direction impulsive noise is not added to the center pixel, and the process proceeds to step S40.

On the other hand, if k ≧ p_peak_thre in step S34, the process proceeds to step S36. In step S36, the value of the evaluation function Jp is calculated by the following equation (1), and the value of Jp is evaluated.
Jp = || x_ (k) −x_ (k−1) || − || x_ (k) −x_ (c) || (1)

  Here, | · | is the norm of the pixel value vector. For example, an I1 norm or an I2 norm is used as the norm. In the case of a monochrome image, || x_ (k + 1) -x_ (k) || is represented as an absolute value of x_ (k + 1) -x_ (k). The same applies to || x_ (k) −x_ (c) ||.

  In step S36, if Jp <0, the process proceeds to step S38, and the output value is determined as y = x_ (k). On the other hand, when Jp ≧ 0, the process returns to step S32, and Jp is calculated again with k = k−1 for evaluation. By repeating this, if Jp satisfying Jp <0 is not obtained until k = p_peak_thre−1, it is determined that no impulsive noise is added to the central pixel, and the process proceeds to step S40. In step S40, the value of the center pixel (output value y) is set as the original pixel value x_ (c) of the center pixel, and y = x_ (c) is output in step S42. In other words, in this case, it is determined that no impulsive noise is added, and the center pixel value is output without being changed.

  If Jp satisfying Jp <0 is obtained before k = p_peak_thr−1, the process proceeds to step S38, and the value of the central pixel (output value y) is set as the provisional output pixel value x_ (k). In step S42, y = x_ (k) is output. Thereby, impulsive noise is removed from the center pixel value. The above is the process of removing the impulsive noise in the plus (+) direction.

  FIG. 6 shows how the provisional output pixel value is changed by decreasing the value of k by 1 when an impulsive noise in the positive direction is generated. In FIG. 6, the provisional output pixel value is indicated by ● and the center pixel is indicated by ◆ (same in each figure). As described above, k = c + 1 is set as the initial value of the provisional output pixel value order, and the value of k is sequentially decreased in the loop of steps S32 to S36 to evaluate Jp. FIG. 6 shows a case where the provisional output pixel value is x_ (7). Then, the temporary output pixel value is shifted in the loop of steps S32 to S36, and the temporary output pixel value when Jp <0 is set as the central pixel value.

  FIG. 7A and FIG. 7B are schematic diagrams showing a Jp value evaluation method using the provisional output pixel value x_ (k) in step S36. 7A and 7B show a case where the order of the central pixel is “8”, as in FIG. The processing in step S36 will be described in detail based on FIGS. 7A and 7B. If k = c + 1 is set as the initial value of the provisional output pixel value order, c = 8, so k = 9. It becomes. If k = k−1 in step S32, k = 8, and the provisional output pixel value x_ (k) is x_ (8). In this case, in the expression (1), || x_ (k) −x_ (k−1) || becomes || x_ (8) −x_ (7) ||, which is the eighth pixel value (x_ (8 )) And the seventh pixel value (x_ (7)) (x2 shown in FIG. 7A) is obtained. In the expression (1), || x_ (k) −x_ (c) || becomes || x_ (8) −x_ (8) || = 0. Therefore, when k = 8, Jp ≧ 0, and the process returns from step S36 to step S32 without exiting the loop.

  FIG. 7A shows a case where k = 7 when the value of k is decreased by 1 in step S32. In this case, the provisional output pixel value x_ (k) = x_ (7), and in the expression (1), || x_ (k) −x_ (k−1) || is || x_ (7) −x_ ( 6) ||, and x1 shown in FIG. 7A is obtained. || x_ (k) -x_ (c) || becomes || x_ (7) -x_ (8) ||, and x2 shown in FIG. 7A is obtained. In this case, since x1> x2, Jp ≧ 0, so that the process returns to step S32 without exiting the loop. In step S32, the value of k is further reduced by 1, and the value of Jp is evaluated again.

  FIG. 7B shows the case where k = 6 (when the provisional pixel value is x_ (6)). In this case, in the expression (1), || x_ (k) −x_ (k−1) || becomes || x_ (6) −x_ (5) ||, and x3 shown in FIG. can get. In the expression (1), || x_ (k) −x_ (c) || becomes || x_ (6) −x_ (8) ||, and x4 shown in FIG. 7B is obtained. In this case, since x3 <x4, Jp <0 is established, the process goes through the loop and proceeds to step S38, and the output y becomes the provisional pixel value x_ (6).

  According to the above processing, when impulsive noise is added to the center pixel, the value of k is decreased by 1 until Jp in equation (1) becomes a negative value, and the equation (1) Evaluate. Accordingly, as shown in FIGS. 7A and 7B, even when there are a plurality of impulsive noises in the region, it can be determined that the central pixel value includes impulsive noises, Impulsive noise can be reliably removed.

  In FIGS. 7A and 7B, it is considered that impulsive noise in the plus (+) direction is also added to the pixel having the rank “7”. Can be removed in the process of using the pixel having a pixel of “7” as the central pixel.

  Next, the removal of the impulsive noise in the minus (−) direction will be described. The method of removing the impulsive noise in the negative (−) direction is basically the same as the removal of the impulsive noise in the positive (+) direction.

  First, in step S20 of FIG. 5, when c <m_peak_thre, k = c + 1 is set as the initial value of the provisional output pixel value order. Next, in step S22, k = k + 1 is set. Then, as described below, in the loop of steps S22 to S26, the provisional output pixel value x_ (k) is set while increasing the value of k by 1, and the first k value that satisfies the evaluation function Jm <0 is set. Is calculated.

  In step S24, it is determined whether k> m_peak_thre. When k> m_peak_thre, the order of the provisional output pixel value is larger than the threshold value m_peak_thre, so it is determined that the impulsive noise in the minus (−) direction is not added to the central pixel, and the process proceeds to step S40.

On the other hand, if k ≦ m_peak_thre in step S24, the process proceeds to step S26. In step S26, the value of the evaluation function Jm is calculated by the following equation (2), and the value of Jm is evaluated.
Jm = || x_ (k + 1) −x_ (k) || − || x_ (k) −x_ (c) || (2)

  In step S26, when Jm <0, the process proceeds to step S28, and the output value is determined as y = x_ (k). On the other hand, if Jm ≧ 0, the process returns to step S22, and Jm is calculated again with k = k + 1 and evaluated. If this is repeated and Jm satisfying Jm <0 is not obtained until k = m_peak_thre + 1, it is determined that no impulsive noise is added to the central pixel, and the process proceeds to step S40. In step S40, the value of the center pixel (output value y) is set as the original pixel value x_ (c) of the center pixel, and y = x_ (c) is output in step S42. In other words, in this case, it is determined that no impulsive noise is added, and the center pixel value is output without being changed.

  If Jm satisfying Jm <0 is obtained before k = m_peak_thre + 1, the process proceeds to step S28, and the value of the central pixel (output value y) is set as the provisional output pixel value x_ (k). Y = x_ (k) is output. Thereby, impulsive noise is removed from the center pixel value. The above is the process of removing the impulsive noise in the minus (−) direction.

  In steps S20 and S30, if c ≧ m_peak_thre and c ≦ p_peak_thre, it is determined that no impulsive noise is added to the center pixel, and the process proceeds from step S30 to step S40 to output y = x_ (c). Value.

(4) Specific examples of noise removal and original image storage Next, several representative images will be described by taking the case of a monochrome multi-value image as an example of how impulsive noise removal is achieved by the processing described above. The case of the configuration will be described. FIG. 8 shows a case where an impulsive noise in the + direction is added only to the center pixel when the pixel values are uniform in the region. In this case, the impulsive noise can be removed most easily. The pixel values in the region are x_ (0) = x_ (1) = x_ (2) = x_ (3) = x_ (4) = x_ (5) = x_ (6) = x_ (7) = 10, Assume that x_ (8) = 100 and the order of the central pixels is c = 8. Then, parameters are set as p_peak_thre> 0, and the evaluation function Jp is evaluated by sequentially decreasing the value of k from k = 8. In this case, when the provisional output pixel value rank k = 7, the evaluation function Jp becomes Jp <0 for the first time, and the output pixel value y = x_ (7) in step S38 of FIG. Accordingly, the value of the central pixel value is changed from x_ (8) = 100 to x_ (7) = 10, and the impulsive noise in the plus (+) direction can be removed.

  Next, a case where impulsive noise in the plus (+) direction is added to two pixels including the central pixel in an area where the pixel values are uniform will be described with reference to FIG. The pixel values in the region are x_ (0) = x_ (1) = x_ (2) = x_ (3) = x_ (4) = x_ (5) = x_ (6) = 10, x_ (7) = 90 , x_ (8) = 100, and the center pixel rank c = 8. The parameter is set as p_peak_thre <7, and the evaluation function Jp is evaluated by sequentially decreasing the value of k from k = 8. In this case, as in FIG. 7B, the evaluation function Jp becomes Jp <0 for the first time when the rank k = 6 of the provisional output pixel value, and the output pixel value y = x_ (6). Thereby, impulsive noise in the plus (+) direction can be removed.

  In FIG. 9, when the order of the central pixel is c = 7, the parameter is set as p_peak_thre <7, and when the value of the evaluation function Jp is evaluated while decreasing the value of k, the provisional output pixel value order k = When 6, the evaluation function Jp becomes Jp <0, and the output pixel value y = x_ (6). As described above, according to the method of the present embodiment, it is possible to remove the impulsive noise even when a plurality of impulsive noises exist in the region as shown in FIG.

  As described above, according to the method of the present embodiment, it is possible to remove the impulsive noise regardless of whether the impulsive noise is generated in only one pixel or a plurality of pixels. On the other hand, according to the present embodiment, when the pixel values in the region are greatly different due to factors such as the edge of the image, the detailed image data of the original image is stored without changing the pixel value. Is possible.

  FIG. 10 shows an example in which an edge exists in a region and no impulsive noise is added to the pixel value in the region as an example in which a detailed portion of the original image can be stored. In such a case, since the impulsive noise is not added, it is desirable to store the original image without performing the noise removal process. In other words, when there is no noise, it is desirable to perform processing so that the center pixel value does not fluctuate due to filtering processing. In the example of FIG. 10, the pixel values in the region are x_ (0) = x_ (1) = x_ (2) = x_ (3) = x_ (4) = 10, x_ (5) = x_ (6) = It is assumed that x_ (7) = x_ (8) = 100 and the rank c of the central pixel is c = 8. If the parameter p_peak_thre> 4 is set, k does not satisfy the evaluation function Jp <0 until k = p_peak_thre−1, so the output pixel value is y = x_ (c), and the pixel value of the central pixel is The original pixel value x_ (c) remains unchanged. Thus, according to the adaptive rank order filter of this embodiment, when impulsive noise is not added to the center pixel, the pixel value of the output image remains the original pixel value, and the original image is stored. Can do.

  From the above, considering the examples of FIGS. 9 and 10, if parameters are set so that 4 <p_peak_thre <7, it is possible to perform filter processing that can achieve both impulsive noise removal and storage of the original image. Therefore, according to the present embodiment, it is possible to achieve both noise removal performance and preservation of a fine portion of an image.

  The threshold value p_peak_thre and the threshold value m_peak_thre can be set as appropriate according to the situation. For example, in the case of impulsive noise in the plus direction, if it is assumed that impulsive noise is arranged with three pixel values x_ (6), x_ (7), and x_ (8), if p_peak_thre = 5, then 3 Even when two impulsive noises are arranged, it is possible to reliably determine the occurrence of the impulsive noise. Similarly, in the minus direction, it is desirable to set the value of m_peak_thre according to the number of impulsive noises that are expected to be arranged adjacent to each other.

(5) Comparison with other filter processing Next, the adaptive rank order filter according to the present embodiment is compared with the other filters such as the median filter, and the adaptive rank order filter according to the present embodiment performs. The remarkable effect will be described.

  FIG. 11 is a schematic diagram showing an example of the operation of the median filter. FIG. 11A shows a positive direction only for the center pixel when the pixel values are uniform in the region, as in FIG. The case where the impulsive noise is added is shown. FIG. 11B shows a case where the pixel values are distributed in various values in ascending order within the region, as in FIGS. 4 and 6.

  The median filter is a filter that sorts pixel values of pixels in an arbitrary region including the center pixel, calculates a median value, and replaces the center pixel with the median value. In the case of the median filter, the pixel value of the central pixel having the highest luminance is replaced with the central pixel value in the region, that is, the middle (center) pixel value. In the example of FIG. 11A, the pixel value x_ (8) of the central pixel is replaced with the pixel value x_ (4) of the central value. Similarly, in the example of FIG. 11B, the pixel value x_ (8) of the center pixel is replaced with the pixel value x_ (4) of the median value.

  In the median filter that performs such an operation, in the case of FIGS. 8 and 9, the impulsive noise may be removed by setting the pixel value of the center pixel as the pixel value of the center order. However, in the case of FIG. 9, if the pixel value of the center pixel is set to the median pixel value, the filtering process is performed although the filtering process is unnecessary because it is an edge portion, and the image The detailed part cannot be saved as it is. Therefore, in the median filter, the detailed portion of the original image deteriorates in the case shown in FIG. 9, and it is difficult to achieve both the noise removal performance and the preservation of a fine portion of the image.

  As an impulsive noise removal filter that improves the shortcomings of the median filter, a filter as described below (hereinafter referred to as a filter using the maximum / minimum replacement method) can be considered. In the filter using the maximum / minimum replacement method, when impulsive noise in the plus (+) direction is assumed, pixel values of pixels in an arbitrary region including the central pixel are sorted, and the pixel value of the central pixel is within the region. If it is the largest, it is determined that impulsive noise is added to the center pixel. When impulsive noise is added to the center pixel, a filter process is performed in which the pixel value of the center pixel is replaced with the second largest pixel value in the region. As a result, filter processing is performed only when impulsive noise is added to the center pixel. When the center pixel is not impulsive noise, the pixel values of the center pixel before and after the filter processing change. Filter processing that there is no can be realized.

  However, as shown in FIGS. 12A and 12B, a plurality of impulsive noises may occur side by side in the region. In particular, a plurality of impulsive noises are likely to occur side by side in an image or the like taken under low illuminance conditions. In this case, in the filter processing using the maximum / minimum replacement method, even if the pixel value of the central pixel is replaced with the second largest pixel value x_ (7) in the region, the second largest pixel value x_ (7) is obtained. However, since impulsive noise is added, noise removal cannot be achieved. Therefore, in the filter processing using the maximum / minimum replacement method, the impulsive noise can be removed in the case of FIG. 8, but the impulsive noise is removed in the case of FIG. 9, FIG. 12 (A), and FIG. 12 (B). I can't. As described above, it is difficult to achieve both the noise removal performance and the preservation of a fine part of an image even in a filter using the maximum / minimum replacement method.

  On the other hand, the adaptive rank order filter according to the present embodiment replaces the pixel value of the central pixel with the first provisional pixel value in which the value of the evaluation function Jp or Jm is smaller than 0, as described above. In addition, it is possible to reliably achieve both noise removal performance and preservation of a fine portion of an image.

  As described above, according to the first embodiment, whether or not impulsive noise is included in the center pixel is determined based on a predetermined threshold value for the sorted pixel values. Therefore, it is possible to reliably determine whether or not the central pixel includes impulsive noise regardless of whether or not the pixel value of the central pixel is the maximum (or minimum) within the region.

  When impulsive noise is included, the provisional output pixel value x_ (k) is set, and the first provisional output pixel value x_ (k) for which the evaluation functions Jp and Jm are negative values is set as the central pixel. By making it an output, it is possible to achieve both noise removal performance and preservation of a fine portion of an image.

<2. Second Embodiment>
Next, a second embodiment of the present invention will be described. In the second embodiment, the parameter diff_thre is introduced in the evaluation of the evaluation functions Jp and Jm in the first embodiment. As described above, in the first embodiment, regarding the impulsive noise in the positive direction, when Jp <0, the output value is y = x_ (k), and when Jp ≧ 0, k = k−1 is set again. Jp was calculated and evaluated. In this regard, in the second embodiment, when Jp <diff_thre, the output value is y = x_ (k), and when Jp ≧ diff_thre, the evaluation is performed by calculating Jp again as k = k−1.

  Similarly, in the first embodiment, for the impulsive noise in the negative direction, when Jm <0, the output value is y = x_ (k), and when Jm ≧ 0, Jm is calculated again with k = k + 1. And evaluated. In this regard, in the second embodiment, when Jm <diff_thre, the output value is set to y = x_ (k), and when Jp ≧ diff_thre, the evaluation is performed by calculating Jm again with k = k + 1. Here, the value of diff_thre is an arbitrary real number.

  In the second embodiment, by newly introducing the parameter diff_thre, it is possible to change the intensity of noise removal according to the value of diff_thre. For example, if the value of diff_thre is increased, it is easy to exit from the loop of steps S22 to S26, and it is easy to escape from the loop of steps S32 to S36, so the output value x_ (k) of y is determined at an earlier stage. As a result, the noise removal strength can be reduced. Further, as the value of diff_thr is closer to 0 as a positive value, it becomes more difficult to escape from the loop of steps S22 to S26 and S32 to S36, so that the strength of noise removal can be increased. Furthermore, if diff_thre is set to a negative value, it is possible to increase the strength of noise removal.

  The preferred embodiments of the present invention have been described in detail above with reference to the accompanying drawings, but the present invention is not limited to such examples. It is obvious that a person having ordinary knowledge in the technical field to which the present invention pertains can come up with various changes or modifications within the scope of the technical idea described in the claims. Of course, it is understood that these also belong to the technical scope of the present invention.

DESCRIPTION OF SYMBOLS 100 Image processing apparatus 106 Sort part 108 Noise determination part 110 Adaptive rank order filter

Claims (15)

A sort processing unit that sorts pixel values of a plurality of pixels included in a predetermined area for image data acquired from pixels arranged on the imaging surface;
Whether or not noise is included in the pixel value of the specific pixel based on the sorting order of the pixel values of the specific pixel included in the predetermined area in the plurality of pixel values sorted by the sort processing unit A noise determination unit for determining;
An image processing apparatus comprising:   The noise determination unit compares a sorting order of pixel values of the specific pixel with a preset threshold value, and determines whether noise is included based on a comparison result. The image processing apparatus described.   The noise determination unit, when the sorting order of the pixel values of the specific pixel is less than or equal to a first threshold value counted from the smaller pixel value, impulsive noise in the negative direction to the pixel value of the specific pixel The image processing apparatus according to claim 1, wherein the image processing apparatus determines that the image is included.   The noise determination unit, when the sorting order of the pixel values of the specific pixel is equal to or greater than a second threshold value counted from the smaller pixel value, impulsive noise in the positive direction to the pixel value of the specific pixel The image processing apparatus according to claim 1, wherein the image processing apparatus determines that the image is included. A noise removal unit that removes noise from the pixel value of the specific pixel when the noise determination unit determines that the pixel value of the specific pixel includes noise;
The noise removing unit
A difference between a pixel value of the specific pixel and a provisional pixel value that is one of the plurality of pixel values; and a difference between the provisional pixel value and a pixel value adjacent to the provisional pixel value in a sort order. The image processing apparatus according to claim 1, wherein noise is removed from a pixel value of the specific pixel based on a result of the comparison of. The noise removing unit
When it is determined that the impulsive noise in the negative direction is included in the pixel value of the specific pixel,
The provisional pixel value is sequentially set from the smaller side of the plurality of pixel values, and the comparison is performed every time the provisional pixel value is set, and the difference between the pixel value of the specific pixel and the provisional pixel value is The noise is removed using the pixel value of the specific pixel as the provisional pixel value when the difference between the provisional pixel value and a pixel value adjacent to the provisional pixel value in the sorting order is larger than the difference. The image processing apparatus described. The noise removing unit
When it is determined that the impulsive noise in the positive direction is included in the pixel value of the specific pixel,
The provisional pixel value is sequentially set from the larger side of the plurality of pixel values, and the comparison is performed every time the provisional pixel value is set, and the difference between the pixel value of the specific pixel and the provisional pixel value is The noise is removed using the pixel value of the specific pixel as the provisional pixel value when the difference between the provisional pixel value and a pixel value adjacent to the provisional pixel value in the sorting order is larger than the difference. The image processing apparatus described.   When the provisional pixel value is counted from the smaller pixel value and becomes a pixel value larger than the first threshold value, the noise removal unit does not perform noise removal processing and performs the specific pixel processing. The image processing apparatus according to claim 6, wherein the pixel value is output as it is.   When the provisional pixel value is counted from a smaller pixel value and becomes a pixel value smaller than a second threshold value, the noise removal unit does not perform noise removal processing and performs the specific pixel operation. The image processing apparatus according to claim 7, wherein the original pixel value is output as it is.   The image processing apparatus according to claim 6, wherein the comparison is performed using a predetermined value serving as a buffer. Sorting the pixel values of a plurality of pixels included in a predetermined area for image data acquired from pixels arranged on the imaging surface; and
Determining whether or not noise is included in the pixel value of the specific pixel based on a sorting order of pixel values of the specific pixel included in the predetermined region in the plurality of sorted pixel values; ,
An image processing method comprising:   In the step of determining whether or not the noise is included, the sorting order of the pixel values of the specific pixel is compared with a preset threshold value, and whether or not the noise is included based on the comparison result The image processing method according to claim 11, wherein the determination is made. A step of removing noise from the pixel value of the specific pixel when it is determined that the pixel value of the specific pixel contains noise;
In the step of removing the noise, the difference between the pixel value of the specific pixel and the provisional pixel value that is one of the plurality of pixel values, and the sort order with respect to the provisional pixel value and the provisional pixel value. The image processing method according to claim 11, wherein noise is removed from a pixel value of the specific pixel based on a result of comparing a difference between adjacent pixel values. A lens optical system that forms a subject image;
An image sensor that has a plurality of pixels on which a subject image is formed by the lens optical system and acquires image data of the formed subject image;
For the image data, a sort processing unit for sorting pixel values of a plurality of pixels included in a predetermined region;
Whether or not noise is included in the pixel value of the specific pixel based on the sorting order of the pixel values of the specific pixel included in the predetermined area in the plurality of pixel values sorted by the sort processing unit A noise determination unit for determining;
An imaging apparatus comprising: A noise removal unit that removes noise from the pixel value of the specific pixel when the noise determination unit determines that the pixel value of the specific pixel includes noise;
The noise removing unit
A difference between a pixel value of the specific pixel and a provisional pixel value that is one of the plurality of pixel values; and a difference between the provisional pixel value and a pixel value adjacent to the provisional pixel value in a sort order. The image pickup apparatus according to claim 14, wherein noise is removed from a pixel value of the specific pixel based on a result of comparing.

Images