[go: up one dir, main page]

US20080043124A1 - Methods and apparatuses providing noise reduction while preserving edges for imagers - Google Patents

Methods and apparatuses providing noise reduction while preserving edges for imagers Download PDF

Info

Publication number
US20080043124A1
US20080043124A1 US11/898,909 US89890907A US2008043124A1 US 20080043124 A1 US20080043124 A1 US 20080043124A1 US 89890907 A US89890907 A US 89890907A US 2008043124 A1 US2008043124 A1 US 2008043124A1
Authority
US
United States
Prior art keywords
pixel
value
target pixel
signal
pixels
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Abandoned
Application number
US11/898,909
Inventor
Igor Subbotin
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Aptina Imaging Corp
Original Assignee
Micron Technology Inc
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Micron Technology Inc filed Critical Micron Technology Inc
Priority to US11/898,909 priority Critical patent/US20080043124A1/en
Assigned to MICRON TECHNOLOGY, INC. reassignment MICRON TECHNOLOGY, INC. ASSIGNMENT OF ASSIGNORS INTEREST (SEE DOCUMENT FOR DETAILS). Assignors: SUBBOTIN, IGOR
Publication of US20080043124A1 publication Critical patent/US20080043124A1/en
Assigned to APTINA IMAGING CORPORATION reassignment APTINA IMAGING CORPORATION ASSIGNMENT OF ASSIGNORS INTEREST (SEE DOCUMENT FOR DETAILS). Assignors: MICRON TECHNOLOGY, INC.
Abandoned legal-status Critical Current

Links

Images

Classifications

    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N1/00Scanning, transmission or reproduction of documents or the like, e.g. facsimile transmission; Details thereof
    • H04N1/40Picture signal circuits
    • H04N1/409Edge or detail enhancement; Noise or error suppression
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N25/00Circuitry of solid-state image sensors [SSIS]; Control thereof
    • H04N25/60Noise processing, e.g. detecting, correcting, reducing or removing noise
    • H04N25/68Noise processing, e.g. detecting, correcting, reducing or removing noise applied to defects
    • GPHYSICS
    • G06COMPUTING OR CALCULATING; COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T5/00Image enhancement or restoration
    • G06T5/20Image enhancement or restoration using local operators
    • GPHYSICS
    • G06COMPUTING OR CALCULATING; COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T5/00Image enhancement or restoration
    • G06T5/70Denoising; Smoothing
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N25/00Circuitry of solid-state image sensors [SSIS]; Control thereof
    • H04N25/10Circuitry of solid-state image sensors [SSIS]; Control thereof for transforming different wavelengths into image signals
    • GPHYSICS
    • G06COMPUTING OR CALCULATING; COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00Indexing scheme for image analysis or image enhancement
    • G06T2207/10Image acquisition modality
    • G06T2207/10024Color image

Definitions

  • the embodiments described herein relate generally to the field of solid state imager devices, and more particularly to methods and apparatuses for noise reduction in a solid state imager device.
  • Solid state imagers including charge coupled devices (CCD), CMOS imagers and others, have been used in photo imaging applications.
  • a solid state imager circuit includes a focal plane array of pixels, each one of the pixels including a photosensor, which may be a photogate, photoconductor or a photodiode having a doped region for accumulating photo-generated charge.
  • the first method includes the use of local smoothing filters, which work by applying a local low-pass filter to reduce the noise component in the image.
  • Typical examples of such filters include averaging, medium and Gaussian filters.
  • One problem associated with local smoothing filters is that they do not distinguish between high frequency components that are part of the image and those created due to noise. As a result, these filters not only remove noise but also blur the edges of the image.
  • FIG. 1 is a top-down view of a conventional microlens and color filter array used in connection with a pixel array.
  • FIG. 2A depicts an image correction kernel for a red, greenred, greenblue, or blue pixel of a pixel array in accordance with an embodiment.
  • FIG. 2B depicts a correction kernel for a green pixel of a pixel array in accordance with an embodiment.
  • FIG. 3 depicts the correction kernel of FIG. 2A in more detail.
  • FIG. 4 shows a flowchart of a method for removing pixel noise in accordance with an embodiment.
  • FIG. 6 shows a block diagram of an imager constructed in accordance with an embodiment described herein.
  • FIG. 7 shows a processor system incorporating at least one imager constructed in accordance with an embodiment described herein.
  • pixel refers to a photo-element unit cell containing a photosensor device and associated structures for converting photons to an electrical signal.
  • a small representative three-color pixel array is illustrated in the figures and description herein.
  • the embodiments may be applied to monochromatic imagers as well as to imagers for sensing fewer than three or more than three color components in an array. Accordingly, the following detailed description is not to be taken in a limiting sense, and the scope of the present invention is defined only by the appended claims.
  • FIG. 1 depicts one known conventional color filter array, arranged in a Bayer pattern, covering a pixel array to focus incoming light. It should be understood that, taken alone, a pixel generally does not distinguish one incoming color of light from another and its output signal represents only the intensity of light received, not any identification of color. However, pixels 80 , as discussed herein, are referred to by color (i.e., “red pixel,” “blue pixel,” etc.) when a color filter 81 is used in connection with the pixel array to focus a particular wavelength range of light, corresponding to a particular color, onto the pixels 80 .
  • color i.e., “red pixel,” “blue pixel,” etc.
  • red pixel when used herein, it is referring to a pixel associated with and receiving light through a red color filter; when the term “blue pixel” is used herein, it is referring to a pixel associated with and receiving light through a blue color filter; and when the term “green pixel” is used herein, it is referring to a pixel associated with and receiving light through a green color filter.
  • green pixel can refer to a “greenred pixel,” which is a green pixel in the same row with red pixels, and can refer to a “greenblue pixel,” which is a green pixel in the same row with blue pixels.
  • FIGS. 2A and 2B illustrate parts of pixel array 100 having an identified target pixel 32 a, 32 b that may undergo a corrective method in accordance with an embodiment described herein.
  • the identified target pixel 32 a shown in FIG. 2A in pixel array 100 may be a red, a greenred, a greenblue, or a blue pixel.
  • Pixel array 100 shown in FIG. 2B has an identified pixel 32 b that for purposes of further description is a green pixel (either greenred or greenblue).
  • the pixel array 100 is associated with a Bayer pattern color filter array 82 ( FIG. 1 ); however, the embodiments may also be used with other color filter patterns or the color filter array may be omitted for a monochrome pixel array 100 .
  • the color filters 81 focus incoming light of a particular wavelength range onto the underlying pixels 80 .
  • every other pixel array row consists of alternating red (R) and green (G) colored pixels, while the other rows consist of alternating green (G) and blue (B) color pixels.
  • embodiments utilize signal values of the nearest neighboring pixels of the identified target pixel 32 a , 32 b .
  • the identified target pixel 32 a , 32 b is the pixel currently being processed.
  • the neighboring pixels are collectively referred to herein as a correction kernel, shown in FIGS. 2A and 2B respectively as kernels 101 a , 101 b .
  • correction kernels 101 a , 101 b are outlined with a dotted line.
  • kernel 101 a there are eight pixels (pixels 10 , 12 , 14 , 34 , 54 , 52 , 50 , and 30 ) having the same color as the identified target pixel 32 a .
  • correction kernel 101 a contains sixteen pixels, it should be noted that half of the pixels are not the same color as the target pixel 32 a , whose signals would not be considered for use in denoising target pixel 32 a .
  • the actual pixels that make up kernel 101 a are shown in greater detail in FIG. 3 .
  • the embodiments described herein may be used to denoise images while preserving edges.
  • the target pixel's signal value (“value”) is averaged with the signal values of pixels in the correction kernel. This averaging is done to minimize the effect noise has on an individual pixel. For example, in a flat-field image, an array of ideal pixels would output the same signal value for every pixel in the array; however, because of noise the pixels of the array do not output the same signal for every pixel in the array. By averaging the signal values from the surrounding pixels having the same color as the target pixel, the effect of noise on the target pixel is reduced.
  • a threshold such that averaging is only performed if the difference between the target pixel signal value and the signal values of pixels in the correction kernel is below a threshold. Only noise that has amplitude of dispersion (the difference between the average maximum and minimum value) lower than a noise amplitude threshold (TH) will be averaged and reduced. Therefore, the threshold should be set such that noise is reduced, but pixels along edges will be subjected to less (or no) averaging thereby preserving edges.
  • An embodiment described herein sets a noise amplitude threshold (TH), which may be a function of analog and digital gains that may have been applied to amplify the original signal. It should be appreciated that the threshold TH can be varied based on, for example, pixel color.
  • An embodiment described herein accomplishes this by processing a central target pixel by averaging it with all its like color neighbors that produce a signal difference less than the set threshold. Another embodiment described herein accomplishes this by processing a central target pixel by averaging it with a selected subset of its like color neighbors that produce a signal difference less than the set threshold.
  • the exemplary noise filter could be applied either to each color separately in Bayer, Red/Green/Blue (RGB), Cyan/Magenta/Yellow/Key (CMYK), luminance/chrominance (YUV), or other color space.
  • the method can be carried out by a processor circuit, such as, for example, an image processor circuit 280 (described below with reference to FIG. 6 ) which can be implemented in hardware logic, or as a programmed processor or some combination of the two.
  • a processor circuit separate from an image processor circuit 280 , such as, for example, a separate hardwired logic or programmed processor circuit or a separate stand alone computer.
  • each pixel has a value that represents an amount of light received at the pixel. Although representative of a readout signal from the pixel, the value is a digitized representation of the readout analog signal. These values are represented in the following description as P(pixel) where “P” is the value and “(pixel)” is the pixel number shown in FIGS. 2A or 2 B. For explanation purposes only, the method 200 is described with reference to the kernel 101 a and target pixel 32 a as illustrated in FIG. 2A .
  • the target pixel 32 a being processed is identified.
  • the kernel 101 a associated with the target pixel 32 a is selected/identified.
  • the difference in values P(pixel) of the central (processed) pixel 32 a and each neighboring pixel 10 , 12 , 14 , 30 , 34 , 50 , 52 , 54 in kernel 101 a are compared with a threshold value TH.
  • the threshold value TH may be preselected, for example, using noise levels from current gain settings, or using other appropriate methods.
  • step 203 neighboring pixels that have a difference in value P(pixel) less than or equal to the threshold value TH are selected.
  • a subset of the neighboring pixels that have a difference in value P(pixel) less than or equal to the threshold value TH are selected.
  • the value could be the red value if target pixel 32 a is a red pixel.
  • a value P(pixel) for each of the kernel pixels located around the target pixel 32 a , which were selected in step 203 are added to a corresponding value for the target pixel 32 a and an average value A(pixel) is calculated.
  • the average value A 32 (P 10 +P 12 +P 14 +P 30 +P 32 a +P 34 +P 50 +P 52 +P 54 )/9 is calculated, if all eight neighboring pixels were selected in step 203 .
  • the calculated value A(pixel) which is, in this example, A 32 , replaces the original target pixel value P 32 a.
  • the methods described herein may be carried out on each pixel signal as it is processed.
  • the values of previously denoised pixels may be used to denoise other pixel values.
  • the method and apparatus is implemented in a partially recursive manner (pixels are denoised using values from previously denoised pixels).
  • the embodiments are not limited to this implementation and may be implemented in a fully recursive (pixels are denoised using values from other denoised pixels) or non-recursive manner (no pixels having been denoised are used to denoise subsequent pixels).
  • the method 200 described above may also be implemented and carried out, as discussed above, on target pixel 32 b and associated image correction kernel 101 b ( FIG. 2B ).
  • the kernel 101 b is selected/identified.
  • the differences in values between each of the neighboring pixels 12 , 21 , 23 , 30 , 34 , 41 , 43 , 52 in kernel 101 b located around target pixel 32 b and the value of target pixel 32 b are compared to a threshold TH in step 203 .
  • the remaining steps 204 , 205 are carried out as discussed above for the pixels corresponding to kernel 101 b.
  • the methods described above provide good denoising. It may be desirable, however, to limit the number of pixels utilized in the averaging of the target pixel signal value and the correction kernel signal values to decrease implementation time and/or decrease die size.
  • the number of pixels averaged may, for example, be limited to an integer power of the number two (e.g., 1, 2, 4, 8, etc.) which limits the averaging to binary division.
  • the average value is an average of 2 n pixel signals where n is an integer.
  • Binary division may be desirable as it can be implemented with register shifts, thereby decreasing die size and time necessary to average the target pixel.
  • FIG. 5 illustrates a method 2000 of noise reduction which can be carried out by an image processor circuit 280 (described below with reference to FIG. 6 ) which can be implemented in hardware logic or as a programmed processor or some combination of the two.
  • the method can be implemented by a processor circuit separate from an image processor circuit 280 , such as, for example, a separate hardwired logic or programmed processor circuit or a separate stand alone computer.
  • the method 2000 is described with reference to the kernel 101 a and target pixel 32 a as illustrated in FIG. 2A .
  • a target pixel p having a signal value p sig is selected/identified, for example, pixel 32 a ( FIG. 2A ).
  • pixel 32 a may be a red, greenred, greenblue, or blue pixel.
  • pixel 32 a will be described as and referred to as a greenblue pixel.
  • first and second register values Pixel sum and Pixel sum — new are initialized to be equal to p sig and first and second counters Pixel count and Pixel count — new , respectively, are initialed to be equal to 1 (step 2020 ).
  • a correction kernel associated with the target pixel p containing N pixels is selected/identified, for example, kernel 101 a ( FIG. 2A ) containing greenblue pixels 10 , 12 , 14 , 30 , 34 , 50 , 52 , 54 (step 2030 ).
  • the N pixels from the kernel are grouped at step 2040 . It may be desirable to process the correction kernel pixels that are closest to the target pixel first, for example, the N pixels can be grouped into one or more groups g by their distance from target pixel p.
  • a first group g can be selected to include pixels 12 , 52 , 30 , and 34 that are closest to target pixel 32 a and a second group g can be selected to include pixels 10 , 14 , 50 , and 54 that are further away from target pixel 32 a than the pixels in the first group g.
  • the groups g can be assessed in order of their distance to target pixel p, such that the pixels in a group closest to target pixel p can be assessed before pixels in a group further from target pixel p are assessed.
  • all of the pixels N can alternatively be grouped into one group g.
  • a group g that has not been previously assessed is selected. For example, it may be desirable to select a group of pixels that has not been previously assessed that is closest to target pixel p.
  • a pixel n having a signal value n sig from the selected group g is selected (step 2060 ).
  • the threshold value TH may be preselected, for example, using noise levels from current gain settings, or using other appropriate methods. Additionally, the threshold value TH can be preselected based on the color of the target pixel p. If the determined difference is greater than the threshold TH (step 2070 ), n sig is not included in the averaging and the method 2000 then determines if all of the pixels in group g have been assessed (step 2130 ).
  • a new value for Pixel sum is determined by adding n sig to Pixel sum (step 2080 ) and a new value for Pixel count is determined by incrementing Pixel count (step 2090 ).
  • the method 2000 then compares the value of Pixel count to a set of at least one predetermined number (step 2100 ). For example, it may be desirable to compare the value of Pixel count to a set of values comprised of integer powers of the number two. As described below in more detail, division by Pixel count is required in step 2150 and when implementing division in hardware, division by a power of two can be accomplished with register shifts, thereby making the operation faster and able to be implemented in a smaller die area.
  • Pixel sum — new will not include the current value for Pixel sum .
  • Pixel sum — new is only set when Pixel count is within the set of at least one predetermined number.
  • the method 2000 determines if all pixels in group g have been assessed (step 2130 ). If not, then the method returns to step 2060 and selects a next pixel n. If all of the pixels in group g have been assessed (step 2130 ), the method 2000 determines if all groups g have been assessed (step 2140 ). If all groups g have not been assessed, the method 2000 continues at step 2050 and selects a next group g. If all groups g have been assessed, then p sig — new is determined by dividing Pixel sum — new by Pixel count — new (step 2150 ). The method 2000 can then be repeated for a next target pixel p at step 2010 .
  • the method 2000 described above may also be implemented and carried out, as discussed above, on target pixel 32 b ( FIG. 2B ) and associated image correction kernel 101 b ( FIG. 2B ). For example, it may be desirable to average both greenred and greenblue pixels together. If target pixel 32 b is a greenred pixel, the correction kernel could be selected to include pixels 30 , 12 , 34 , 52 , 21 , 23 , 41 , and 43 where pixels 31 , 12 , 23 , and 52 are greenred pixels and pixels 21 , 23 , 41 , and 43 are greenblue pixels.
  • embodiments of the invention may not provide sufficient denoising to remove spurious noise (i.e., noise greater than 6 standard deviations). Accordingly, embodiments of the invention are better utilized when implemented after the image data has been processed by a filter which will remove spurious noise.
  • a program for operating a processor embodying the methods may be stored on a carrier medium which may include RAM, floppy disk, data transmission, compact disk, etc. which can be executed by an associated processor.
  • a carrier medium which may include RAM, floppy disk, data transmission, compact disk, etc. which can be executed by an associated processor.
  • embodiments may be implemented as a plug-in for existing software applications or may be used on their own. The embodiments are not limited to the carrier mediums specified herein and may be implemented using any carrier medium as known in the art or hereinafter developed.
  • FIG. 6 illustrates an example imager 300 having an exemplary CMOS pixel array 240 with which described embodiments may be used. Row lines of the array 240 are selectively activated by a row driver 245 in response to row address decoder 255 . A column driver 260 and column address decoder 270 are also included in the imager 300 .
  • the imager 300 is operated by the timing and control circuit 250 , which controls the address decoders 255 , 270 .
  • the timing and control circuit 250 also controls the row and column driver circuitry 245 , 260 .
  • a sample and hold circuit 261 associated with the column driver 260 reads a pixel reset signal Vrst and a pixel image signal Vsig for selected pixels of the array 240 .
  • a differential signal (Vrst ⁇ Vsig) is produced by differential amplifier 262 for each pixel and is digitized by analog-to-digital converter 275 (ADC).
  • the analog-to-digital converter 275 supplies the digitized pixel signals to an image processor circuit 280 which forms and may output a digital image.
  • the method 200 ( FIG. 4 ) and method 2000 ( FIG. 5 ) may be implemented by a processor circuit.
  • the processor circuit may be the image processor circuit 280 which is implemented as a digital logic processor pipeline or as a programmed processor that is capable of performing the method 200 ( FIG.
  • the processor circuit may be implemented as a hardwired circuit that processes the analog output of the pixel array and is located between the amplifier 262 and ADC 275 (not shown).
  • the imager 300 has been described as a CMOS imager, this is merely one example imager that may be used. Embodiments of the invention may also be used with other imagers having a different readout architecture. While the imager 300 has been shown as a stand-alone imager, it should be appreciated that the embodiments are not so limited.
  • the embodiments may be implemented on a system-on-a-chip or the imager 300 can be coupled to a separate signal processing chip which implements disclosed embodiments.
  • raw imaging data can be output from the image processor circuit 280 (which can be implemented in hardware logic, or as a programmed processor or some combination of the two) and stored and denoised elsewhere, for example, in a system as described in relation to FIG. 7 below or in a stand-alone image processing system.
  • FIG. 7 shows system 1100 , a typical processor system modified to include the imager 300 ( FIG. 6 ) of an embodiment.
  • the system 1100 is an example of a system having digital circuits that could include imagers. Without being limiting, such a system could include a computer system, still or video camera system, scanner, machine vision, video phone, and auto focus system, or other imager systems.
  • System 1100 may comprise a central processing unit (CPU) 1102 , such as a microprocessor, that communicates with one or more input/output (I/O) devices 1106 over a bus 1104 .
  • Imager 300 also communicates with the CPU 1102 over the bus 1104 .
  • the processor-based system 1100 also includes random access memory (RAM) 1110 , and can include removable memory 1115 , such as flash memory, which also communicates with the CPU 1102 over the bus 1104 .
  • the imager 300 may be combined with a processor, such as a CPU, digital signal processor, or microprocessor, with or without memory storage on a single integrated circuit or on a different chip than the processor.
  • raw image data from the pixel array 240 can be output from the imager 300 image processor circuit 280 and stored, for example in the random access memory 1110 or the CPU 1102 . Denoising can then be performed on the stored data by the CPU 1102 , or can be sent outside the system 1100 and stored and operated on by a stand-alone processor, e.g., a computer, external to system 1100 in accordance with the embodiments described herein.
  • a stand-alone processor e.g., a computer

Landscapes

  • Engineering & Computer Science (AREA)
  • Multimedia (AREA)
  • Signal Processing (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • Color Television Image Signal Generators (AREA)
  • Transforming Light Signals Into Electric Signals (AREA)
  • Image Processing (AREA)
  • Facsimile Image Signal Circuits (AREA)

Abstract

Methods and apparatuses of reducing noise in an image by obtaining a first value for a target pixel, obtaining a respective second value for neighboring pixels surrounding the target pixel, for each neighboring pixel, comparing a difference between the first value and the second value to a threshold value and selectively replacing the first value with an average value obtained from the first value and at least a subset of the second values from the neighboring pixels which have an associated difference which is less than the threshold value based on a result of the comparing step. In a further modification, less than all neighboring pixels which have an associated difference which is less than the threshold value are used in the averaging.

Description

  • This application is a continuation-in-part of U.S. patent application Ser. No. 11/295,445, filed on Dec. 7, 2005, the subject matter of which is incorporated in its entirety by reference herein.
  • FIELD OF THE INVENTION
  • The embodiments described herein relate generally to the field of solid state imager devices, and more particularly to methods and apparatuses for noise reduction in a solid state imager device.
  • BACKGROUND OF THE INVENTION
  • Solid state imagers, including charge coupled devices (CCD), CMOS imagers and others, have been used in photo imaging applications. A solid state imager circuit includes a focal plane array of pixels, each one of the pixels including a photosensor, which may be a photogate, photoconductor or a photodiode having a doped region for accumulating photo-generated charge.
  • One of the most challenging problems for solid state imagers is noise reduction, especially for imagers with a small pixel size. The effect of noise on image quality increases as pixel sizes continue to decrease and may have a severe impact on image quality. Specifically, noise impacts image quality in smaller pixels because of reduced dynamic range. One of the ways of solving this problem is by improving fabrication processes; the costs associated with such improvements, however, are high. Accordingly, engineers often focus on other methods of noise reduction. One such solution applies noise filters during image processing. There are many complicated noise reduction algorithms which reduce noise in the picture without edge blurring, however, they require huge calculating resources and cannot be easily implemented in a system-on-a-chip application. Most simple noise reduction algorithms which can be implemented in system-on-a-chip applications blur the edges of the images.
  • Two known methods that may be used for image denoising are briefly now discussed. The first method includes the use of local smoothing filters, which work by applying a local low-pass filter to reduce the noise component in the image. Typical examples of such filters include averaging, medium and Gaussian filters. One problem associated with local smoothing filters is that they do not distinguish between high frequency components that are part of the image and those created due to noise. As a result, these filters not only remove noise but also blur the edges of the image.
  • A second group of denoising methods work in the spatial frequency domain. These methods typically first convert the image data into a frequency space (forward transform), then filter the transformed image and finally convert the image back into the image space (reverse transform). Typical examples of such filters include DFT filters and wavelength transform filters. The utilization of these filters for image denoising, however, is impeded by the large volume of calculations required to process the image data. Additionally, block artifacts and oscillations may result from the use of these filters to reduce noise. Further, these filters are best implemented in a YUV color space (Y is the luminance component and U and V are the chrominance components). Accordingly, there is a need and desire for an efficient image denoising method and apparatus which does not significantly blur the edges of the image.
  • BRIEF DESCRIPTION OF THE DRAWINGS
  • FIG. 1 is a top-down view of a conventional microlens and color filter array used in connection with a pixel array.
  • FIG. 2A depicts an image correction kernel for a red, greenred, greenblue, or blue pixel of a pixel array in accordance with an embodiment.
  • FIG. 2B depicts a correction kernel for a green pixel of a pixel array in accordance with an embodiment.
  • FIG. 3 depicts the correction kernel of FIG. 2A in more detail.
  • FIG. 4 shows a flowchart of a method for removing pixel noise in accordance with an embodiment.
  • FIG. 5 shows a flowchart of a method for removing pixel noise in accordance with another embodiment.
  • FIG. 6 shows a block diagram of an imager constructed in accordance with an embodiment described herein.
  • FIG. 7 shows a processor system incorporating at least one imager constructed in accordance with an embodiment described herein.
  • DETAILED DESCRIPTION OF THE INVENTION
  • In the following detailed description, reference is made to the accompanying drawings, which form a part hereof and show by way of illustration specific embodiments that may be practiced. These embodiments are described in sufficient detail to enable those of ordinary skill in the art to make and use them, and it is to be understood that other embodiments may be utilized, and that structural, logical, procedural, and electrical changes may be made to the specific embodiments disclosed. The progression of processing steps described is an example of the embodiments; however, the sequence of steps is not limited to that set forth herein and may be changed as is known in the art, with the exception of steps necessarily occurring in a certain order.
  • The term “pixel,” as used herein, refers to a photo-element unit cell containing a photosensor device and associated structures for converting photons to an electrical signal. For purposes of illustration, a small representative three-color pixel array is illustrated in the figures and description herein. However, the embodiments may be applied to monochromatic imagers as well as to imagers for sensing fewer than three or more than three color components in an array. Accordingly, the following detailed description is not to be taken in a limiting sense, and the scope of the present invention is defined only by the appended claims.
  • FIG. 1 depicts one known conventional color filter array, arranged in a Bayer pattern, covering a pixel array to focus incoming light. It should be understood that, taken alone, a pixel generally does not distinguish one incoming color of light from another and its output signal represents only the intensity of light received, not any identification of color. However, pixels 80, as discussed herein, are referred to by color (i.e., “red pixel,” “blue pixel,” etc.) when a color filter 81 is used in connection with the pixel array to focus a particular wavelength range of light, corresponding to a particular color, onto the pixels 80. Accordingly, when the term “red pixel” is used herein, it is referring to a pixel associated with and receiving light through a red color filter; when the term “blue pixel” is used herein, it is referring to a pixel associated with and receiving light through a blue color filter; and when the term “green pixel” is used herein, it is referring to a pixel associated with and receiving light through a green color filter. It should be appreciated that the term “green pixel” can refer to a “greenred pixel,” which is a green pixel in the same row with red pixels, and can refer to a “greenblue pixel,” which is a green pixel in the same row with blue pixels.
  • FIGS. 2A and 2B illustrate parts of pixel array 100 having an identified target pixel 32 a, 32 b that may undergo a corrective method in accordance with an embodiment described herein. The identified target pixel 32 a shown in FIG. 2A in pixel array 100 may be a red, a greenred, a greenblue, or a blue pixel. Pixel array 100 shown in FIG. 2B has an identified pixel 32 b that for purposes of further description is a green pixel (either greenred or greenblue).
  • In the illustrated examples, it is assumed that the pixel array 100 is associated with a Bayer pattern color filter array 82 (FIG. 1); however, the embodiments may also be used with other color filter patterns or the color filter array may be omitted for a monochrome pixel array 100. The color filters 81 focus incoming light of a particular wavelength range onto the underlying pixels 80. In the Bayer pattern, as illustrated in FIG. 1, every other pixel array row consists of alternating red (R) and green (G) colored pixels, while the other rows consist of alternating green (G) and blue (B) color pixels.
  • To denoise identified target pixel 32 a, 32 b, embodiments utilize signal values of the nearest neighboring pixels of the identified target pixel 32 a, 32 b. The identified target pixel 32 a, 32 b is the pixel currently being processed. The neighboring pixels are collectively referred to herein as a correction kernel, shown in FIGS. 2A and 2B respectively as kernels 101 a, 101 b. For example, it may be desirable to select the pixels in the correction kernel 101 a to have the same color as the target pixel 32 a, such as, for example, red, greenred, greenblue, and blue and to select the pixels in the correction kernel 101 b to have the same color as the target pixel 32 b, such as, for example green (without differentiating between greenred and greenblue). A total of eight neighboring pixels are included in each kernel 101 a, 101 b. It should be noted, that the illustrated correction kernels 101 a, 101 b are examples, and that other correction kernels may be chosen for pixel arrays using color filter patterns other than the Bayer pattern. In addition, a correction kernel could encompass more or less than eight neighboring pixels, if desired.
  • In FIGS. 2A and 2B, the illustrated correction kernels 101 a, 101 b are outlined with a dotted line. For kernel 101 a there are eight pixels ( pixels 10, 12, 14, 34, 54, 52, 50, and 30) having the same color as the identified target pixel 32 a. Although it appears that correction kernel 101 a contains sixteen pixels, it should be noted that half of the pixels are not the same color as the target pixel 32 a, whose signals would not be considered for use in denoising target pixel 32 a. The actual pixels that make up kernel 101 a are shown in greater detail in FIG. 3. Kernel 101 b also includes eight pixels ( pixels 12, 23, 34, 43, 52, 41, 30, and 21) having the same green color (without differentiating between greenred and greenblue) as the identified pixel 32 b.
  • As described in detail below, the embodiments described herein may be used to denoise images while preserving edges. Rather than outputting the actual pixel signal value for the target pixel, the target pixel's signal value (“value”) is averaged with the signal values of pixels in the correction kernel. This averaging is done to minimize the effect noise has on an individual pixel. For example, in a flat-field image, an array of ideal pixels would output the same signal value for every pixel in the array; however, because of noise the pixels of the array do not output the same signal for every pixel in the array. By averaging the signal values from the surrounding pixels having the same color as the target pixel, the effect of noise on the target pixel is reduced.
  • In order to preserve edges, it is desirable to set a threshold such that averaging is only performed if the difference between the target pixel signal value and the signal values of pixels in the correction kernel is below a threshold. Only noise that has amplitude of dispersion (the difference between the average maximum and minimum value) lower than a noise amplitude threshold (TH) will be averaged and reduced. Therefore, the threshold should be set such that noise is reduced, but pixels along edges will be subjected to less (or no) averaging thereby preserving edges. An embodiment described herein sets a noise amplitude threshold (TH), which may be a function of analog and digital gains that may have been applied to amplify the original signal. It should be appreciated that the threshold TH can be varied based on, for example, pixel color. An embodiment described herein accomplishes this by processing a central target pixel by averaging it with all its like color neighbors that produce a signal difference less than the set threshold. Another embodiment described herein accomplishes this by processing a central target pixel by averaging it with a selected subset of its like color neighbors that produce a signal difference less than the set threshold. Further, the exemplary noise filter could be applied either to each color separately in Bayer, Red/Green/Blue (RGB), Cyan/Magenta/Yellow/Key (CMYK), luminance/chrominance (YUV), or other color space.
  • With reference to FIG. 4, one example method 200 is now described. The method can be carried out by a processor circuit, such as, for example, an image processor circuit 280 (described below with reference to FIG. 6) which can be implemented in hardware logic, or as a programmed processor or some combination of the two. Alternatively, the method can be implemented by a processor circuit separate from an image processor circuit 280, such as, for example, a separate hardwired logic or programmed processor circuit or a separate stand alone computer.
  • It should be understood that each pixel has a value that represents an amount of light received at the pixel. Although representative of a readout signal from the pixel, the value is a digitized representation of the readout analog signal. These values are represented in the following description as P(pixel) where “P” is the value and “(pixel)” is the pixel number shown in FIGS. 2A or 2B. For explanation purposes only, the method 200 is described with reference to the kernel 101 a and target pixel 32 a as illustrated in FIG. 2A.
  • Initially, at step 201, the target pixel 32 a being processed is identified. Next, at step 202, the kernel 101 a associated with the target pixel 32 a is selected/identified. After the associated kernel 101 a is selected, at step 203, the difference in values P(pixel) of the central (processed) pixel 32 a and each neighboring pixel 10, 12, 14, 30, 34, 50, 52, 54 in kernel 101 a are compared with a threshold value TH. The threshold value TH may be preselected, for example, using noise levels from current gain settings, or using other appropriate methods. In the illustrated example, at step 203, neighboring pixels that have a difference in value P(pixel) less than or equal to the threshold value TH are selected. Alternatively, at step 203, a subset of the neighboring pixels that have a difference in value P(pixel) less than or equal to the threshold value TH are selected. For example purposes only, the value could be the red value if target pixel 32 a is a red pixel.
  • Next, at step 204, a value P(pixel) for each of the kernel pixels located around the target pixel 32 a, which were selected in step 203, are added to a corresponding value for the target pixel 32 a and an average value A(pixel) is calculated. For example, for target pixel 32 a, the average value A32=(P10+P12+P14+P30+P32 a+P34+P50+P52+P54)/9 is calculated, if all eight neighboring pixels were selected in step 203. At step 205, the calculated value A(pixel), which is, in this example, A32, replaces the original target pixel value P32 a.
  • The methods described herein may be carried out on each pixel signal as it is processed. As pixels values are denoised, the values of previously denoised pixels may be used to denoise other pixel values. Thereby, when the method described herein and the values of previously denoised pixels are used to denoise other pixels, the method and apparatus is implemented in a partially recursive manner (pixels are denoised using values from previously denoised pixels). However, the embodiments are not limited to this implementation and may be implemented in a fully recursive (pixels are denoised using values from other denoised pixels) or non-recursive manner (no pixels having been denoised are used to denoise subsequent pixels).
  • The method 200 described above may also be implemented and carried out, as discussed above, on target pixel 32 b and associated image correction kernel 101 b (FIG. 2B). For example, in step 202 the kernel 101 b is selected/identified. After the associated kernel 101 b is selected for target pixel 32 b, the differences in values between each of the neighboring pixels 12, 21, 23, 30, 34, 41, 43, 52 in kernel 101 b located around target pixel 32 b and the value of target pixel 32 b are compared to a threshold TH in step 203. The remaining steps 204, 205 are carried out as discussed above for the pixels corresponding to kernel 101 b.
  • The methods described above provide good denoising. It may be desirable, however, to limit the number of pixels utilized in the averaging of the target pixel signal value and the correction kernel signal values to decrease implementation time and/or decrease die size. For example, as illustrated in the flowchart of FIG. 5, the number of pixels averaged may, for example, be limited to an integer power of the number two (e.g., 1, 2, 4, 8, etc.) which limits the averaging to binary division. In other words, the average value is an average of 2n pixel signals where n is an integer. Binary division may be desirable as it can be implemented with register shifts, thereby decreasing die size and time necessary to average the target pixel. The flowchart of FIG. 5 illustrates a method 2000 of noise reduction which can be carried out by an image processor circuit 280 (described below with reference to FIG. 6) which can be implemented in hardware logic or as a programmed processor or some combination of the two. Alternatively, the method can be implemented by a processor circuit separate from an image processor circuit 280, such as, for example, a separate hardwired logic or programmed processor circuit or a separate stand alone computer. For explanation purposes only, the method 2000 is described with reference to the kernel 101 a and target pixel 32 a as illustrated in FIG. 2A.
  • Initially, at step 2010, a target pixel p having a signal value psig is selected/identified, for example, pixel 32 a (FIG. 2A). It should be appreciated if a Bayer pattern color filter array is utilized with pixel array 100 (FIG. 2A), that pixel 32 a may be a red, greenred, greenblue, or blue pixel. For explanation purposes, pixel 32 a will be described as and referred to as a greenblue pixel. Next, first and second register values Pixelsum and Pixelsum new, respectively, are initialized to be equal to psig and first and second counters Pixelcount and Pixelcount new, respectively, are initialed to be equal to 1 (step 2020). Then, a correction kernel associated with the target pixel p containing N pixels is selected/identified, for example, kernel 101 a (FIG. 2A) containing greenblue pixels 10, 12, 14, 30, 34, 50, 52, 54 (step 2030). The N pixels from the kernel are grouped at step 2040. It may be desirable to process the correction kernel pixels that are closest to the target pixel first, for example, the N pixels can be grouped into one or more groups g by their distance from target pixel p. For example, a first group g can be selected to include pixels 12, 52, 30, and 34 that are closest to target pixel 32 a and a second group g can be selected to include pixels 10, 14, 50, and 54 that are further away from target pixel 32 a than the pixels in the first group g. Then the groups g can be assessed in order of their distance to target pixel p, such that the pixels in a group closest to target pixel p can be assessed before pixels in a group further from target pixel p are assessed. It should be appreciated that all of the pixels N can alternatively be grouped into one group g. Then in step 2050, a group g that has not been previously assessed is selected. For example, it may be desirable to select a group of pixels that has not been previously assessed that is closest to target pixel p. Next, a pixel n having a signal value nsig from the selected group g is selected (step 2060).
  • In step 2070, a determination is made to see if the absolute value of the difference between nsig and Psig is less than a threshold TH. The threshold value TH may be preselected, for example, using noise levels from current gain settings, or using other appropriate methods. Additionally, the threshold value TH can be preselected based on the color of the target pixel p. If the determined difference is greater than the threshold TH (step 2070), nsig is not included in the averaging and the method 2000 then determines if all of the pixels in group g have been assessed (step 2130). However, if the determined difference is less than the threshold TH (step 2070), a new value for Pixelsum is determined by adding nsig to Pixelsum (step 2080) and a new value for Pixelcount is determined by incrementing Pixelcount (step 2090). The method 2000 then compares the value of Pixelcount to a set of at least one predetermined number (step 2100). For example, it may be desirable to compare the value of Pixelcount to a set of values comprised of integer powers of the number two. As described below in more detail, division by Pixelcount is required in step 2150 and when implementing division in hardware, division by a power of two can be accomplished with register shifts, thereby making the operation faster and able to be implemented in a smaller die area. If Pixelcount is contained in the set of at least one predetermined number, for example, if Pixelcount is 4 and the set of at least one predetermined number includes 1, 2, 4, and 8, Pixelcount new is determined by setting Pixelcount —new =Pixelcount (step 2110) and Pixelsum n, is determined by setting Pixelsum new=Pixelsum (step 2120). If Pixelcount is not contained in the set of at least one predetermined number (step 2100), for example, if Pixelcount is 7 and the set of at least one predetermined number includes 1, 2, 4, and 8, Pixelsum new will not be determined and the method 2000 continues by determining if all pixels in group g have been assessed (step 2130). It should be appreciated that if Pixelcount is not in the set of at lest one predetermined number, then Pixelsum new will not include the current value for Pixelsum. In other words, Pixelsum new is only set when Pixelcount is within the set of at least one predetermined number.
  • Then the method 2000 determines if all pixels in group g have been assessed (step 2130). If not, then the method returns to step 2060 and selects a next pixel n. If all of the pixels in group g have been assessed (step 2130), the method 2000 determines if all groups g have been assessed (step 2140). If all groups g have not been assessed, the method 2000 continues at step 2050 and selects a next group g. If all groups g have been assessed, then psig new is determined by dividing Pixelsum new by Pixelcount new (step 2150). The method 2000 can then be repeated for a next target pixel p at step 2010.
  • The method 2000 described above may also be implemented and carried out, as discussed above, on target pixel 32 b (FIG. 2B) and associated image correction kernel 101 b (FIG. 2B). For example, it may be desirable to average both greenred and greenblue pixels together. If target pixel 32 b is a greenred pixel, the correction kernel could be selected to include pixels 30, 12, 34, 52, 21, 23, 41, and 43 where pixels 31, 12, 23, and 52 are greenred pixels and pixels 21, 23, 41, and 43 are greenblue pixels.
  • The above described embodiments may not provide sufficient denoising to remove spurious noise (i.e., noise greater than 6 standard deviations). Accordingly, embodiments of the invention are better utilized when implemented after the image data has been processed by a filter which will remove spurious noise.
  • In addition to the above described embodiments, a program for operating a processor embodying the methods may be stored on a carrier medium which may include RAM, floppy disk, data transmission, compact disk, etc. which can be executed by an associated processor. For example, embodiments may be implemented as a plug-in for existing software applications or may be used on their own. The embodiments are not limited to the carrier mediums specified herein and may be implemented using any carrier medium as known in the art or hereinafter developed.
  • FIG. 6 illustrates an example imager 300 having an exemplary CMOS pixel array 240 with which described embodiments may be used. Row lines of the array 240 are selectively activated by a row driver 245 in response to row address decoder 255. A column driver 260 and column address decoder 270 are also included in the imager 300. The imager 300 is operated by the timing and control circuit 250, which controls the address decoders 255, 270. The timing and control circuit 250 also controls the row and column driver circuitry 245, 260.
  • A sample and hold circuit 261 associated with the column driver 260 reads a pixel reset signal Vrst and a pixel image signal Vsig for selected pixels of the array 240. A differential signal (Vrst−Vsig) is produced by differential amplifier 262 for each pixel and is digitized by analog-to-digital converter 275 (ADC). The analog-to-digital converter 275 supplies the digitized pixel signals to an image processor circuit 280 which forms and may output a digital image. The method 200 (FIG. 4) and method 2000 (FIG. 5) may be implemented by a processor circuit. For example, the processor circuit may be the image processor circuit 280 which is implemented as a digital logic processor pipeline or as a programmed processor that is capable of performing the method 200 (FIG. 4) or method 2000 (FIG. 5) on the digitized signals from the pixel array 240. Alternatively, the processor circuit may be implemented as a hardwired circuit that processes the analog output of the pixel array and is located between the amplifier 262 and ADC 275 (not shown). Although the imager 300 has been described as a CMOS imager, this is merely one example imager that may be used. Embodiments of the invention may also be used with other imagers having a different readout architecture. While the imager 300 has been shown as a stand-alone imager, it should be appreciated that the embodiments are not so limited. For example, the embodiments may be implemented on a system-on-a-chip or the imager 300 can be coupled to a separate signal processing chip which implements disclosed embodiments. Additionally, raw imaging data can be output from the image processor circuit 280 (which can be implemented in hardware logic, or as a programmed processor or some combination of the two) and stored and denoised elsewhere, for example, in a system as described in relation to FIG. 7 below or in a stand-alone image processing system.
  • FIG. 7 shows system 1100, a typical processor system modified to include the imager 300 (FIG. 6) of an embodiment. The system 1100 is an example of a system having digital circuits that could include imagers. Without being limiting, such a system could include a computer system, still or video camera system, scanner, machine vision, video phone, and auto focus system, or other imager systems.
  • System 1100, for example a camera system, may comprise a central processing unit (CPU) 1102, such as a microprocessor, that communicates with one or more input/output (I/O) devices 1106 over a bus 1104. Imager 300 also communicates with the CPU 1102 over the bus 1104. The processor-based system 1100 also includes random access memory (RAM) 1110, and can include removable memory 1115, such as flash memory, which also communicates with the CPU 1102 over the bus 1104. The imager 300 may be combined with a processor, such as a CPU, digital signal processor, or microprocessor, with or without memory storage on a single integrated circuit or on a different chip than the processor. As described above, raw image data from the pixel array 240 (FIG. 6) can be output from the imager 300 image processor circuit 280 and stored, for example in the random access memory 1110 or the CPU 1102. Denoising can then be performed on the stored data by the CPU 1102, or can be sent outside the system 1100 and stored and operated on by a stand-alone processor, e.g., a computer, external to system 1100 in accordance with the embodiments described herein.
  • While the embodiments have been described in detail in connection with preferred embodiments known at the time, it should be readily understood that the claimed invention is not limited to the disclosed embodiments. Rather, the embodiments can be modified to incorporate any number of variations, alterations, substitutions, or equivalent arrangements not heretofore described. For example, the methods can be used with pixels in other patterns than the described Bayer pattern, and the correction kernels would be adjusted accordingly. While the embodiments are described in connection with a CMOS imager, they can be practiced with other types of imagers. Thus, the claimed invention is not to be seen as limited by the foregoing description, but is only limited by the scope of the appended claims.

Claims (23)

1. A method of processing an image, comprising the steps of:
selecting a set of pixel signals from pixels surrounding an identified target pixel having a target pixel signal;
for each of the surrounding pixel signals of the set, determining a respective difference value between the target pixel signal and the surrounding pixel signal;
for each of the surrounding pixel signals of the set, determining if the respective difference value is less than a predetermined threshold; and
substituting an average value as a value for the target pixel signal, the average value is based on the target pixel signal and at least a selected subset of the pixel signals having a respective difference value less than a predetermined threshold.
2. The method of claim 1, wherein the average value is an average of 2n pixel signals where n is an integer.
3. The method of claim 1, wherein the target pixel is one of a red, greenred, greenblue, or blue pixel and each of the pixels in the selected set are the same color as the target pixel.
4. The method of claim 1, wherein at least one of the surrounding pixels has been previously denoised.
5. The method of claim 1, wherein the predetermined threshold is based on at least one of an analog and digital gain used in image capture.
6. The method of claim 1, wherein the predetermined threshold is based on the color of the target pixel.
7. A method of processing an image comprising the steps of:
selecting a target pixel having a first signal value;
replacing a first register value with a sum of the first register value and the first signal value;
incrementing a first counter;
selecting a correction kernel having a number of pixels surrounding the target pixel, each of the kernel pixels having a respective second signal value;
grouping the selected kernel pixels into at least one pixel group;
comparing a difference between the first signal value and a one of the respective second signal values to a threshold value;
replacing the first register value with a sum of the first register value and one of the respective second signal values based on a result of the comparing the difference step;
incrementing the first counter based on a result of the comparing the difference step;
comparing a value of the first counter to a set of at least one predetermined number;
replacing a second register value with the first register value based on a result of the comparing the value step;
replacing a value of a second counter with the value of the first counter based on a result of the comparing the value step; and
replacing the first signal value with the result of a division of the second register value by the second counter.
8. The method of claim 7, further comprising repeating the comparing the difference step through replacing the value of the second counter step for each of the respective second signal values.
9. The method of claim 7, further comprising repeating the comparing the difference step through replacing the value of the second counter step for each of the respective second signal values in a first pixel group before repeating the comparing the difference step through replacing the value of the second counter step for each of the respective second signal values in a second pixel group.
10. The method of claim 7, wherein each selected kernel pixel has a respective distance from the target pixel and are grouped based on each kernel pixels' distance from the target pixel.
11. The method of claim 7, wherein the set of at least one predetermined number is comprised of integer powers of the number two.
12. The method of claim 7, wherein the target pixel is one of a red, green, or blue pixel and each of the selected kernel pixels are the same color as the target pixel.
13. A method of processing an image, comprising the steps of:
selecting a target pixel having a signal value;
selecting a correction kernel associated with the target pixel containing a set of correction kernel pixels each having a respective pixel signal and a respective distance from the target pixel;
for each of the correction kernel pixel signals, determining a respective difference value between the target pixel signal and the correction kernel pixel signal;
for each of the correction kernel pixel signals, determining if the respective difference value is less than a predetermined threshold; and
substituting an average value as a value for the target pixel signal, the average value being an average of 2n pixel signals where n is an integer, wherein the 2n signals are comprised of the target pixel signal and at least a subset of the correction kernel pixel signal having a respective difference value less than a predetermined threshold.
14. The method of claim 13, wherein the at least a subset of correction kernel pixel signals is selected based on the correction kernel pixel signals respective distances from the target pixel.
15. An imager comprising:
a pixel array for capturing an image and comprising a plurality of pixels, each pixel outputting a signal representing an amount of light received; and
a circuit for denoising an image captured by the array,
the circuit being configured to:
select from the captured image a target pixel having a signal value;
select a correction kernel associated with the target pixel containing a set of correction kernel pixels each having a respective pixel signal and a respective distance from the target pixel;
for each of the correction kernel pixel signals, determine a respective difference value between the target pixel signal and the correction kernel pixel signal;
for each of the correction kernel pixel signals, determine if the respective difference value is less than a predetermined threshold; and
substitute an average value as a value for the target pixel signal, the average value being an average of 2n pixel signals where n is an integer, wherein the 2n signals are comprised of the target pixel signal and at least a subset of the correction kernel pixel signals having a respective difference value less than a predetermined threshold.
16. The imager of claim 15, wherein the at least a subset of correction kernel pixel signals is selected based on the correction kernel pixel signals respective distances from the target pixel.
17. The imager of claim 15, wherein the imager is part of a camera system.
18. An imager comprising:
a pixel array for capturing an image and comprising a plurality of pixels, each pixel outputting a signal representing an amount of light received; and
a processing circuit for denoising an image captured by the array, the processing circuit being configured to:
select from the captured image a set of pixel signals from pixels surrounding an identified target pixel having a target pixel signal;
for each of the surrounding pixel signals of the set, determine a respective difference value between the target pixel signal and the surrounding pixel signal;
for each of the surrounding pixel signals of the set, determine if the respective difference value is less than a predetermined threshold; and
substitute an average value as a value for the target pixel signal, the average value is based on the target pixel signal and at least a selected subset of the pixel signals having a respective difference value less than a predetermined threshold.
19. The imager of claim 18, wherein the average value is an average of 2n pixel signals where n is an integer.
20. The imager of claim 18, wherein the target pixel and each pixel in the selected set are the same color.
21. The imager of claim 18, wherein at least one of surrounding pixels has been previously denoised.
22. The imager of claim 18, wherein the imager is part of a camera system.
23. A storage medium containing a program for execution by a processor, the processor when executing the program, performs the steps of:
selecting a set of pixel signals from pixels surrounding an identified target pixel having a target pixel signal;
for each of the surrounding pixel signals of the set, determining a respective difference value between the target pixel signal and the surrounding pixel signal;
for each of the surrounding pixel signals of the set, determining if the respective difference value is less than a predetermined threshold; and
substituting an average value as a value for the target pixel signal, the average value is based on the target pixel signal and at least a selected subset of the pixel signals having a respective difference value less than a predetermined threshold.
US11/898,909 2005-12-07 2007-09-17 Methods and apparatuses providing noise reduction while preserving edges for imagers Abandoned US20080043124A1 (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
US11/898,909 US20080043124A1 (en) 2005-12-07 2007-09-17 Methods and apparatuses providing noise reduction while preserving edges for imagers

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
US11/295,445 US7929798B2 (en) 2005-12-07 2005-12-07 Method and apparatus providing noise reduction while preserving edges for imagers
US11/898,909 US20080043124A1 (en) 2005-12-07 2007-09-17 Methods and apparatuses providing noise reduction while preserving edges for imagers

Related Parent Applications (1)

Application Number Title Priority Date Filing Date
US11/295,445 Continuation-In-Part US7929798B2 (en) 2005-12-07 2005-12-07 Method and apparatus providing noise reduction while preserving edges for imagers

Publications (1)

Publication Number Publication Date
US20080043124A1 true US20080043124A1 (en) 2008-02-21

Family

ID=37839118

Family Applications (3)

Application Number Title Priority Date Filing Date
US11/295,445 Active 2027-09-20 US7929798B2 (en) 2005-12-07 2005-12-07 Method and apparatus providing noise reduction while preserving edges for imagers
US11/898,909 Abandoned US20080043124A1 (en) 2005-12-07 2007-09-17 Methods and apparatuses providing noise reduction while preserving edges for imagers
US13/040,648 Expired - Lifetime US8218898B2 (en) 2005-12-07 2011-03-04 Method and apparatus providing noise reduction while preserving edges for imagers

Family Applications Before (1)

Application Number Title Priority Date Filing Date
US11/295,445 Active 2027-09-20 US7929798B2 (en) 2005-12-07 2005-12-07 Method and apparatus providing noise reduction while preserving edges for imagers

Family Applications After (1)

Application Number Title Priority Date Filing Date
US13/040,648 Expired - Lifetime US8218898B2 (en) 2005-12-07 2011-03-04 Method and apparatus providing noise reduction while preserving edges for imagers

Country Status (7)

Country Link
US (3) US7929798B2 (en)
EP (1) EP1964387A1 (en)
JP (2) JP5067635B2 (en)
KR (1) KR101000268B1 (en)
CN (1) CN101326808B (en)
TW (1) TWI357256B (en)
WO (1) WO2007067505A1 (en)

Cited By (9)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20060146139A1 (en) * 2005-01-04 2006-07-06 Etron Technology, Inc. Digitized image stabilization using energy analysis method
US20070127836A1 (en) * 2005-12-07 2007-06-07 Micron Technology, Inc. Method and apparatus providing noise reduction while preserving edges for imagers
US20090295952A1 (en) * 2008-05-29 2009-12-03 John Robertson Tower Gain matching for electron multiplication imager
US20100239164A1 (en) * 2009-03-18 2010-09-23 Novatek Microelectronics Corp. Method and apparatus for reducing spatial noise of images
US20120321179A1 (en) * 2009-01-07 2012-12-20 International Business Machines Corporation Focus-based Edge Detection
CN103384145A (en) * 2013-06-28 2013-11-06 中国航天科技集团公司第五研究院第五一三研究所 Method for utilizing time window to eliminate pulse per second jitter
US10924695B2 (en) * 2017-10-30 2021-02-16 Imagination Technologies Limited Processing received pixel values in an image processing system using pixel subset average values of grouped subsets
US20220197634A1 (en) * 2020-12-21 2022-06-23 Intel Corporation Efficient divide and accumulate instruction when an operand is equal to or near a power of two
US11688047B2 (en) 2017-10-30 2023-06-27 Imagination Technologies Limited Systems and methods for processing a stream of data values using data value subset groups

Families Citing this family (35)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US8184905B2 (en) * 2005-12-29 2012-05-22 Mtekvision Co., Ltd. Apparatus for color interpolation using adjustable threshold
US8831111B2 (en) * 2006-05-19 2014-09-09 The Hong Kong University Of Science And Technology Decoding with embedded denoising
US8369417B2 (en) * 2006-05-19 2013-02-05 The Hong Kong University Of Science And Technology Optimal denoising for video coding
JP4686496B2 (en) * 2007-03-30 2011-05-25 株式会社東芝 Imaging device
JP4703601B2 (en) * 2007-04-27 2011-06-15 株式会社東芝 Imaging device
CN101971612B (en) * 2007-12-04 2013-05-01 索尼公司 Image processing device and method
US8254718B2 (en) * 2008-05-15 2012-08-28 Microsoft Corporation Multi-channel edge-aware chrominance noise reduction
CN101594457B (en) * 2008-05-30 2012-11-21 深圳艾科创新微电子有限公司 Grading noise reduction system and grading noise reduction method
CN101778297B (en) * 2009-01-08 2012-08-08 华晶科技股份有限公司 Interference elimination method of image sequence
JP5251637B2 (en) * 2009-03-16 2013-07-31 株式会社リコー Noise reduction device, noise reduction method, noise reduction program, recording medium
CN102006430B (en) * 2009-09-02 2013-07-10 英属开曼群岛商恒景科技股份有限公司 Noise removing method applied to image sensor and relevant device
JP4585602B1 (en) * 2009-09-18 2010-11-24 株式会社東芝 Image processing apparatus, display apparatus, and image processing method
US20110085086A1 (en) * 2009-10-12 2011-04-14 Miaohong Shi Method for adaptive noise reduction and apparatus thereof
KR101665137B1 (en) * 2010-04-07 2016-10-12 삼성전자주식회사 Apparatus and method for reducing noise from image sensor
TWI491266B (en) * 2010-05-12 2015-07-01 Himax Imagimg Inc Noise reduction method and apparatus for pixels of a frame
KR20110137624A (en) * 2010-06-17 2011-12-23 삼성전자주식회사 Image processing method for brush effect processing and display device applying the same
CN102316248B (en) * 2010-06-29 2013-04-17 北京创毅视讯科技有限公司 Noise removing method and device
US8866945B2 (en) * 2010-09-26 2014-10-21 Hiok Nam Tay Color image sampling and reconstruction
WO2012046522A1 (en) * 2010-10-04 2012-04-12 富士フイルム株式会社 Imaging device and colour mixing correction method
CN102608601A (en) * 2012-03-08 2012-07-25 北京北科安地科技发展有限公司 Method for recognizing artificial corner reflector in SAR (synthetic aperture radar) image
KR102074857B1 (en) * 2012-09-26 2020-02-10 삼성전자주식회사 Proximity sensor and proximity sensing method using design vision sensor
JP6338469B2 (en) * 2014-06-23 2018-06-06 キヤノン株式会社 Image processing apparatus and image processing method
EP3123710B1 (en) 2014-11-21 2018-08-08 SZ DJI Technology Co., Ltd. System and method of fast adaptive blending for high dynamic range imaging
CN104618701B (en) * 2015-01-13 2017-03-29 小米科技有限责任公司 Image processing method and device, electronic equipment
EP3167429B1 (en) 2015-05-15 2019-12-25 SZ DJI Technology Co., Ltd. System and method for supporting image denoising based on neighborhood block dimensionality reduction
CN105611196B (en) * 2015-12-18 2017-07-21 广东欧珀移动通信有限公司 Dead pixel points of images compensation method, device and terminal device
CN105592270B (en) * 2015-12-18 2018-02-06 广东欧珀移动通信有限公司 image brightness compensation method, device and terminal device
JP6727814B2 (en) * 2016-01-14 2020-07-22 キヤノン株式会社 IMAGING DEVICE, IMAGING DEVICE CONTROL METHOD, AND PROGRAM
KR102208302B1 (en) * 2016-10-24 2021-01-27 삼성전자주식회사 Display apparatus and calibration method thereof
US10510233B2 (en) * 2017-05-26 2019-12-17 Jemez Technology LLC Reducing and eliminating the effects of bright lights in a field-of-view of a camera
CN112419161B (en) * 2019-08-20 2022-07-05 RealMe重庆移动通信有限公司 Image processing method and device, storage medium and electronic equipment
EP3859656B1 (en) * 2020-01-31 2025-03-05 Leica Instruments (Singapore) Pte. Ltd. System, method and computer program for processing raw image data of a microscope
CN114596212B (en) * 2020-12-07 2025-01-17 上海奕瑞光电子科技股份有限公司 Image processing method, device and equipment
KR20220148423A (en) 2021-04-29 2022-11-07 삼성전자주식회사 Denoising method and denosing device of reducing noise of image
EP4181495B1 (en) * 2021-11-11 2025-03-05 Axis AB Method and device for suppressing chromatic aberration in a digital image

Citations (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US4703513A (en) * 1985-12-31 1987-10-27 The United States Of America As Represented By The Administrator Of The National Aeronautics And Space Administration Neighborhood comparison operator
US5594816A (en) * 1989-08-28 1997-01-14 Eastman Kodak Company Computer based digital image noise reduction method based on over-lapping planar approximation
US6229578B1 (en) * 1997-12-08 2001-05-08 Intel Corporation Edge-detection based noise removal algorithm
US20010019633A1 (en) * 2000-01-13 2001-09-06 Livio Tenze Noise reduction
US6633683B1 (en) * 2000-06-26 2003-10-14 Miranda Technologies Inc. Apparatus and method for adaptively reducing noise in a noisy input image signal
US6718068B1 (en) * 2000-03-10 2004-04-06 Eastman Kodak Company Noise reduction method utilizing statistical weighting, apparatus, and program for digital image processing
US20070127836A1 (en) * 2005-12-07 2007-06-07 Micron Technology, Inc. Method and apparatus providing noise reduction while preserving edges for imagers
US20070217836A1 (en) * 2006-03-20 2007-09-20 Kabushiki Kaisha Toshiba Fixing device of image forming apparatus

Family Cites Families (47)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US3009016A (en) * 1959-10-06 1961-11-14 Bell Telephone Labor Inc Noise suppressing video circuit
US4023199A (en) * 1976-03-09 1977-05-10 Bell Telephone Laboratories, Incorporated Method and apparatus for encoding color video signals
US4581588A (en) * 1981-12-29 1986-04-08 Tandy Corporation Switchable bandpass filter
FR2674354A1 (en) * 1991-03-22 1992-09-25 Thomson Csf Method of analysing sequences of road images, device for implementing it and its application to obstacle detection
US5799111A (en) * 1991-06-14 1998-08-25 D.V.P. Technologies, Ltd. Apparatus and methods for smoothing images
JP3044501B2 (en) * 1991-11-29 2000-05-22 松下電器産業株式会社 Video signal processing method and apparatus
JPH0686104A (en) 1992-09-04 1994-03-25 Matsushita Electric Ind Co Ltd Noise reduction device
US5742355A (en) * 1993-12-02 1998-04-21 U.S. Philips Corporation Method and apparatus for reducing noise in a video signal
US6515285B1 (en) * 1995-10-24 2003-02-04 Lockheed-Martin Ir Imaging Systems, Inc. Method and apparatus for compensating a radiation sensor for ambient temperature variations
US6847737B1 (en) * 1998-03-13 2005-01-25 University Of Houston System Methods for performing DAF data filtering and padding
US6625325B2 (en) * 1998-12-16 2003-09-23 Eastman Kodak Company Noise cleaning and interpolating sparsely populated color digital image using a variable noise cleaning kernel
US6995794B2 (en) * 1999-06-30 2006-02-07 Logitech Europe S.A. Video camera with major functions implemented in host software
JP3526020B2 (en) * 1999-12-02 2004-05-10 松下電器産業株式会社 Noise reduction device
US6372074B1 (en) * 2000-06-14 2002-04-16 Avery Dennison Corporation Method of forming a protective coating for color filters
US6757442B1 (en) * 2000-11-22 2004-06-29 Ge Medical Systems Global Technology Company, Llc Image enhancement method with simultaneous noise reduction, non-uniformity equalization, and contrast enhancement
US7129975B2 (en) * 2001-02-07 2006-10-31 Dialog Imaging System Gmbh Addressable imager with real time defect detection and substitution
JP2002259965A (en) * 2001-02-26 2002-09-13 Sony Corp Image noise reduction method and apparatus
JP2002262094A (en) * 2001-02-27 2002-09-13 Konica Corp Image processing method and image processor
US7183532B2 (en) * 2001-04-30 2007-02-27 Hewlett-Packard Development Company, L.P. Detecting a defect in an image scanner
US20030035586A1 (en) * 2001-05-18 2003-02-20 Jim Chou Decoding compressed image data
JP3713574B2 (en) * 2001-05-29 2005-11-09 コニカミノルタビジネステクノロジーズ株式会社 Image processing apparatus, image processing method, and program
WO2002102086A2 (en) 2001-06-12 2002-12-19 Miranda Technologies Inc. Apparatus and method for adaptive spatial segmentation-based noise reducing for encoded image signal
JP4014399B2 (en) 2001-12-13 2007-11-28 松下電器産業株式会社 Noise reduction apparatus and method
US7031543B2 (en) * 2002-01-02 2006-04-18 Xerox Corporation Grayscale image de-speckle algorithm
US20040025791A1 (en) * 2002-08-09 2004-02-12 Applied Materials, Inc. Etch chamber with dual frequency biasing sources and a single frequency plasma generating source
FR2844131B1 (en) * 2002-09-03 2004-11-12 St Microelectronics Sa METHOD AND DEVICE FOR INTERPOLATING IMAGES WITH MOTION COMPENSATION
US7084906B2 (en) * 2002-10-15 2006-08-01 Eastman Kodak Company Reducing computation time in removing color aliasing artifacts from color digital images
JP4487477B2 (en) 2002-11-06 2010-06-23 日本ビクター株式会社 Noise suppression device
US7142234B2 (en) * 2002-12-10 2006-11-28 Micron Technology, Inc. Method for mismatch detection between the frequency of illumination source and the duration of optical integration time for imager with rolling shutter
TWI220360B (en) * 2003-04-23 2004-08-11 Pixart Imaging Inc Defect compensating method and device of color image
KR20070003937A (en) * 2004-03-19 2007-01-05 코닌클리케 필립스 일렉트로닉스 엔.브이. Active matrix display with improved pixel-to-pixel non-uniformity at low luminance levels
JP4624054B2 (en) 2004-03-29 2011-02-02 三洋電機株式会社 Noise reduction device, noise reduction method, and imaging device
KR100573487B1 (en) * 2004-04-27 2006-04-24 매그나칩 반도체 유한회사 Image sensor
EP1605403A1 (en) * 2004-06-08 2005-12-14 STMicroelectronics S.r.l. Filtering of noisy images
US7426314B2 (en) * 2004-07-30 2008-09-16 Hewlett-Packard Development Company, L.P. Adjusting pixels by desired gains and factors
JP2006060744A (en) 2004-08-24 2006-03-02 Matsushita Electric Ind Co Ltd Noise reduction device
US7782998B2 (en) * 2004-12-21 2010-08-24 General Electric Company Method and apparatus for correcting motion in image reconstruction
US7512284B2 (en) * 2005-03-29 2009-03-31 General Electric Company Volumetric image enhancement system and method
US7683948B2 (en) * 2005-03-31 2010-03-23 Freescale Semiconductor, Inc. System and method for bad pixel replacement in image processing
US7580070B2 (en) * 2005-03-31 2009-08-25 Freescale Semiconductor, Inc. System and method for roll-off correction in image processing
US7551799B2 (en) * 2005-05-13 2009-06-23 Seiko Epson Corporation Apparatus and method for image noise reduction
US7349645B2 (en) * 2005-07-08 2008-03-25 Kyocera Mita Corporation Image forming apparatus with trays that are movable to a recording sheet discharge position and a controller for controlling movement of the trays
US7756355B2 (en) * 2006-05-05 2010-07-13 Aptina Imaging Corp. Method and apparatus providing adaptive noise suppression
US7989364B2 (en) * 2006-08-28 2011-08-02 National University Corporation Nagoya University Plasma oxidation processing method
KR100838394B1 (en) * 2007-01-03 2008-06-13 주식회사 하이닉스반도체 Etching method of semiconductor device using hard mask layer
US7718538B2 (en) * 2007-02-21 2010-05-18 Applied Materials, Inc. Pulsed-plasma system with pulsed sample bias for etching semiconductor substrates
JP5138261B2 (en) * 2007-03-30 2013-02-06 東京エレクトロン株式会社 Silicon oxide film forming method, plasma processing apparatus, and storage medium

Patent Citations (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US4703513A (en) * 1985-12-31 1987-10-27 The United States Of America As Represented By The Administrator Of The National Aeronautics And Space Administration Neighborhood comparison operator
US5594816A (en) * 1989-08-28 1997-01-14 Eastman Kodak Company Computer based digital image noise reduction method based on over-lapping planar approximation
US6229578B1 (en) * 1997-12-08 2001-05-08 Intel Corporation Edge-detection based noise removal algorithm
US20010019633A1 (en) * 2000-01-13 2001-09-06 Livio Tenze Noise reduction
US6718068B1 (en) * 2000-03-10 2004-04-06 Eastman Kodak Company Noise reduction method utilizing statistical weighting, apparatus, and program for digital image processing
US6633683B1 (en) * 2000-06-26 2003-10-14 Miranda Technologies Inc. Apparatus and method for adaptively reducing noise in a noisy input image signal
US20070127836A1 (en) * 2005-12-07 2007-06-07 Micron Technology, Inc. Method and apparatus providing noise reduction while preserving edges for imagers
US20070217836A1 (en) * 2006-03-20 2007-09-20 Kabushiki Kaisha Toshiba Fixing device of image forming apparatus

Cited By (17)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20060146139A1 (en) * 2005-01-04 2006-07-06 Etron Technology, Inc. Digitized image stabilization using energy analysis method
US7961966B2 (en) * 2005-01-04 2011-06-14 Etron Technology, Inc. Digitized image stabilization using energy analysis method
US20070127836A1 (en) * 2005-12-07 2007-06-07 Micron Technology, Inc. Method and apparatus providing noise reduction while preserving edges for imagers
US7929798B2 (en) 2005-12-07 2011-04-19 Micron Technology, Inc. Method and apparatus providing noise reduction while preserving edges for imagers
US20110149122A1 (en) * 2005-12-07 2011-06-23 Igor Subbotin Method and apparatus providing noise reduction while preserving edges for imagers
US8218898B2 (en) 2005-12-07 2012-07-10 Micron Technology, Inc. Method and apparatus providing noise reduction while preserving edges for imagers
US20090295952A1 (en) * 2008-05-29 2009-12-03 John Robertson Tower Gain matching for electron multiplication imager
WO2009145803A1 (en) * 2008-05-29 2009-12-03 Sarnoff Corporation Gain matching for electron multiplication imager
US7948536B2 (en) 2008-05-29 2011-05-24 Sri International Gain matching for electron multiplication imager
US8509562B2 (en) * 2009-01-07 2013-08-13 International Business Machines Corporation Focus-based edge detection
US20120321179A1 (en) * 2009-01-07 2012-12-20 International Business Machines Corporation Focus-based Edge Detection
US20100239164A1 (en) * 2009-03-18 2010-09-23 Novatek Microelectronics Corp. Method and apparatus for reducing spatial noise of images
US9031353B2 (en) 2009-03-18 2015-05-12 Novatek Microelectronics Corp. Method and apparatus for reducing spatial noise of images
CN103384145A (en) * 2013-06-28 2013-11-06 中国航天科技集团公司第五研究院第五一三研究所 Method for utilizing time window to eliminate pulse per second jitter
US10924695B2 (en) * 2017-10-30 2021-02-16 Imagination Technologies Limited Processing received pixel values in an image processing system using pixel subset average values of grouped subsets
US11688047B2 (en) 2017-10-30 2023-06-27 Imagination Technologies Limited Systems and methods for processing a stream of data values using data value subset groups
US20220197634A1 (en) * 2020-12-21 2022-06-23 Intel Corporation Efficient divide and accumulate instruction when an operand is equal to or near a power of two

Also Published As

Publication number Publication date
TWI357256B (en) 2012-01-21
JP5067635B2 (en) 2012-11-07
US20070127836A1 (en) 2007-06-07
TW200735640A (en) 2007-09-16
US20110149122A1 (en) 2011-06-23
JP2012090309A (en) 2012-05-10
JP2009518957A (en) 2009-05-07
CN101326808A (en) 2008-12-17
EP1964387A1 (en) 2008-09-03
CN101326808B (en) 2013-04-03
US7929798B2 (en) 2011-04-19
KR20080084996A (en) 2008-09-22
KR101000268B1 (en) 2010-12-10
US8218898B2 (en) 2012-07-10
WO2007067505A1 (en) 2007-06-14

Similar Documents

Publication Publication Date Title
US8218898B2 (en) Method and apparatus providing noise reduction while preserving edges for imagers
US8442345B2 (en) Method and apparatus for image noise reduction using noise models
US9287316B2 (en) Systems and methods for mitigating image sensor pixel value clipping
US8462220B2 (en) Method and apparatus for improving low-light performance for small pixel image sensors
US8035704B2 (en) Method and apparatus for processing a digital image having defective pixels
WO2007130503A2 (en) Method and apparatus providing adaptive noise suppression
US20070133893A1 (en) Method and apparatus for image noise reduction
US8135237B2 (en) Apparatuses and methods for noise reduction
US20080298716A1 (en) Solid-State Imaging Device and Pixel Correction Method
US7471820B2 (en) Correction method for defects in imagers
US20080024618A1 (en) Adaptive binning method and apparatus
JP6697087B2 (en) Imaging device
US20090237530A1 (en) Methods and apparatuses for sharpening images
US9479746B2 (en) Methods and apparatus for demosaicking artifacts suppression

Legal Events

Date Code Title Description
AS Assignment

Owner name: MICRON TECHNOLOGY, INC., IDAHO

Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNOR:SUBBOTIN, IGOR;REEL/FRAME:019882/0711

Effective date: 20070907

AS Assignment

Owner name: APTINA IMAGING CORPORATION, CAYMAN ISLANDS

Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNOR:MICRON TECHNOLOGY, INC.;REEL/FRAME:023245/0186

Effective date: 20080926

Owner name: APTINA IMAGING CORPORATION,CAYMAN ISLANDS

Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNOR:MICRON TECHNOLOGY, INC.;REEL/FRAME:023245/0186

Effective date: 20080926

STCB Information on status: application discontinuation

Free format text: ABANDONED -- FAILURE TO RESPOND TO AN OFFICE ACTION