JP2012065069A - Defect detection and compensation apparatus, program and storage medium - Google Patents

Abstract

Processing for correcting defective pixels in consideration of white balance is performed in a short time.
In a digital camera, a plurality of defective pixel maps are stored in advance. In the defective pixel maps 27 to 28, the positions of defective pixels detected based on the pixel value of CFA image data obtained from the image sensor when exposed for a predetermined time in a light-shielded state and the threshold value changed for each color according to the photographing light source. Information is recorded. The white balance gain calculation circuit 17 calculates a white balance gain based on the CFA image data acquired from the image sensor 25. The defective pixel map selection circuit 18 selects one of the plurality of defective pixel maps 27 to 29 according to the white balance gain. The defect correction circuit 19 corrects the defective pixel based on the position information of the defective pixel described in the selected defective pixel map.
[Selection] Figure 1

Description

  The present invention relates to a defect detection and correction apparatus, a program, and a recording medium that detect defective pixels generated in an image signal, particularly on a solid-state imaging device.

  In a solid-state imaging device (hereinafter referred to as “imaging device”) that constitutes a color image sensor or the like, the sensitivity is reduced due to local crystal defects of the semiconductor, or scratches caused by some stress factor after shipment. As a result, pixel defects may occur. In such a case, a white spot defect or a black spot defect occurs in the image due to the pixel defect, and the image quality deteriorates. For this reason, in a digital camera using an image sensor, position information of a defective pixel is detected in advance, and after acquiring image data, an output value of the defective pixel is corrected using a neighboring pixel value based on the position information. A defective pixel correction circuit is incorporated.

  By the way, in a general digital camera, correction of a defective image is not considered when each RGB output value of each pixel is changed by image processing or the like. For example, even when the white balance processing does not output an abnormal value before amplification and does not correspond to a defective pixel, there may be data corresponding to the defective pixel depending on the magnitude of the gain. is there. Since the pixel corresponding to the defective pixel by the white balance correction is not a defect correction target, if it is displayed as it is as an image, there is a possibility that a white point defect due to this pixel will appear on the image.

  Therefore, an image processing apparatus that corrects defective pixels in consideration of white balance correction is known (Patent Document 1). In the image processing apparatus described in Patent Document 1, position information of a pixel in which an output value of each pixel obtained from the image sensor is equal to or more than a first threshold when exposed for a predetermined time in a completely dark state, and RGB information of the pixel position And the output value are stored in the defective pixel candidate table in advance, and the gain corresponding to the white balance is set to the output value of the pixel described in the defective pixel candidate table in response to the white balance setting performed before or after photographing. Multiplying is performed, and defective pixels are corrected by detecting, as a defective pixel, a pixel whose multiplied value exceeds a second threshold value that is larger than the first threshold value.

JP 2007-140646 A

  However, in the invention described in Patent Document 1, every time white balance is set, an operation for multiplying the digital data of the defect candidate pixel by the gain, and the multiplied data and the second threshold value are compared to extract the defective pixel. Since it is necessary to perform the two processes of comparison and determination, it takes a long time to perform defect correction and obtain an image, and there is a possibility that the next photographing cannot be performed quickly.

  The present invention has been made in view of the above problems, and a defective pixel correction apparatus, a program, and a storage medium that can perform a process of correcting a defective pixel in a short time in consideration of shooting conditions such as white balance. The purpose is to provide.

  In order to achieve the above-described goal, in the present invention, shooting condition setting means for setting shooting conditions, and position information of defective pixels detected based on color image data obtained from the image sensor when shooting at a predetermined brightness Defective pixel information storage means for storing a plurality of defective pixel information recorded in advance for each color constituting color image data, and any one of the plurality of defective pixel information according to the set photographing condition Selection means for selecting defective pixel information, and defective pixel correction means for correcting defective pixels based on position information of defective pixels recorded in the selected defective pixel information.

  As a photographing condition, white balance gain or ISO sensitivity is suitable. Note that a blank grid interpolation process (pixel interpolation process), a continuous shooting mode, or the like may be used. The defective pixel information may be a memory map, a table memory, or the like that records position information of defective pixels detected based on the threshold value determined for each color according to the shooting conditions.

  The color image data may be image data obtained from a single image sensor having a three-color color filter array. In this case, a plurality of pixels that output color information of the first color component having the highest spatial frequency, and a second color component that does not output color information of the first color component and has a spatial frequency lower than that of the first color component Alternatively, the image data is generated by an imaging device in which a plurality of pixels that output color information of the third color component are two-dimensionally arranged. Color image data obtained from a three-plate CCD or the like may be used.

  When the shooting condition is white balance, the first defective pixel information detected using a uniform threshold for each color with respect to color image data obtained from the image sensor when shooting at a predetermined brightness, The second defective pixel information detected using the threshold value for determining the color information of the second color component more strictly than the other color information and the threshold value for determining the color information of the third color component more strictly than the other color information are used. The third defective pixel map detected in advance is stored in the defective pixel information storage means in advance, and the color information of the second and third color components is determined based on the color image data obtained at each photographing. A white balance gain calculation circuit for calculating a white balance gain is provided, and the selection unit selects one of the first to third defective pixel information based on the two white balance gains. It can be formed.

  The present invention may be a defective pixel correction method in which color image data is corrected based on defective pixel information for each color. In this case, the setting of the shooting condition setting step for setting the shooting condition and the plurality of pieces of defective pixel information in which the position information of the defective pixel is recorded for each color constituting the color image data is set. And a defective pixel information selecting step to be selected according to the shooting conditions, and a defective pixel correcting step for correcting the defective pixel based on the position information of the defective pixel recorded in the selected defective pixel information. Further, the present invention may be a computer-readable program that executes a defective pixel correction method. Moreover, it is good also as a recording medium which recorded the computer readable program.

  In the present invention, it is possible to reliably detect defective pixels that affect the photographing conditions by selecting any defective pixel information from among a plurality of defective pixel information, and therefore, the calculation and comparison described in the related art Therefore, it is possible to quickly perform defective pixel correction in consideration of shooting conditions.

It is a block diagram which shows the structure of the digital camera which employ | adopted this invention. It is explanatory drawing which shows the color filter array of a primary color Bayer arrangement. It is explanatory drawing which shows roughly the output value of each pixel which comprises the CFA image data for 1 screen obtained from an image pick-up element by a two-dimensional arrangement | sequence. It is explanatory drawing which shows a 1st defective pixel map roughly. It is explanatory drawing which shows another example which replaced with the 1st defective pixel map and was made into the table memory. It is explanatory drawing which shows a 2nd defective pixel map roughly. It is explanatory drawing which shows another example which replaced with the 2nd defective pixel map and was made into the table memory. It is explanatory drawing which shows a 3rd defective pixel map roughly. It is explanatory drawing which shows another example which replaced with the 3rd defective pixel map and was made into the table memory. It is explanatory drawing which shows roughly the determination table used when selecting a defective pixel map. It is a flowchart figure which shows the operation | movement procedure of a digital camera. It is a graph which shows roughly the sensitivity specific of an image sensor. It is a block diagram which shows another embodiment of the digital camera which records a Raw file. It is explanatory drawing which shows the content of the Raw file produced | generated with the camera demonstrated in FIG. It is a block diagram which shows the structure of the Raw file developing device which develops the Raw file demonstrated in FIG. It is explanatory drawing which shows the content of the Raw file produced | generated with the digital camera which sets arbitrary white balance gains. FIG. 17 is a block diagram illustrating an embodiment of a raw file developing device that reads the raw file described in FIG. 16 and sets an arbitrary white balance gain. It is explanatory drawing which shows schematically the structure of the Raw file which records a defective pixel level map. FIG. 19 is a block diagram illustrating an embodiment of a raw file developing device that reads the raw file described in FIG. 18 and sets an arbitrary empty grid interpolation process. It is explanatory drawing which shows the relationship between the range which divided each pixel value in steps in order to produce | generate a defective pixel level map, and the correction level corresponding to each range. It is explanatory drawing which shows the outline of a defective pixel level map. It is explanatory drawing which shows the relationship between the correction level demonstrated in FIG. 21, and a defective pixel map.

  As shown in FIG. 1, the digital camera 10 includes an optical system 11, a color image sensor 12, an analog front end (hereinafter referred to as “AFE”) 13, an image processing unit 14, an image compression unit 15, and a recording unit 16. ing. The image processing unit 14 includes a white balance gain calculation circuit 17, a defective pixel map selection circuit 18, a defect correction circuit 19, a white balance processing circuit 20, a sky lattice interpolation circuit 21, a color conversion circuit 22, and the like. A RAM 23 for work memory is connected to the image processing unit 14.

  The optical system 11 includes a photographic lens, a diaphragm, and the like. The color image sensor (Bayer array type single-plate image pickup device) 12 includes a color filter array (CFA) 24 and a solid-state image pickup device 25 (hereinafter referred to as “image pickup device”) 25 such as a CCD. Charges obtained by photoelectrically converting the optical image to be formed are accumulated for a certain time for each light receiving cell, and an electric signal corresponding to the amount of light received for each light receiving cell (pixel) is output.

  In the color filter array 24, for example, as shown in detail in FIG. 2, the resolution of G (green) is N / 2, and R (red) and B (blue) with respect to the total number N of pixels of the image sensor 25. The primary color Bayer array has a resolution of N / 4.

  Here, each pixel includes a pixel that outputs G color and a pixel that does not output G color. That is, the image sensor 25 outputs a plurality of pixels that output color information of the first color component having the highest spatial frequency, and a second pixel that does not output color information of the first color component and has a spatial frequency lower than that of the first color component. A plurality of pixels that output color information of the color component or the third color component are two-dimensionally arranged.

  The AFE 13 converts an analog electrical signal output from the color image sensor 12 into digital and outputs it as CFA image data, and performs processing such as noise removal at the time of digital conversion.

Based on the CFA image data, the white balance gain calculation circuit 17 sets, for example, the color information of R (red) and B (blue) of the three colors to be equal to the peak level of the color information of G (green). In order to adjust, a coefficient for multiplying each RGB value at the time of white balance correction processing, that is, a white balance gain is calculated. The white balance adjusting amount is composed of the value W _R of R / G gain (red gain for green), and the value W _B of B / G gain (blue gain to green).

As an example, the white balance gain calculation circuit 17 converts the CFA image data into TC-Duv coordinate data according to JIS Z8725 “Measurement method of distribution temperature and color temperature / correlated color temperature of light source”, and achromatic portion in the image data. Determine. When the data after coordinate conversion is determined to be achromatic, the white balance gain calculation circuit 17 applies the correlated color temperature obtained from the TC-Duv coordinates to a predetermined color temperature-white balance gain relational expression. R and B white balance gains W _R and W _B for G are calculated.

Information of the white balance gain W _R, W _B are sent respectively to the defective pixel map selection circuit 18 and the white balance processing circuit 20. The defective pixel map selection circuit 18 includes a storage unit 30 that stores the first to third defective pixel maps 27 to 29, and the first to third based on the calculated two white balance gains W _R and W _B. Any one of the defective pixel maps 27 to 29 is selected.

  In each of the defective pixel maps 27 to 29, position information of defective pixels detected based on a predetermined threshold is recorded in advance. The threshold value is determined for each color according to the imaging light source determined by the color temperature and the spectral radiation intensity of the light source. This is because the influence of the white spot defect at the R or B pixel position is different from the influence of the white spot defect at the pixel position of the other color due to the relationship between the spectral sensitivity of the image sensor 25 and the photographing light source.

The defect correction circuit 19 corrects the defective pixels recorded in the selected defective pixel maps 27-29. The white balance processing circuit 20 uses the two white balance gains W _R and W _B calculated by the white balance gain calculation circuit 17 as pixel values of corresponding colors so that the achromatic subject becomes an achromatic subject image. Multiply by.

  Since the CFA image data is any one of RGB data, the empty lattice interpolation circuit 21 performs a process (pixel interpolation process) for compensating for colors other than the pixels by estimating from the color of the surrounding pixels for each pixel. Go to make RGB image data.

  As an interpolation method used here, a luminance component image may be generated from CFA image data and empty grid interpolation may be performed. Alternatively, any of interpolation methods such as a well-known interpolation method for maintaining hue, an interpolation method using median processing, an interpolation method based on gradient, and an adaptive color brain interpolation method (ACPI) can be used.

The obtained RBG image data is data based on the sensitivity characteristics of the image sensor 25. Therefore, the color conversion circuit 22 performs color conversion and gradation conversion, the _S RGB or the like suitable for display to the RGB image data.

The image compression unit 15 encodes the _S RGB image data by an irreversible compression method, for example, the JPEG method, and creates compressed image data. The recording unit 16 records the compressed image data in, for example, an internal memory or a removable memory card.

  Next, the operation of the above configuration will be briefly described. The assembly factory of the digital camera 10 includes defective pixel map generation means and writing means. The defective pixel map generating unit generates a plurality of defective pixel maps 27 to 29 in advance according to the photographing light source, and the writing unit records the generated defective pixel maps 27 to 29 in the storage unit 30 of the digital camera 10.

  The defective pixel maps 27 to 29 extract pixels that output abnormal values from the respective pixels of the image sensor 25 based on CFA image data obtained by imaging in advance, and position information of only the extracted defective pixels. It is recorded as a table.

  Specifically, a value indicating CFA image data output from each pixel of the image sensor 25 when exposed for a predetermined time in a completely dark state as 12-bit digital data is compared with a predetermined threshold, and the threshold Is extracted as a defective pixel, and the position information of the extracted defective pixel is stored.

  For example, it is assumed that CFA image data is obtained as shown in FIG. Here, the numbers in FIG. 3 are digital data. Based on the CFA image data, first to third defective pixel maps 27 to 29 having position information of defective pixels corresponding to the photographing light source are created.

For example, the first defective pixel map 27 is a table in which defective pixels are extracted without distinguishing RGB and their position information is recorded. The threshold values used at this time are all uniform values, that is, “T _R ” = “T _G ” = “T _B ”. Here, “T _R ” is a threshold value for R pixel, “T _G ” is a threshold value for G pixel, and T _B is a threshold value for B pixel. Here, when the thresholds “T _R ”, “T _G ”, and “T _B ” are set to “150”, for example, the first defective pixel map 27 displays the pixel position “×” shown in FIG. The map is “1” representing a pixel and “0” for normal pixels other than “x”. The first defective pixel information is not limited to the map 27. For example, as shown in FIG. 5, the first defective pixel information may be a table that records only the position information of the defective pixels.

The second defective pixel map 28 is a table in which defective pixels are extracted with stricter determination criteria for R pixels and their positional information is recorded. The threshold used at this time has a relationship of “K _R ” × “T _R ” = “T _G ” = “T _B ”. “K _R ” is a constant equal to or less than “1”. Here, when the threshold values “T _R ”, “T _G ”, and “T _B ” described above are set to “150” and “K _R ” is set to “0.7”, for example, the G and B pixels The threshold values “T _G ” and “T _B ” are “150”, whereas the threshold value for the R pixel is “K _R ” × “T _R ”, so “105”. Therefore, as shown in FIG. 6, a pixel position “x” is detected as a defective pixel. The defective pixels detected at this time are strictly detected by R pixels with respect to the first defective pixel map 27, and the second defective pixel map 28 indicates, for example, a pixel position of “x” shown in FIG. Is a map in which normal pixels other than “x” are “0”. As described above, the second defective pixel information may be a table that records only the positional information of the defective pixels, as shown in FIG.

The third defective pixel map is a table in which defective pixels are extracted with strict criteria for determining B pixels and their positional information is recorded. The threshold value used at this time has a relationship of “T _R ” = “T _G ” = “T _B ” × “K _R ”. “K _R ” is a constant equal to or less than “1”. Here, assuming that the threshold values “T _R ”, “T _G ”, and “T _B ” are “150” and “K _R ” is “0.7”, the threshold values “T _R ” for R and G pixels are used. , “T _G ” is “150”, and the threshold value for the B pixel is “K _R ” × “T _B ”, so “105”. Therefore, the pixel position “x” shown in FIG. 8 is detected as a defective pixel. The defective pixels detected at this time are strictly detected by only B pixels with respect to the first defective pixel map 27, and the third defective pixel map 29 indicates the pixel position of “x” shown in FIG. This is a map in which “1” is expressed, and normal pixels other than “x” are “0”. The third defective pixel information is not limited to the map, and may be a table that records only the position information of the defective pixels as shown in FIG. 9, for example.

Defective pixel map selection circuit 18, the two white balance gains W _R determined by the imaging light _source, based on W _B, any one of a plurality of defective pixel map 27 to 29, or a combination of any select.

When selecting a map, for example, as shown in FIG. 10, two white balance gains W _R, the other one of W _B to the longitudinal axis horizontal axis, depending on the range of the horizontal and vertical axes defect A decision table to which pixel maps 27 to 29 are assigned is used. This determination table is stored in the storage unit 30 in advance.

Determination table, white balance gain W _B for the B pixel is "0" in the range between "1.8" and the white balance gain W _R for R pixel is in the range of "0" to "1.8" Sometimes the first defective pixel map (Map1) is selected. Further, in the range white balance gain W _B for the B pixel is "1.8" to "3", and when the white balance gain W _R for R pixel is in the range of "0" to "1.8" A second defective pixel map (Mpa2) is selected. Furthermore, to the extent the white balance gain W _B for B pixels are "0" to "1.8", and when the white balance gain W _R for R pixel is in the range of "1.8" to "3" A third defective pixel map (Map3) is selected. Then, the range of the white balance gain W _B for the B pixel is "1.8" to "3", and when the white balance gain W _R for R pixel is in the range of "1.8" to "3" is Second and third defective pixel maps (Map2 + Map3) are selected. In this case, at least one of the position information of the defective pixels obtained by the logical sum (OR) with the second and third defective pixel maps, that is, the position information of the defective pixels of the second and third defective pixel maps is included. Position information obtained by the position information and the second or third defective pixel map is defined as a defective pixel.

Next, the operation of the digital camera 10 will be described with reference to FIG. The CFA image data captured from the image sensor 25 in response to the shutter release is captured by the white balance gain calculation circuit 17 and the defect correction circuit 19 via the AFE 13. White balance gain calculation circuit 17 automatically calculates the white balance gain _W R, the _{W B} on the basis of the CFA image data.

For example, in the case of CFA image data captured in the environment of the CIE standard light source D65 using the image sensor 25 having the spectral sensitivity characteristic shown in FIG. 12, the white balance gain “W _R ” is approximately “2.08”. , “W _B ” becomes approximately “1.25”. Further, in the case of CFA image data shot under the environment of the CIE standard light source A, the white balance gain “W _R ” is substantially “1.01”, and “W _B ” is substantially “2.89”. In the case of CFA image data photographed in the environment of the CIE standard light source A, the influence of the light source is left without completely aligning the R, G, B color image data with the achromatic subject. There is also. In this case, “W _R ”> “1.01” and “W _B ” <“2.89” may be satisfied. As described above, the white balance gains W _R and W _B are calculated based on the spectral sensitivity of the image sensor 25 and the spectral intensity of a predetermined light source obtained from the CFA image data.

WB gain calculating circuit 17, the calculated white balance gain W _R, and sends the information of W _B to the defective pixel map selection circuit 18 and the white balance processing circuit 20. Defective pixel map selection circuit 18, the two white balance gains _W R, selects a defective pixel map 27 to 29 based on the _{W B.}

For example, when the white balance gain “W _R ” is approximately “2.08” and “W _B ” is approximately “1.25”, “Map 2”, that is, the second defective pixel map as described with reference to FIG. 28, and when the white balance gain “W _R ” is approximately “1.01” and “W _B ” is approximately “2.89”, “Map 3”, that is, the third defective pixel map 29 Is selected. The selected defective pixel maps 27 to 29 are sent to the defect correction circuit 19.

  The defect correction circuit 19 collates the CFA image data against the defective pixel maps 27 to 29, detects the position of the defective pixel on the CFA image data, and detects the same four colors of the same color adjacent to the detected defective pixel. The defective pixel correction is performed by replacing the average value of the digital data at the pixel position with the digital data of the defective pixel. If a defective pixel table is used instead of a map, the position information of the defective pixel described in the selected table may be read to detect the position of the defective pixel on the CFA image data.

Thereafter, the white balance processing circuit 20 performs white balance processing. White balance processing circuit 20, the calculated white balance gain W _R, the W _B, multiplies the R pixel, each digital data of the B pixel.

Subsequently, pixel interpolation processing is performed by the empty lattice interpolation circuit 21 to generate RGB image data, and then color conversion processing to _S RGB image data is performed by the color conversion circuit 22 based on the RGB image data. . After performing the color conversion process creates a compressed image data by compressing the _S RGB image data by the image compression unit 15. The compressed image data is recorded by the recording unit 16 on, for example, an internal memory or a memory card.

  In the above embodiment, the image sensor 25 is described as a CCD. However, the present invention is not limited to this and may be a MOS type image sensor. The present invention is not limited to the digital camera 10 and can be applied to image forming apparatuses such as a video camera, a television camera, a camera-equipped mobile phone, a security monitor camera, and a scanner.

  The CFA 24 may also use various CFAs other than the Bayer array. Furthermore, although the description is made on the RGB three-color Bayer array CFA 24, the present invention is also applicable to the case where the CFA has four or more colors.

  In the above embodiment, the digital camera 10 having the auto white balance function is described. However, a white balance mode setting unit that sets a white balance mode desired by the user may be provided. In this case, a gain storage unit that stores in advance the white balance gain associated with the white balance mode and a gain selection unit that selects the white balance gain from the gain recording unit according to the set white balance mode are provided. The gain selection unit may send the selected white balance gain information to the defective pixel map selection circuit 18 and the white balance processing circuit 20. Further, gain input means may be provided, and the user may input an arbitrary white balance gain by the gain input means.

  Further, in each of the above embodiments, the defect correction circuit 19 performs the process of setting the adjacent pixel value of the same color as the defective pixel to the pixel value at the position of the defective pixel. The correlation may be examined, and the pixel value of the same color in the correlated direction may be interpolated to obtain the pixel value at the position of the defective pixel.

  In each of the above-described embodiments, a demosaic process for generating a full color image by performing white balance processing, empty grid interpolation processing, color conversion processing, and the like on CFA image data is performed, and general-purpose data such as JPEG is performed on the completed image data. Although stored in a compressed image format, the present invention is not limited to this, and the present invention can be applied to a digital camera having a function of saving raw data before demosaic processing as it is.

  In this case, the calculated white balance gain and the defective pixel map selected by the defective pixel map selection circuit may be stored in association with the raw data.

  Such a digital camera 31 includes, for example, an optical system 11, a color image sensor 12, an A / D 32, an image processing unit 33, and a recording unit 34, as shown in FIG. A white balance gain calculation circuit 35, a defective pixel map selection circuit 36, an image compression circuit 37, and a file output circuit 38.

The A / D 32 outputs the raw CFA image data converted to digital to the white balance gain calculation circuit 35 and the image compression circuit 37. White balance gain calculation circuit 35, the white balance gain W _R based on the CFA image data as described _above, to calculate the W _B, calculated white balance gain W _R, W _B defective pixel map selection circuit 36 and the file output To the circuit 38. Defective pixel map selection circuit 36, based on the white balance gain W _R, the W _B, selecting one of a plurality of defective pixel map 27 to 29 stored in the storage unit 39, the defective pixel map 27 to the selected 29 is sent to the file output circuit 38.

The image compression circuit 37 performs lossless compression or lossy compression (high bit rate) on the CFA image data and outputs the result to the file output circuit 38. As shown in FIG. 14, the file output circuit 38 associates the CFA compressed image data, the white balance gains W _R and W _B , and the selected defective pixel map with each other and records them in a single file, thereby recording the Raw file. Output as. The recording unit 34 records the raw file in, for example, an internal memory or a memory card.

  The raw file is developed by a raw file developing device. For example, the raw file development device 40 shown in FIG. 15 includes a raw file storage unit 41, a development processing execution unit 42, a display unit 43, and the like. The RAW file recorded in the digital camera 31 is transferred to the RAW file storage unit 41 via the interface according to a transfer instruction from the input unit 44.

  The development processing execution unit 42 includes a program including a defective pixel correction process 45, a white balance process 46, a pixel interpolation process 47, a color conversion process 48, and the like. These programs are recorded on a recording medium such as a hard disk. ing. When a raw file development instruction is input from the input unit 44, the defective pixel correction processing 45 reads CFA image data and a defective pixel map recorded in the raw file, and the defective pixel is read from the CFA image data. Defect correction is performed according to the position information of the defective pixels recorded on the map. The defective pixel map recorded in the RAW file is a defective pixel map selected based on the white balance gain calculated by the digital camera 31.

Then, white balance processing 46, the white balance gain W _B recorded in the Raw _file, reads the W _R, the white balance gain B of the defective pixel corrected CFA image data for each pixel value of R W _B, the white balance processing by multiplying the _{W R.} Thereafter, the pixel interpolation processing 47 creates RGB image data based on the CFA image data that has been subjected to white balance processing, and the color conversion processing part performs _S RGB conversion based on the RGB image data. An image is displayed on the display unit 43 based on the _S RGB image data.

Incidentally, the white balance gain W _B calculated by the digital camera _31, without the use of W _R, the white balance gain W _B which the user desires via an input unit _44, when performing the white balance processing to input W _R There is. In this case, on the digital camera 31 side, not only the selected defective pixel map, but also all the defective pixel maps 27 to 29 recorded in the storage unit 39 as shown in FIG. Configure to record in the Raw file.

  In this case, as shown in FIG. 17, in the raw file development device 50, a defective pixel map selection unit 52 is provided in the development processing execution unit 51. The defective pixel map selection unit 52 selects any defective pixel map from the plurality of defective pixel maps 27 to 29 read from the Raw file based on the white balance gain designated by the white balance gain input unit 53. . The defective pixel correction processing 45 performs defective pixel correction on the CFA image data read from the raw file based on the selected defective pixel map. The white balance processing 46 multiplies the CFA image data after the defective pixel correction by a designated white balance gain.

  In each of the above embodiments, the defective pixel map is selected based on the white balance gain. However, the present invention is not limited to this, and the influence on the image quality of the defective pixel is not limited to the empty grid interpolation process (pixel interpolation process). ), The defective pixel map may be selected in accordance with the empty grid interpolation process.

For example, there are the following empty grid interpolation processes. In this processing, the correlation between the upper and lower G pixels and the correlation between the left and right G pixels are examined at the R and B pixel positions of the CFA image data described above, and the average of the higher correlation G pixels is determined as the G of the pixel position. G is interpolated as a pixel value, color differences C _R = (R−G) and C _B = (B−G) are calculated at the R pixel position and the B pixel position, respectively, and then C _R and C _B are interpolated. each is seeking _{G, C} R, and _{C B under, R = C R + G,} B = C B + from B of each pixel R, a method of determining G, and B.

  When such a vacancy interpolation process is performed, the white spot defect of G remains as it is, whereas the white spot defect of R and B pixels is blurred due to smoothing by interpolation, and this affects the image quality. Therefore, it is preferable to create the following defective pixel map.

For example, in the first defective pixel map described with reference to FIG. 10, in order to strictly detect the G pixel, a threshold value of “T _R ” = “K _G ” × “T _G ” = “T _B ” is set. A map in which defective pixels are detected. Here, “T _R ” is a threshold value for R pixel, “T _G ” is a threshold value for G pixel, and “T _B ” is a threshold value for B pixel. “K _R ” is a constant equal to or less than “1”.

The second defective pixel map, in order to detect severely G and B pixels, and set the "T _R" = "K _G" × "T _G" = thresholds as "K _G" × "T _B" And a map in which defective pixels are detected.

As the third defective pixel map, in order to strictly detect the R pixel and the G pixel, a threshold value of “K _G ” × “T _R ” = “K _G ” × “T _G” = “T _B” is set. And a map in which defective pixels are detected. Since the white point defect of the B pixel is difficult to see, the third defective pixel map has a threshold value of “T _R ” = “K _G ” × “T _G” = “T _B” to set the defective pixel. The detected map may be the same as the first defective pixel map.

  According to this, a plurality of defective pixel maps are created according to the empty grid interpolation process, and any one of the plurality of defective pixel maps is selected based on the white balance gain to perform defective pixel correction.

  In addition, a vacancy interpolation selection unit that selects any one of a plurality of vacancy interpolation processes may be provided in the Raw file developing device.

  In this case, for example, in the digital camera 31, a defective pixel level map 60 (see FIG. 13) generated based on the digital data of all the pixels at the time of shading shooting is stored in the storage unit 39 at the time of factory shipment, and shooting is performed. The defective pixel level map 60 is recorded in the RAW file together with the CFA image data every time. Therefore, as shown in FIG. 18, a defective pixel level map 60 is recorded together with CFA image data and the like in the RAW file created at this time.

  In the raw file developing device 65, as shown in FIG. 19, a defective pixel level selection unit 69 is provided. The defective pixel level selection unit 69 reads out the defective pixel level map 60 from the raw file, and detects a defective pixel corresponding to the empty lattice interpolation processing selected by the empty lattice interpolation processing selection unit 70 based on the defective pixel level map 60. To do. Thereafter, as described above, the defective pixel correction process 45 corrects the defective pixel based on the position information of the defective pixel.

  As shown in FIG. 20, the defective pixel level map 60 divides each pixel value into a range of seven stages according to the conditions for performing defective pixel correction, and divides each division into correction levels “0” to “6”. As shown in FIG. 21, the map is obtained by assigning a number representing a correction level to each pixel position.

  Here, for example, the correction level “0” represents a pixel that does not need to be corrected, and the correction level “1” is a case where a white spot defect is conspicuous in the pixel and a white spot defect is conspicuous. Represents a pixel that is preferably corrected. The correction level “2” represents a pixel that is preferably corrected when a white spot defect is conspicuous in the pixel. The correction level “3” represents a pixel that is preferably corrected when the white spot defect is not noticeable in the pixel and the subject image is easily noticeable. The correction levels “4” to “6” represent pixels that are preferably corrected even when white point defects are not noticeable in the pixel. Here, the subject image in which white point defects are conspicuous is a dark and flat subject image, that is, an image with little luminance change or color change.

  Therefore, the defective pixel level selection unit 69 selects a correction level suitable for the selected empty grid interpolation process from the defective pixel level map read from the Raw file, and the output value is included in the range of the selected correction level. The pixel position is detected as a defective pixel. In this case, as shown in FIG. 22, since it can be understood that there are defective pixel maps 1 to 7 in which position information of defective pixels is described for each correction level, the defective pixel level selection unit 69 includes a plurality of defective pixels. This corresponds to defective pixel information selection means for selecting one of the maps 1 to 7.

  By storing such a defective pixel level map, the defective pixel level map is based on the degree of influence of the defective pixel in the pixel interpolation method, white balance gain, color conversion processing, information on the subject in the image data, and the like. From the above, defective pixel maps 1 to 7 describing position information of defective pixels to be corrected can be selected. In this case, in addition to selecting one of the defective pixel maps 1 to 7, a plurality of defective pixel maps 5 to 7 may be selected, for example.

  In the above-described embodiment, the defective pixel map is selected according to the empty lattice (pixel) interpolation method, the white balance gain, the color conversion process, the information of the subject in the image data, and the like as the shooting conditions. For example, the defective pixel map may be selected based on the ISO sensitivity, the continuous shooting mode, or the like.

  When based on the ISO sensitivity, when a high sensitivity exceeding a predetermined threshold is set on the digital camera side, a defective pixel map is selected in which the pixel value of each color photographed in a light-shielded state is strictly determined, and less than the threshold When low sensitivity is set, it is preferable to select a defective pixel map in which the pixel value of each color is determined to be loose. Conversely, a defective pixel map determined loosely in the case of high sensitivity may be selected, and a defective pixel map determined strictly in the case of low sensitivity may be selected.

  In addition, when the continuous shooting mode is set, it is preferable to select a defective pixel map that is determined to be looser than the single shooting mode.

  In each of the above-described embodiments, correction of white spot defects has been described, but the present invention can also be adopted for correction of black spot defects. When correcting the black spot defect, the aperture and the shutter are opened to a predetermined amount, and an image is captured under a predetermined exposure condition. The CFA image data obtained by this imaging is used for each color according to the imaging condition. What is necessary is just to produce | generate several defective pixel information which detected the black spot defect based on the threshold value. In this case, the threshold value set for each color may be set to a value lower than the pixel value in the normal range of the pixel values, and a pixel having a value less than the threshold value may be detected as a black spot defect.

  In the above embodiments, various embodiments and modifications have been described. However, the present invention is not limited to these contents. Other embodiments conceivable within the scope of the technical idea of the present invention are also included in the scope of the present invention.

10, 31 Digital camera 17, 35 White balance gain calculation circuit 18, 36 Defective pixel map selection circuit 40, 50, 65 Raw file development device

Claims (8)

In a defective pixel correction device that corrects a defective pixel detected based on color image data obtained from an image sensor when photographing at a predetermined brightness,
Shooting condition setting means for setting shooting conditions;
Defective pixel information storage means for storing a plurality of pieces of defective pixel information recorded for each color constituting position information of the defective pixels in the color image data;
Selecting means for selecting any defective pixel information from the plurality of defective pixel information according to the set photographing condition;
Defective pixel correction means for correcting defective pixels based on position information of defective pixels recorded in the selected defective pixel information;
A defective pixel correction apparatus comprising: The defective pixel correction device according to claim 1,
The defective pixel correction apparatus according to claim 1, wherein the photographing condition is a white balance gain or an ISO sensitivity. The defective pixel correction device according to claim 1 or 2,
The defective pixel correction device, wherein the defective pixel information is recorded with positional information of defective pixels detected based on a threshold value determined for each color according to the photographing condition. The defective pixel correction device according to any one of claims 1 to 3,
The color image data is
A plurality of pixels that output color information of the first color component having the highest spatial frequency, and a second color component or third color that does not output color information of the first color component and has a spatial frequency lower than that of the first color component. A defective pixel correction apparatus, wherein a plurality of pixels that output color information of color components are image data generated by an image sensor that is two-dimensionally arranged. The defective pixel correction device according to claim 4,
The shooting condition is white balance,
For color image data obtained from an image sensor when photographing at a predetermined brightness, the first defective pixel information detected using a uniform threshold value for each color and the color information of the second color component for other colors Second defective pixel information detected using a threshold that is determined more strictly than information, and a third defective pixel map detected using a threshold that determines the color information of the third color component more strictly than other color information. While pre-stored in the defective pixel information storage means,
A white balance gain calculating circuit for calculating two white balance gains according to the color information of the second and third color components based on color image data obtained for each shooting;
The defective pixel correction apparatus, wherein the selection unit selects one of the first to third defective pixel information based on the two white balance gains. In a defective pixel correction method for correcting color image data based on defective pixel information for each color,
A shooting condition setting step for setting shooting conditions;
A defective pixel information selection step for selecting any one of a plurality of defective pixel information recorded for each color constituting the color image data, according to the set photographing condition; ,
A defective pixel correction step of correcting the defective pixel based on the position information of the defective pixel recorded in the selected defective pixel information;
A defective pixel correction method comprising:   A computer-readable program for executing the defective pixel correction method according to claim 6.   A recording medium on which the computer-readable program according to claim 7 is recorded.

Images