JP2010193112A - Image processing apparatus and digital still camera - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To prevent a mixture ratio of signal levels of R, G, B from subtly changing to change hue in correcting brightness of a captured image, in an image processing apparatus for processing a digital image signal of a color captured image obtained by shooting. <P>SOLUTION: This image processing apparatus includes: a YUV generation part 121 for generating, from a digital image signal of a color captured image obtained by shooting, respective signals such as a luminance signal representing luminance of the color captured image, a first difference signal representing the difference between the luminance signal and a blue component of the color captured image, and a second difference signal representing the difference between the luminance signal and a red component of the color captured image; and a brightness correction part 100b for correcting respective signal levels of the luminance signal and the first and second difference signals to make a dark part of the color captured image bright. <P>COPYRIGHT: (C)2010,JPO&INPIT

Description

  The present invention relates to an image processing apparatus and a digital still camera, and more particularly to an image processing apparatus for correcting the brightness of a dark part of an image at the time of backlight photographing and a digital still camera including such an image processing apparatus. .

  Conventionally, there are digital still cameras using a solid-state imaging device such as a CCD image sensor or a CMOS image sensor.

  In such a digital still camera, image data is captured from a CCD module, which is an image sensor, and a color space from RGB values (each color signal) to YUV values (luminance, color difference signals) of each pixel in the Bayer array in the image data. Subject to image processing such as conversion, auto iris (output value adjustment according to light and darkness), white balance (color adjustment), and other image signal processing such as image compression to JPEG format. Is being taken.

  FIG. 2 is a schematic configuration diagram for explaining an image processing apparatus having a conventional gradation correction function used in a digital still camera.

  A digital still camera (imaging device) 200 shown in FIG. 2 receives a signal from a solid-state image sensor 201 such as a CCD image sensor or a CMOS image sensor and image data (output signal) output as pixel data from the solid-state image sensor 201. A sample hold unit 202 that extracts and samples and holds components, an AD converter 203 that converts the signal components of each pixel data sampled and held by the sample hold unit 202 into digital data, and digital data output from the AD converter An image processing apparatus 200a that performs image processing and has a gradation correction function.

  Here, as described above, the image sensor includes a CCD image sensor and a CMOS image sensor. For example, a CCD image sensor (CCD module) includes a pixel array in which pixels are arranged in a matrix, and the pixel array. A vertical CCD that is arranged along each pixel row and transfers the signal charge obtained in each pixel in the vertical direction, and a horizontal CCD that is arranged on one end side of the vertical CCD and transfers the signal charge from the vertical CCD in the horizontal direction. Each pixel is provided with a color filter of R (red), G (green) or B (blue). In the following description, pixels having R, G, and B color filters are also referred to as R pixels, G pixels, and B pixels, respectively.

  The image processing apparatus 200a fixes a black level of each pixel data converted into digital data by an AD converter to a specified value, and R pixel and B pixel data from the clamp processing unit 211. White that adjusts the gain of the pixel data so that white adjustment is performed by multiplying a predetermined value (that is, the pixel data of each of the R, G, and B pixels becomes equal when a white subject is imaged) Brightness correction is performed on the pixel data of the R, G, and B pixels output from the balance processing unit 212 and the white balance processing unit 212 for each display device used as a specific monitor device such as a cathode ray tube or a liquid crystal panel. And a gamma (γ) correction unit 213 that adjusts gradation display characteristics.

  In addition, the image processing apparatus 200a has a low-pass filter unit 214 that attenuates high-frequency components of pixel data from the gamma correction unit 213 and each pixel so that aliasing due to subsequent reduction processing (such as thinning-out processing) is suppressed. An interpolation processing unit 215 that obtains pixel data of a color other than the filter color of the pixel (hereinafter also referred to as image data) by interpolation, and a reduction process that performs a reduction process of thinning out image data corresponding to each color (RGB) From the RGB image data output from the unit 216 and the reduction processing unit 216, luminance signal data (Y) of the captured image, and difference signal data (U) between the luminance signal data and the blue component of the captured image (first) Difference signal) and luminance signal data, and a YUV generation unit 221 that generates difference signal data (V) (second difference signal) of the red component of the captured image. There.

  Further, the image processing unit 200a, based on the signal level for each of R, G, and B extracted from the input digital image signal, for each of a plurality of level ranges obtained by dividing the dynamic range of the signal level, A histogram generation unit 222 that detects the frequency of appearance of pixels having signal levels within each level range and generates a histogram represented by the frequency of the signal level for each range, and the gradation of the signal level from the shape of the generated histogram A gradation correction determination unit 223 for determining whether correction is necessary. The gamma (γ) correction unit 213, the histogram generation unit 222, and the gradation correction determination unit 223 constitute a brightness correction unit 200b.

  Next, the operation will be described.

  In the digital still camera 200, pixels are arranged in a row direction and a column direction, and image data is obtained by imaging a subject using an image sensor or the like having a color filter.

  When this image data is input to the sample hold unit 202, a signal component of the image data is extracted and held, and this signal component is further converted into digital data (digital value) by the AD converter 203 to be image processing apparatus 200a. Is input. In the image processing apparatus 200a, the digital data is subjected to digital clamp processing, white balance processing, generation processing of luminance signal data and color difference signal data, and the generated luminance signal data and color difference signal data are stored in an external memory. (Not shown).

  Next, a detailed operation of the image processing apparatus 200a will be described.

  The clamp processing unit 211 fixes the black level of each pixel data (digital data) converted into digital data by the AD converter to a specified value. The white balance processing unit 212 adjusts the gain by multiplying the R and B pixel data from the clamp processing unit 211 by a predetermined value in order to adjust the white color.

  Further, the gamma (γ) correction unit 213 performs processing for correcting the brightness on the image data from the white balance processing unit 212. This brightness correction processing is performed to adjust the gradation display characteristics for each display device used as a specific monitor device such as a cathode ray tube or a liquid crystal panel. The image data on which the gamma correction has been performed passes through the low-pass filter unit 214, so that the high-frequency component is attenuated. The attenuation of the high frequency component is performed in order to prevent aliasing due to a subsequent reduction process (such as a thinning process).

  Further, the interpolation processing unit 215 obtains, for each pixel, color components other than the color filter color of the pixel of the image data from the low-pass filter 214 by interpolation. That is, for example, in a pixel (G pixel) provided with a green filter, image data of a pixel (G pixel) provided with a red filter and a pixel (B pixel) provided with a blue filter, that is, red and blue images. Since information (pixel value) cannot be obtained, the interpolation processing unit 215 obtains the red and blue pixel value information for the G pixel from the pixel values of the surrounding R and B pixels. Similarly, for other B pixels and R pixels, pixel values of pixels of other colors are obtained by interpolation.

  Further, the reduction processing unit 216 performs a thinning process on the RGB color image data (hereinafter also referred to as RGB image data) from the interpolation processing unit 215 to reduce the image size.

  Further, the YUV generation unit 221 generates YUV image data from the RGB image data output from the reduction processing unit 216. This YUV image data includes luminance signal data (Y), difference signal data (U) between the luminance signal and the blue component, and difference signal data (V) between the luminance signal and the red component.

  Further, the RGB image data output from the white balance processing unit 212 is input to the histogram generation unit 222, and the histogram generation unit 222 outputs image data for each of R, G, and B extracted from the input image data. Based on the signal level, for each of a plurality of level ranges obtained by dividing the entire range that the signal level can take, the frequency of appearance of pixels having a signal level within the level range is detected. A histogram representing how much exists is generated.

  The gradation correction determination unit 223 determines whether gradation correction is necessary from the shape of the generated histogram. When it is determined that the tone correction is necessary, the gamma correction unit 213 performs tone correction together with the gamma correction.

  Next, a backlight correction method using gradation correction will be described.

  As described above, in a digital still camera, a captured image is obtained by imaging using an image sensor (for example, a CCD module) in which pixels are arranged in a row direction and a column direction and each has a color filter.

  However, since the dynamic range of the image sensor is not so large, it is difficult to capture both images with appropriate exposure when the brightness difference between the bright and dark areas included in the captured image is large, such as during backlighting. .

  For example, as shown in FIG. 3, when photographing in a room with a window, in the entire captured image V in which the subject T is in the room and the outside scenery is reflected through the window, the bright outside scenery B that can be seen through the window. If the exposure is adjusted to, the captured image of the subject T becomes dark without properly adjusting the exposure for the subject. An image processing method for brightly correcting the captured image of the subject T that has become dark in this way is called backlight correction, and has been conventionally performed by the following method.

  In order to correct the backlight image, it is first necessary to determine whether or not the entire captured image V is a backlight image.

  As a technique for making this determination, a technique using a histogram has been proposed.

  In the backlight image, a saturated portion of brightness (scenery seen from the window of FIG. 3) B and a dark portion (subject of FIG. 3) T occupy a large region of the entire image V, and the area of the region where the brightness is intermediate It is characterized by being narrow. For this reason, as shown in FIG. 4, the histogram of the backlight image has a high mountain in the high luminance region (saturation region) Rs and the low region Rd, and there are low frequency portions R4 and R5 between them. Become a shape. Therefore, the backlight image is determined depending on whether the frequency is high in the two regions R2 and R6, and when it is determined that the backlight image is true, the backlight is illuminated by the following method. Corrections have been made. Note that the regions R1, R3, R4, and R5 in the histogram of FIG. 4 are regions that are less frequent than the two regions R2 and R6.

  In the backlight image, it is necessary to correct the brightness of the captured image of the subject T photographed dark to an appropriate brightness. This brightness correction corresponds to increasing the luminance of the low-luminance region R2 in the histogram by gradation correction. Conventionally, the gamma correction unit 213 performs gradation display for each display device used as a monitor device. In general, the brightness of the backlight is corrected at the same time as the brightness of the characteristic.

  However, if the brightness due to backlight is also corrected at the time of gamma correction, the color reproducibility changes slightly, such as the skin color of the subject being bluish, and therefore the hue may be different. This is due to the non-linearity of brightness correction for adjusting the gradation display characteristics for each display device performed by the gamma correction unit 213.

  FIG. 5 shows a gamma correction curve (hereinafter also referred to as a gamma correction curve) used in the gamma correction unit 213. In the gamma correction unit 213, brightness correction is performed for each color of R (red), G (green), and B (blue). A gamma correction curve for adjusting gradation display characteristics corresponding to a display device is normally used. As shown in FIG.

  The solid curve La shown in FIG. 5 is a normal gamma correction curve, the dotted curve Lb is a gamma correction curve including brightness correction, and the gamma correction curve including brightness correction is compared with the normal gamma correction curve. Thus, as shown in FIG. 5, the brightness correction amount is set to be different by ΔY at the maximum.

  For example, R (red), G (green), and B (blue) image data before gamma correction have RGB values (pixel values of each color) as R0, G0, and B0, respectively, and a normal gamma correction curve (curve La) The RGB values corrected by the above are R1, G1, and B1, respectively, and the RGB values corrected by the gamma correction curve (curve Lb) including the brightness correction are R2, G2, and B2, respectively.

  At this time, due to the non-linearity of the gamma correction curve, the mixture ratio of RGB values R1, G1, and B1 corrected using the normal gamma correction curve (curve La) and the gamma correction curve (curve Lb) including brightness correction are used. The mixture ratio of the RGB values R2, G2, and B2 corrected by using S is slightly different, and the hue of the captured image changes between when the backlight correction is not performed and when the backlight correction is performed. This caused the skin color of the subject to be bluish when the backlight correction was performed.

  In the conventional technique for correcting the brightness of a captured image taken with a digital still camera in this way, a technique for correcting the brightness with a gamma correction curve or a lookup table at the time of backlight correction has been proposed. A correction method that supports even a slight change in hue after correction is not supported (for example, Patent Document 1).

JP 2006-5014 A

  As described above, in the conventional imaging technology, when correcting the brightness of a subject photographed dark in a backlight image to an appropriate brightness, it is common to correct the brightness by a gamma correction unit. there were.

  However, if the brightness due to backlight is also corrected at the time of gamma correction, the color reproducibility changes slightly, and the hue may be different, for example, the subject's skin color may be bluish. This is due to the non-linearity of brightness correction performed by the gamma correction unit for adjusting the gradation display characteristics for each display device.

  In other words, the gamma correction unit performs brightness correction for each color of R (red), G (green), and B (blue), but the gamma correction curve for adjusting the gradation display characteristics corresponding to the display device is usually As shown in FIG. 5, the mixture ratio of R, G, and B pixel values after correction using this correction curve is slightly different from the mixture ratio before correction, and this is a change in hue. Will appear.

  As described above, conventionally, in the field of digital still cameras and the like, a method of performing brightness correction using a gamma correction curve or a lookup table at the time of backlight correction has been proposed. There is a problem that the subtle hue change causes image quality degradation.

  The present invention has been made to solve the above-described problems. Brightness correction is performed by slightly changing the hue of the mixture ratio of the pixel data of each of the R, G, and B pixels. An object of the present invention is to obtain an image processing apparatus that can be performed without causing image quality degradation.

  An image processing apparatus according to the present invention is an image processing apparatus that processes a digital image signal of a color captured image obtained by imaging, and from the digital image signal, a luminance signal indicating the luminance of the color captured image, the luminance Signal generation for generating a first difference signal indicating a difference between a signal and a blue component of the color captured image, and a second difference signal indicating a difference between the luminance signal and a red component of the color captured image And a brightness correction unit that corrects the signal level of each of the luminance signal and the first and second difference signals so that a dark part in the color captured image is brightened. The objective is achieved.

  In the image processing apparatus according to the aspect of the invention, the brightness correction unit may determine how much luminance the color captured image has based on a luminance value that is a signal level of a luminance signal of each pixel constituting the color captured image. A brightness information creation unit that creates brightness information indicating how many pixels are included, and a correction necessity determination that determines whether or not brightness correction is necessary for the color captured image based on the brightness information It is preferable that the signal level of each of the luminance signal and the first and second difference signals is corrected according to the determination result in the correction necessity determination unit.

  The present invention is the above image processing apparatus, wherein the brightness information creation unit is a histogram generation unit that generates a histogram representing the frequency of pixels included in each of a plurality of preset luminance intervals, and whether or not the correction is necessary. The determination unit is preferably configured to determine whether or not brightness correction is necessary for the color captured image from the shape of the histogram.

  In the image processing apparatus according to the aspect of the invention, the brightness correction unit may include a correction coefficient calculation unit that calculates a correction coefficient indicating a degree of brightness correction for the color captured image based on the shape of the histogram. preferable.

  In the image processing apparatus according to the aspect of the invention, the correction necessity determination unit may be configured such that the shape of the histogram includes a high luminance region and a low luminance region, and an intermediate luminance region therebetween. It is preferable to determine that it is necessary to correct the brightness of the color captured image when the shape has a high frequency.

  In the image processing apparatus according to the aspect of the invention, when the correction necessity determination unit determines that brightness correction is necessary for the color captured image, the frequency in the low luminance region is greater than a preset threshold value. It is preferable that the power is large.

  In the image processing apparatus according to the aspect of the invention, the correction coefficient calculation unit indicates the degree of brightness correction for the color captured image as the difference between the frequency in the high-luminance region and a preset threshold value increases. As the correction coefficient, it is preferable to calculate a correction coefficient that further increases brightness.

  In the image processing apparatus according to the aspect of the invention, the correction coefficient calculator may include a first difference between the frequency in the high luminance region and a preset first threshold, and the frequency in the low luminance region. It is preferable to calculate a correction coefficient indicating a degree of brightness correction for the color captured image based on a second difference from a preset second threshold.

  In the image processing apparatus according to the aspect of the invention, it is preferable that the correction coefficient calculation unit includes a register that sets the first and second threshold values.

  In the image processing apparatus according to the aspect of the invention, the brightness correction unit may perform tone correction on the signal level of each of the luminance signal and the first and second difference signals based on the correction coefficient. It is preferable to have.

  According to the present invention, in the image processing apparatus, the gradation correction unit includes a function storage unit that stores a conversion function that defines a correspondence relationship between the luminance value and the gain, and a gain based on the conversion function from the luminance value. And a multiplier for calculating a correction gain by multiplying the calculated gain by the correction coefficient, the correction gain being used as the luminance signal and the first and second difference signals. It is preferable to perform gradation correction of these signals by multiplying each signal level.

  In the image processing apparatus according to the aspect of the invention, the conversion function may include a gain for a pixel value of a pixel within a certain luminance range including a luminance value of a pixel constituting a dark portion in the color captured image. It is preferable that it is defined so as to be larger than the gain for the luminance value of the pixel.

  According to the present invention, in the image processing apparatus, the digital image signal is an analog image signal of a color captured image obtained by imaging with a solid-state imaging device in which a plurality of pixels are arranged and each has a corresponding color filter. The signal generation unit performs an interpolation process for generating pixel data of a color other than the color of the color filter of the pixel based on the digital image signal. It is preferable that a processing unit is included and an image signal corresponding to the color of each color filter output from the interpolation processing unit is output to the signal generation unit as the digital image signal.

  In the image processing apparatus according to the aspect of the invention, it is preferable that the signal generation unit includes a gamma correction unit that is provided before the interpolation processing unit and performs gamma correction processing on the digital image signal.

  According to the present invention, in the image processing apparatus, the signal generation unit is provided in a preceding stage of the gamma correction unit, and the digital image signal in a white captured image is converted into the color of each color filter included in the digital image signal. It is preferable to have a white balance processing unit that corrects the level of the image signal corresponding to.

  A digital still camera according to the present invention is a digital still camera including a solid-state image pickup device that picks up an image of a subject, and samples and holds an output signal of the solid-state image pickup device to output a signal component of a color picked-up image A holding unit; an AD converter that AD-converts the signal component and outputs a digital image signal; and an image processing device that processes the digital image signal. The image processing device includes the image according to the present invention described above. A processing device, whereby the above object is achieved.

  In the digital still camera according to the aspect of the invention, it is preferable that the solid-state imaging device is a CCD image sensor.

  According to the present invention, in the digital still camera, the solid-state imaging device is preferably a CMOS image sensor.

  Next, the operation will be described.

  In the present invention, from a digital image signal of a color captured image obtained by imaging, a luminance signal indicating the luminance of the color captured image, and a first difference indicating a difference between the luminance signal and the blue component of the color captured image. A signal generation unit that generates a signal and a second difference signal indicating a difference between the luminance signal and a red component of the color captured image, and the luminance signal is set so that a dark portion in the color captured image is brightened. Since the signal level of each of the first and second difference signals is corrected, the brightness correction of the captured image is performed with the mixture ratio of the pixel data of each pixel of red (R), green (G), and blue (B). Can be performed without causing image quality degradation such as a slight change in hue and a change in hue.

  In the present invention, based on the shape of a histogram representing the frequency of pixels included in each of a plurality of preset luminance intervals, a correction coefficient indicating the degree of brightness correction for the color captured image is calculated, Since the gain applied to the signal level of each of the luminance signal and the first and second difference signals is adjusted by the correction coefficient, it is automatically and continuously gained by the correction coefficient according to the backlight condition of the captured image. Can be suppressed.

  Furthermore, in the present invention, gamma correction and gradation correction by backlight can be adjusted by separate circuits, and each can be set independently.

  Further, in the present invention, when the shape of the histogram is a shape having a higher frequency in a high luminance region and a low luminance region than in an intermediate luminance region therebetween, Since it is determined that the brightness correction for the color captured image is necessary, it is possible to easily determine whether the brightness correction is necessary based on the shape of the histogram.

  As described above, according to the present invention, in an image processing apparatus that processes a digital image signal of a color captured image obtained by imaging, from the digital image signal, a luminance signal indicating the luminance of the color captured image, the luminance Signal generation for generating a first difference signal indicating a difference between a signal and a blue component of the color captured image, and a second difference signal indicating a difference between the luminance signal and a red component of the color captured image And a brightness correction unit that corrects the signal level of each of the luminance signal and the first and second difference signals so that a dark part in the color captured image becomes brighter. There is an effect that can be performed without causing deterioration in image quality such that the mixing ratio of the pixel data of the R, G, and B pixels slightly changes and the hue changes.

  According to the present invention, the correction coefficient indicating the degree of brightness correction for the color captured image is calculated based on the shape of the histogram representing the frequency of the pixels included in each of the plurality of preset luminance intervals. Since the gain applied to the signal level of each of the luminance signal and the first and second difference signals is adjusted by the correction coefficient, the correction coefficient automatically and continuously according to the backlight condition of the captured image. There is an effect that the gain can be suppressed.

FIG. 1 is a schematic configuration diagram illustrating an image processing apparatus having a gradation correction function according to Embodiment 1 of the present invention, and shows a digital still camera including the image processing apparatus. FIG. 2 is a schematic configuration diagram showing an image processing apparatus having a conventional gradation correction function, and shows a digital still camera provided with the image processing apparatus. FIG. 3 is a diagram for explaining backlight correction in a conventional image processing apparatus, and shows a photographed image at the time of backlight. FIG. 4 is a diagram for explaining backlight correction in a conventional image processing apparatus, and is a histogram showing the frequency distribution of pixel values during backlight generated from RGB values. FIG. 5 is a diagram for explaining backlight correction in a conventional image processing apparatus, and is used for performing a gamma correction curve used when normal gamma correction is performed on RGB values, and when performing brightness correction during the gamma correction. A comparison with a gamma correction curve is shown. FIG. 6 is a diagram for explaining gradation correction in the image processing apparatus according to the first embodiment of the present invention, and shows a histogram at the time of backlight generated from the luminance value. FIG. 7 is a diagram for explaining gradation correction in the image processing apparatus according to the first embodiment of the present invention, and shows an example of a conversion curve (gain curve) for converting a luminance value used in the gradation correction into a gain. . FIG. 8 is a diagram for explaining the image processing apparatus according to the first embodiment of the present invention. A YUV gradation correction unit (FIG. 8A) and a gradation correction coefficient calculation unit (FIG. 8) constituting the image processing apparatus. The circuit configuration of (b)) is shown.

  An image processing system that corrects a dark portion of a captured image, which is an object of the present invention, is used for a digital still camera, for example, and realizes desired data signal processing by combining a series of image signal processing. .

  Hereinafter, embodiments of the present invention will be described with reference to the drawings.

(Embodiment 1)
FIG. 1 is a diagram for explaining a main configuration of an image processing apparatus according to Embodiment 1 of the present invention, and shows a digital camera equipped with the image processing apparatus.

  A digital still camera 100 shown in FIG. 1 takes out signal components from, for example, a solid-state image pickup device 101 that is an image sensor and image data output as pixel data from the solid-state image pickup device 101, as with the conventional digital still camera 200. A sample-and-hold unit 102 for sample-holding, an AD converter 103 for converting the signal component of each pixel data sampled and held by the sample-and-hold unit 102 into digital data, and image processing for the digital data output from the AD converter 103 And an image processing apparatus 100a having a backlight correction processing function.

  The image sensor includes a CCD type image sensor and a CMOS type image sensor. Here, a CCD module is used, and this CCD module is the same as that of the conventional digital still camera shown in FIG.

  Similar to the conventional image processing apparatus 200a, the image processing apparatus 100a includes a clamp processing unit 111 that fixes a black level for each pixel data converted into digital data by an AD converter to a predetermined value, and an R from the clamp processing unit 111. The white balance processing unit 112 that multiplies pixel data of the pixels and B pixels by a predetermined value to adjust the gain so that white adjustment is performed, and the brightness for the RGB pixel data output from the white balance processing unit 112 And a gamma (γ) correction unit 113 that adjusts the gradation display characteristics of each display device used as a specific monitor device such as a cathode ray tube or a liquid crystal panel.

  Further, the image processing apparatus 100a applies a low-pass filter unit 114 that attenuates a high-frequency component of image data from the gamma correction unit 113 and each pixel so that aliasing due to subsequent reduction processing (such as thinning-out processing) is suppressed. An interpolation processing unit 115 that obtains a color component other than the color filter color of the pixel by interpolation, and a reduction processing unit 116 that performs a reduction process of thinning out image data (RGB image data) corresponding to each color of RGB from the interpolation processing unit 115. And luminance signal data (Y), difference signal data (U) between the luminance signal and the blue component, and difference signal data (V) between the luminance signal and the red component from the RGB image data output from the reduction processing unit 216. And a YUV generation unit 121 for generation.

  The image processing apparatus 100 a according to the first embodiment includes a brightness correction unit 100 b that performs brightness correction on the signal data output from the YUV generation unit 121.

  The brightness correction unit 100b includes a histogram generation unit 122 that generates a luminance histogram using the generated luminance signal data (Y), and a gradation correction coefficient calculation unit 140 that calculates a correction coefficient C based on the luminance histogram. And a YUV gradation correction unit 130 that performs brightness correction on pixels in a low luminance range using the correction coefficient C.

  Here, as shown in FIG. 8A, the YUV tone correction unit 130 stores a conversion function that defines the correspondence between the luminance signal data (luminance value) output from the YUV generation unit 121 and the gain. A function storage unit 131a, a conversion unit (gain derivation unit) 131 that converts the luminance value into a gain G based on the conversion function, and a correction gain k is calculated by multiplying the obtained gain G and correction coefficient C. The gain correction unit (multiplication unit) 132 that performs the correction and the correction gain k are multiplied by the luminance signal data (Y0), the first difference signal data (U0), and the second difference signal data (V0) before brightness adjustment. Multipliers 133a to 133c for calculating brightness signal data (Y1), first difference signal data (U1), and second difference signal data (V1) whose brightness has been adjusted.

  Here, as shown in FIG. 7, the conversion function has a gain for a pixel value of a pixel in a certain luminance range X including a luminance value of a pixel constituting a dark portion in the color captured image, except for the certain luminance range. It is defined to be larger than the gain for the luminance value of the pixel.

  Further, as shown in FIG. 8B, the gradation correction coefficient calculation unit 140 determines whether or not brightness correction is necessary for the color captured image based on the information H indicating the luminance histogram. 141, and a coefficient creation unit (correction coefficient) that calculates a correction coefficient C indicating the degree of brightness correction for the color captured image based on the determination result in the correction necessity determination unit 141 and the information H indicating the luminance histogram. Calculation unit) 142.

  Here, as shown in FIG. 6, the correction necessity determination unit 141 includes a region R4 having a luminance intermediate between a high luminance region R4 and a low luminance region R2, as shown in FIG. In contrast, when the shape has a higher frequency, it is determined that brightness correction is required for the color captured image. As a further determination condition, when the correction necessity determination unit 141 determines that brightness correction is necessary for the color captured image, the frequency in the low-luminance region R2 is set to a preset threshold value (HIST_DIF_L ) The condition that the frequency is larger than that is added. Here, the coefficient creating unit 142 indicates the degree of brightness correction for the color captured image as the difference ΔH between the frequency in the high-luminance region R4 and a preset threshold value (HIST_DIF_H) is larger. As the correction coefficient, a correction coefficient that further increases brightness is calculated.

  The coefficient creating unit 142 uses the first difference ΔH between the frequency (HIST_H) in the high-luminance region R4 and the first threshold (HIST_DIF_H) set in advance, and the frequency in the low-luminance region R2. Based on the second difference ΔL between (HIST_L) and a preset second threshold (HIST_DIF_L), a correction coefficient indicating the degree of brightness correction for the color captured image may be calculated. For example, the coefficient creating unit 142 may generate a correction coefficient that performs brightness correction so that the larger the integrated value of the first and second differences is, the brighter it is.

  In this case, it is desirable that the coefficient creating unit 142 includes a register for setting the first and second threshold values.

  Next, the operation will be described.

  In the digital camera 100 according to the first embodiment, when image data obtained by imaging by the CCD module 101 that is an image sensor is input to the sample hold unit 102, the sample hold unit 102 generates a signal component from the image data. The signal is taken out and output to an AD converter, and the AD converter 103 converts the signal component into digital data.

  In the image processing apparatus 100a, the black level of the digital data from the AD converter 103 is fixed to a specified value by the clamp processing unit 111, and the R pixel and the B pixel from the clamp processing unit 111 are fixed by the white balance processing unit 112. The pixel data is multiplied by a predetermined value to adjust the gain in order to adjust the white color. The operations of the clamp processing unit 111 and the white balance processing unit 112 are the same as those in the conventional image processing apparatus 200a.

  In the first embodiment, the gamma (γ) correction unit 113 performs gamma correction processing on the image data from the white balance processing unit 212. This gamma correction processing is performed to adjust the gradation display characteristics of each display device used as a specific monitor device such as a cathode ray tube or a liquid crystal panel. The image data on which the gamma correction has been performed passes through the low-pass filter unit 114, so that the high-frequency component is attenuated. The attenuation of the high frequency component is performed in order to prevent aliasing due to a subsequent reduction process (such as a thinning process).

  Further, the interpolation processing unit 115 obtains, for each pixel, color components other than the color of the color filter of the pixel of the image data from the low-pass filter 114 by interpolation. That is, in the pixel (G pixel) provided with the green filter, the image data of the pixel (G pixel) provided with the red filter and the pixel (B pixel) provided with the blue filter, that is, red and blue image information Since it cannot be obtained, the interpolation processing unit 215 obtains the red and blue pixel value information at the G pixel from the pixel values of the surrounding R and B pixels.

  Further, the reduction processing unit 116 creates RGB image data whose image size is reduced by thinning out the RGB image data from the interpolation processing unit 115.

  The YUV generation unit 121 generates YUV image data from the RGB image data output from the reduction processing unit 116. The YUV image data includes luminance signal data (Y), difference signal data (U) between the luminance signal and the blue component, and difference signal data (V) between the luminance signal and the red component.

  Thus, the image data obtained from the CCD module as the image sensor is subjected to data processing by various circuits as shown in FIG. 1, and the YUV generation unit 121 at the front stage of the brightness correction unit 100 b performs luminance signal data. A difference (U) between the luminance signal and the blue component and a difference (V) between the luminance signal and the red component are generated, and these signals are input to the brightness correction unit 100b.

  Next, the brightness correction operation performed by the YUV tone correction unit 130 in the brightness correction unit 100b will be described.

  In the YUV tone correction unit 100b, the histogram generation unit 122 based on the luminance signal data (Y) from the YUV generation unit 121 indicates a luminance histogram indicating how many pixels exist (see FIG. 6). Next, the correction coefficient C necessary for the brightness correction is calculated by the gradation correction coefficient calculation unit 140, and further, the YUV gradation correction unit 130 converts each of the Y, U, and V signal data. The brightness adjustment is performed by the correction coefficient C calculated for the above.

  Hereinafter, an operation for calculating the correction gain based on the luminance histogram generated by the histogram generation unit 122 will be described.

  When a histogram of a captured image (backlight image) at the time of backlighting is generated, it is said that a portion that is too bright and a portion that is too dark occupy a large area of the entire captured image, and that the area of brightness is intermediate There are features.

  For this reason, in the luminance histogram of the backlight image, as shown in FIG. 6, there is a high frequency peak in the high luminance range R4 and a low luminance range R2, and there is a relatively low frequency region R3 between them. Become. The range R1 is a region having a lower luminance than the range R2.

  First, in the gradation correction coefficient calculation unit 140, the determination as to whether or not to perform brightness correction is performed by the correction necessity determination unit 141 based on the information of the histogram, and the coefficient generation unit 142 further performs the above-described determination. In addition, the correction coefficient C is calculated based on the correction necessity determination result and the shape of the histogram.

  Then, the YUV gradation correction unit 130 causes the conversion unit 131 to follow the function information stored in the function information storage unit 131a, that is, a gain curve (conversion function) Lg for converting luminance into gain as shown in FIG. Thus, the gain G corresponding to the pixel is calculated from the luminance value of each pixel. The gain curve Lg is set so that the gain G becomes larger in the vicinity of the low-luminance range R2, so that the gain G can be multiplied only by the luminance value of the low-luminance pixel. In addition, no gain is applied to the luminance value of the pixel.

  The gain G obtained by the conversion unit 131 is multiplied by the correction coefficient C created by the coefficient creation unit 142 by the gain correction unit 132 to generate a correction gain k.

  Hereinafter, a method for calculating the correction gain from the histogram will be further described.

  In the present invention, brightness correction is brightness correction during backlighting, and the luminance histogram during backlighting generated by the histogram generator 122 has a shape as shown in FIG.

  Therefore, in the present invention, the frequency of all luminance regions is not detected, but only the high range R4 and the low range R2 are set, the frequencies of these two ranges are detected, and the luminance levels for these luminance ranges are set. A histogram is generated according to the frequency of.

  Using this histogram, the gradation correction coefficient calculator 140 calculates a correction coefficient for brightness correction by the following method.

  That is, in the coefficient creating unit 142 of the gradation correction coefficient calculating unit 140, as shown in FIG. 6, the H threshold value (HIST_DIF_H) relating to the frequency is set in the above-described high luminance range R4 and low luminance range R2, respectively. ), L threshold value (HIST_DIF_L) is set, brightness correction is performed only when the frequency of each area exceeds the threshold value, and the difference between the threshold value and luminance frequency (HIST_L, HIST_H) ( A correction coefficient is created so that the lower the luminance range R2, the higher the value of [Delta] L, [Delta] H) (that is, the greater the values of "HIST_L-HIST_DIF_L" and "HIST_H-HIST_DIF_H").

For example,
HIST_SUP = {(HIST_L−HIST_DIF_L) × K_L} x {(HIST_H−HIST_DIF_H) × K_H}
A correction coefficient C that increases or decreases according to the value is calculated, and the brightness is corrected.

  Here, K_L is a sensitivity adjustment coefficient for the low brightness range R2, and K_H is a sensitivity adjustment coefficient for the high brightness range R4.

  As the value of HIST_SUP is increased, the correction coefficient C is increased, so that the captured image is corrected to be brighter. As the value is decreased, the brightness correction is suppressed, and the brightness of the captured image is not corrected. When the gain G of each pixel calculated from the above-described gain curve Lg is GAIN0 (see FIG. 7), for example, the brightness correction gain k of each pixel is calculated by the following calculation.

Correction gain k = (GAIN0−1) × HIST_SUP + 1
Using this correction gain k, the brightness of the dark part is corrected.

  By using this correction gain, the brightness can be corrected continuously.

  Next, brightness correction is performed on the pixels in the low-luminance range R2 using the correction gain k calculated by the method described above.

  The calculation (multiplication) of the luminance signal data (Y0), the difference between the luminance signal and the blue component (U0), the difference between the luminance signal and the red component (V0), and the correction gain k described above is performed for each calculator. By performing the processing at 133a to 133c, it is possible to generate Y1, U1, and V1 after brightness correction, and to brightly correct a low luminance area.

  By doing so, the mixing ratio of the R image data, the G image data, and the B image data does not change before and after the brightness correction, so that the color reproducibility can be maintained and the hue is not different. Only the brightness can be corrected.

  The reason why the hue does not differ by multiplying the Y value, U value, and V value by the correction gain as described above will be briefly described.

  The correction gain is k, the brightness value Y before brightness correction, the color difference value U, and the color difference value V are Y0, U0, and V0, and the brightness value Y, color difference value U, and color difference value V after brightness correction are Y1, U1. , V1, the relationship before and after brightness correction is expressed as follows.

  When Y1 = kY0, U1 = kU0, and V1 = kV0, the brightness value Y1 after brightness correction and the color difference values U1 and V1 are converted into R, G, and B, the relationship is developed as follows: .

R = 1.000Y1 + 1.402V1
= K (1.000Y0 + 1.402V0)
G = 1.000Y1-0.344U1-0.714V1
= K (1.000Y0-0.344U0-0.714V0)
B = 1.000Y1 + 1.772U1
= K (1.000Y0 + 1.772U0)
As can be seen from this equation, even when a gain k that varies depending on the luminance is applied to each of the Y, U, and V data, the mixing ratio of R, G, and B does not change, so the hue does not change.

  Thus, in the first embodiment, the following effects can be obtained.

  In the prior art, when correcting the brightness of a subject photographed dark in a backlight image to an appropriate brightness, the brightness is generally corrected by a gamma correction unit. However, if the brightness due to backlighting is also corrected at the time of gamma correction, the color reproducibility changes slightly, and the hue may be different, for example, the subject's skin color is bluish.

  On the other hand, in the embodiment of the present invention, brightness adjustment is not performed on the image data of each color of R, G, and B at the time of gamma correction but brightness signal data ( Y), a difference between the luminance signal and the blue component (U) and a difference between the luminance signal and the red component (V) are generated, and then a luminance histogram is generated for the luminance signal data (Y). Further, it is determined whether the backlight correction is necessary from the shape of the luminance histogram. If the correction is necessary, a correction gain k for brightness adjustment according to the frequency of the histogram is calculated, and the signal data (Y, U, V ) To adjust the brightness corresponding to the correction gain calculated.

  By adopting such a configuration, when the brightness of the captured image is corrected, the mixing ratio of the signal levels of R, G, and B does not change subtly, and the hue does not change. Further, the gain can be automatically and continuously suppressed by the correction coefficient C in accordance with the backlight condition, and therefore, blinking of light and dark does not occur. In this case, the gain is adjusted by simply multiplying the correction coefficient C by the gain, so that the correction gain can be obtained with a simple circuit configuration.

  Furthermore, gamma correction and gradation correction by backlight can be adjusted by separate circuits, and each can be set independently.

  Further, in the present embodiment, when the histogram has a shape in which the frequency is higher in the high luminance region and the low luminance region than in the intermediate luminance region therebetween, the color Since it is determined that brightness correction is necessary for the captured image, it is possible to easily determine whether brightness correction is necessary based on the shape of the histogram.

  As mentioned above, although this invention has been illustrated using preferable embodiment of this invention, this invention should not be limited and limited to this embodiment. It is understood that the scope of the present invention should be construed only by the claims. It is understood that those skilled in the art can implement an equivalent range based on the description of the present invention and the common general technical knowledge from the description of specific preferred embodiments of the present invention. Patents, patent applications, and documents cited herein should be incorporated by reference in their entirety, as if the contents themselves were specifically described herein. Understood.

  In the field of an image processing apparatus that performs brightness correction of a dark part of an image at the time of backlight photographing and the like and a digital still camera including such an image processing apparatus, brightness correction is performed for each of R, G, and B. This can be performed without causing deterioration in image quality, such as a slight change in the pixel data mixing ratio and a change in hue, and continuous brightness correction can be performed for each pixel in the dark area. It is useful for a digital camera equipped with a solid-state image sensor such as a CCD image sensor or a CMOS image sensor.

DESCRIPTION OF SYMBOLS 100 Digital still camera 100a Image processing apparatus 100b Brightness correction part 101 Solid-state image sensor 102 Sample hold part 103 AD converter 111 Clamp process part 112 White balance process part 113 Gamma ((gamma)) correction part 114 Low pass filter part 115 Interpolation process part 116 Reduction | decrease Processing unit 121 YUV generation unit 122 Histogram generation unit 130 YUV tone correction unit 140 Tone correction coefficient calculation unit 131 Conversion unit (gain derivation unit)
131a Function storage unit 132 Gain correction unit (multiplication unit)
133a to 133c Multiplier 140 Gradation correction coefficient calculation unit 141 Correction necessity determination unit 142 Coefficient creation unit (correction coefficient calculation unit)

Claims (18)

An image processing apparatus that processes a digital image signal of a color image obtained by imaging,
From the digital image signal, a luminance signal indicating the luminance of the color captured image, a first difference signal indicating a difference between the luminance signal and the blue component of the color captured image, and a red color of the luminance signal and the color captured image A signal generation unit that generates each signal of the second difference signal indicating a difference from the component;
An image processing apparatus comprising: a brightness correction unit that corrects the signal level of each of the luminance signal and the first and second difference signals so that a dark part in the color captured image becomes bright. The image processing apparatus according to claim 1.
The brightness correction unit
Brightness that creates brightness information indicating how much pixels are included in the color captured image based on a brightness value that is a signal level of a brightness signal of each pixel constituting the color captured image An information creation unit;
A correction necessity determination unit that determines whether or not brightness correction is necessary for the color captured image based on the brightness information;
An image processing apparatus that corrects the signal level of each of the luminance signal and the first and second difference signals in accordance with a determination result in the correction necessity determination unit. The image processing apparatus according to claim 2,
The brightness information creation unit
A histogram generation unit that generates a histogram indicating the frequency of pixels included in each of a plurality of preset luminance intervals;
The correction necessity determination unit
An image processing apparatus configured to determine whether or not brightness correction is necessary for the color captured image from the shape of the histogram. The image processing apparatus according to claim 3.
The brightness correction unit
An image processing apparatus comprising: a correction coefficient calculation unit that calculates a correction coefficient indicating a degree of brightness correction for the color captured image based on the shape of the histogram. The image processing apparatus according to claim 4.
The correction necessity determination unit
Brightness correction for the color-captured image when the histogram has a shape in which the frequency is higher in the high luminance region and the low luminance region than in the intermediate luminance region between them. An image processing apparatus that determines that is necessary. The image processing apparatus according to claim 5.
When the correction necessity determination unit determines that brightness correction is required for the color captured image, the frequency in the low luminance region is a frequency greater than a preset threshold. . The image processing apparatus according to claim 5.
The correction coefficient calculator is
Image processing for calculating a correction coefficient for increasing brightness as a correction coefficient indicating the degree of brightness correction for the color captured image as the difference between the frequency in the high-luminance region and a preset threshold value increases. apparatus. The image processing apparatus according to claim 5.
The correction coefficient calculator is
Based on a first difference between the frequency in the high brightness area and a preset first threshold value, and a second difference between the frequency in the low brightness area and a preset second threshold value An image processing apparatus that calculates a correction coefficient indicating a degree of brightness correction for the color captured image. The image processing apparatus according to claim 8.
The correction coefficient calculator is
An image processing apparatus having a register for setting the first and second threshold values. The image processing apparatus according to claim 4.
The brightness correction unit
An image processing apparatus having a gradation correction unit that performs gradation correction on the signal level of each of the luminance signal and the first and second difference signals based on the correction coefficient. The image processing apparatus according to claim 10.
The gradation correction unit
A function storage unit that stores a conversion function that defines the correspondence between the luminance value and the gain;
A gain deriving unit for deriving a gain from the luminance value based on the conversion function;
A multiplier that calculates a correction gain by multiplying the calculated gain and the correction coefficient;
An image processing apparatus that performs gradation correction of these signals by multiplying the signal level of each of the luminance signal and the first and second difference signals by the correction gain. The image processing apparatus according to claim 11.
In the conversion function, the gain for the pixel value of a pixel within a certain luminance range including the luminance value of the pixel constituting the dark portion in the color captured image is larger than the gain for the luminance value of a pixel outside the certain luminance range. An image processing apparatus defined as follows. The image processing apparatus according to claim 1.
The digital image signal is obtained by AD conversion of an analog image signal of a color image obtained by imaging with a solid-state imaging device having a plurality of pixels arranged and corresponding color filters,
The signal generator is
Each color output from the interpolation processing unit has an interpolation processing unit for performing interpolation processing for creating pixel data of a color other than the color filter color of the pixel based on the digital image signal An image processing apparatus configured to output an image signal corresponding to a color of a filter to the signal generation unit as the digital image signal. The image processing apparatus according to claim 13.
The image processing apparatus includes a gamma correction unit that is provided before the interpolation processing unit and that performs gamma correction processing on the digital image signal. The image processing apparatus according to claim 14.
The signal generation unit is provided before the gamma correction unit, and the level of the image signal corresponding to the color of each color filter included in the digital image signal in the white captured image is equal to the digital image signal. An image processing apparatus having a white balance processing unit that corrects the image. A digital still camera equipped with a solid-state imaging device for imaging a subject,
Sample and hold the output signal of the solid-state image sensor, and output a signal component of a color captured image; and
An AD converter that AD converts the signal component and outputs a digital image signal;
2. A digital camera having the image processing apparatus according to claim 1, which processes the digital image signal. The digital still camera according to claim 16, wherein
The solid-state imaging device is a digital still camera which is a CCD image sensor. The digital still camera according to claim 16, wherein
The solid-state imaging device is a digital still camera which is a CMOS image sensor.

Images